NPS (7) – 7th in the series – Transactional / Episode surveys – Best avoided? Use very selectively!

Transactional surveys are by far the most common type of customer research. It is helpful to think of them as feedback about ‘episodes’ or ‘jobs to be done’. Customers interact with you to try to accomplish some individual thing as part of their overall experience with you. An overall season of a TV series, for example, consists of a series of episodes. A customer may phone your support center to try to resolve a problem. Resolving the problem is the job they want to get done. Your research should be about that entire job, and not about a sub-component. Customer feedback about the components is only useful in the context of the overall ‘job’ they need you to do.

Three main situations where such research is useful

Transactional / episode research is most appropriate in three situations:

1. Use it to periodically validate that improvements in your customer-centric operational metrics do indeed improve customer satisfaction. For example, your customer complaint handling metrics may show that all complaints are fully resolved within 48 hours. Do your customers agree that this is the case? Perhaps your agents have incentives that push them to close cases that are not actually resolved.
2. Obtaining customer feedback that cannot easily be derived from operational metrics. Let’s suppose you have done a metric validation survey and have discovered that there is a disconnect between your operational metrics and what customers think about complaint handling. You know there is a problem, but don’t have the information necessary to fix it. What exactly is going on, as perceived by customers? Resist the temptation to ask about the politeness of the person on the phone and a long list of items you might think of. Just use the NPS format and ask the customers why, and what should be improved.
3. As quality indicators for subcontracting agreements. Let’s suppose you have contracted with a trucking company to deliver goods, or have contracted with a third party to process your employees’ travel expense claims. You may have specified certain quality or satisfaction levels in the contract, and need to check that the company you are using meets your standards.

You may be surprised that I do not list something like “100% validation that customers are happy with telephone support”. You will never get 100% of the customers to answer, and your operational metrics and responses to the scripts you use should tell you this in any case. Over the years, I have seen that many call center scripts do not actually ask customers whether their issue has been resolved. I find this odd and recommend always asking, if you don’t already know for certain. If you work in software, please resist the temptation to consider a software defect as resolved if it has been agreed that it will be fixed in the next version. The customer’s issue is only resolved when the software has been released and the customer has installed it. You should leave the case open until then, to ensure correct follow up. Customers love to be remembered, and will appreciate the call at the appropriate time.

NPS may not be the solution for everything

There are types of transactional surveys where I feel NPS is not the best solution. You may still like to use it, simply to avoid having to communicate an additional measurement system within your company. Willingness to recommend a company or product is mainly an overall brand metric. While an individual transaction can destroy your standing with a customer, it is really unlikely that an individual transaction can cause a customer to recommend you all on its own. NPS does not have predictive value for the revenue of telephone support businesses, for example. If you can deal with the communication challenges, you may be better off using Customer Effort Score for many types of transactional surveys. Customer Effort Score measures how hard customers found it to do certain things, like solve a problem. The authors of The Effortless Experience (see below) have demonstrated that it predicts revenue for support businesses.

NPS has additional questions for transactional surveys

In the current version of the Net Promoter System for transactional surveys, the usual Recommendation and Why questions are followed by additional questions: “To what extent has your latest experience made you more or less likely to recommend [company]?” followed by an additional Why question. While this makes the format longer, it provides information on the positive or negative impact individual transactions may have on your overall brand perception.

Difficulties for service center research

It is challenging to use NPS, or indeed any other research system to gain insights about telephone service centers. The reason is simple. As documented in The Effortless Experience[1] by Dixon, Toman and DeLisi, most customers who contact you by phone are simultaneously trying to answer their questions or resolve their issues on your website or elsewhere. It is therefore difficult to know precisely what resolution process their survey responses refer to. The customer could have found 90% of the information they needed from a third-party website, and just received confirmation from you by phone. Think of this as you choose the wording of your questionnaire.

Pop-ups

Personally, I find pop-up web surveys for transactions irritating. I suppose they must work for some companies, or we would not see them anymore. Best practice is to have at least the rating scale directly in the pop-up, like the Windows 10 satisfaction pop-up you may have seen if you are a Windows user. If you are not able to work out how to capture text feedback within the pop-up, clicking on one of the numbers should take you to a web page to answer the remaining questions.

Conclusion

Transactional surveys are over-used in my opinion. Customers are almost never informed about what you have learned or what you are doing with the results. Indeed, even where I have been responsible for the implementation of transactional surveys, I never included a process for summarizing what we had learned and informing customers, so shame on me too. In most cases, the surveys are used to generate metrics, not action. Surveys are sometimes implemented when neither people nor funding are available to work on the improvements that might be required. Yes, there may well be a ‘Hawthorne effect'[2], in that many studies have shown that the simple fact of demonstrating you are paying attention to something does indeed improve its performance. I liken that to the placebo effect of ‘alternative’ medicines. These effects are worthwhile, but are not a great substitute for intervening and fixing what ails you.


[1] Matthew Dixon, Nick Toman and Rick DeLisi: The Effortless Experience, Portfolio, ISBN1591845815
[2] The psychological experiments at the GE Hawthorne plant are described in the Wikipedia article at https://en.wikipedia.org/wiki/Hawthorne_effect



Looking forward

The next article in this series will cover will answer a frequently-asked question: what is a good score?’

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.