NPS (9) – 9th in the series – How to use the answers to the open questions

While our rational minds like to see scores and trends, the primary value of Net Promoter System surveys lies in the answers to the “Why?” and “What should we improve?” questions. Interpreting and using these answers well is quite challenging. Indeed, it is so challenging that many companies don’t really try. Communicating trends in numbers is far easier than finding and communicating trends in text. Avoiding our personal biases when looking at the answers is tricky too. We will cover software solutions for text analysis in the section on infrastructure. Here are some considerations on other points.

The answers do not change quickly

Once you have been gathering feedback for some time, you will see that the most common answers to the verbatim questions change more slowly than NPS scores. This makes it difficult to maintain management interest. The main way to make people want to listen is to focus on personalization. By this I mean using quotes from specific named customers as examples of each response theme you are presenting. I suppose you can think of it as though you were directing a reality TV show. The most extreme comments are the ones that will attract the most attention. Of course, many comments will be entertaining, but not actually useful. Care is needed to ensure they support the improvement initiatives you are proposing.

Some answers matter more than others

You will quickly learn that some things are only mentioned by Detractors, and others only by Promoters. Understanding which are which is critical to understanding the items that are hygiene factors, meaning they just need to be ‘good enough’. Psychologist Frederick Herzberg developed his two-factor theory of motivation. At its core, ‘motivators’ can only motivate people, and ‘hygiene factors’ can only demotivate them. Once hygiene factors are good enough, nobody cares about them. When communicating them, I suggest separating the groupings, presenting just one at a time.

• ‘Just not good enough’ is the set of three to five things that Detractors most often give as reasons for being detractors. If Passives and Promoters don’t mention them in their answers to the “Why” question, they are clearly hygiene factors. Once you improve them to the point where they are good enough, you don’t need to spend any more.
• ‘Potential loyalty drivers'” are the most common improvement suggestions made by Promoters that do not come up as much from the other categories.
• ‘Communication opportunities’ are things that commonly come up as improvement suggestions, but which you have already implemented. To pick an example, this might be a suggestion for a feature that is already in your software program, but is not as easy to find as it might be, or may be missing from your manual or help file.

Competitive comparisons are the key

The key to competitive advantage lies of course in the comparison between what customers say about you and what they say about your competitors. Using hygiene factors as an example, competitive comparisons are the only way you can know whether something some customers complain about is ‘good enough’ or not. A hygiene factor is ‘good enough’ if customers don’t complain about you more than they do about your competitors. (If you have not seen the reference to hygiene factors in an earlier post in this series, a good example of a hygiene factor in retail is the credit card payment process. It can only make your customers unhappy if your process fails. If it works perfectly, nobody will even notice it, let alone recommend your brand because of it.)

Beyond hygiene factors, if the most common improvement suggestion for the leading competitor’s product is something your product already does, this fact needs to be in your sales people’s ‘battle cards’ for competitive bids, or clearly communicated as such to your resellers. Maintain interest by showing trends over time.

If, for example, you can see that comments on a competitor’s product quality have been improving markedly, while yours have been getting worse, that should grab your audience’s attention. Support the relative trend discussion by including direct customer quotes.

The difference between the Why and Improve responses

When a customer is angry with you, the Why and Improve responses are often about the same problem. “I had to wait 20 minutes in the checkout queue” is often followed by a suggestion to “Hire more people to shorten queues at peak times.” So, for Detractors, asking both questions can seem superfluous. This is not the case for Promoters. For the same store as the Detractor, a Promoter might give “Great variety of modern furniture that suits my budget” as the reason they like you, and still say “Hire more people to shorten checkout queues” as an improvement suggestion.

Using people to study text responses

If you decide not to use software to study verbatim responses, ask a variety of people to study the data and propose the top three themes in each category. This is a type of work where using an ‘outsider’ is counter-productive. Everyone you ask to help must understand the subject matter in some detail. You are looking for expert views. If you have responses in languages your experts do not speak, using Google or Microsoft’s automatic translation software works well enough, and is of course far better than discarding those answers.



Looking forward

The next article in this series will look at how to avoid ‘confirmation bias’ as you examine survey results.

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.