Satmetrix

The NPS metric is just one part of a much larger system. The Bain & Company Net Promoter System Framework has five elements, as shown in the diagram at the top of this article. It comes from the Bain www.netpromotersystem.com website. The website describes the elements in some detail. This series of articles discusses how to implement each element. The material on some parts of the framework is more detailed than others, notably everything that is involved in ensuring NPS is a reliable and trusted metric in your organization.

Leadership matters

Leadership is important. It is unlikely that the system can be implemented by consensus. Attempts to get everyone possible to agree to what needs to be done will lead to excessive complexity, and lack of clarity about the priorities. Personal leadership is required, and it can only be completely successful in the context of sustained commitment from senior leaders. The articles on each part of the framework will include examples of good practice. Real-world counter-examples are provided between framework discussions. There is somewhat of a theme in the counter-examples: excessive focus on metrics, rather than improvement. We will start by discussing how to ensure NPS is a ‘reliable, trusted currency/metric’, work down the framework, and finish by covering leadership commitment.

Reliable trusted metric – Introduction

The Net Promoter System relies on a metric that is easy to understand, and perhaps paradoxically, easy to misunderstand or even deliberately misuse. There are several different ways of generating a Net Promoter Score. Each has its own specific use. Each is reliable in a different way. It is quite common to see companies advance marketing statements like “Our NPS is 92, with a 30-point lead over our nearest competitor.” No explanation is typically provided about where the number comes from. I have investigated a number of such claims in detail. Typically, the two numbers being compared are from different types of surveys, and are not actually comparable.

This part of the blog series is about how to ensure both a reliable score and a trusted feedback-gathering process. The trust has internal and external aspects. Internally, your colleagues must believe that the feedback is indeed representative of the customer and competitor base it claims to represent. Externally, your feedback and improvement process must provide more value to customers than it extracts. If not, fewer customers will respond to your next feedback request, decreasing reliability.

Making it matter as much as financial metrics

You want NPS to be included in leadership team discussions and to have similar status to any other robust metric, such as your P&L. You need to be able to report your key research results with the same frequency and reliability. If your main financial discussions are quarterly, your main NPS reports should be quarterly too. Just as your P&L is compared to that of your competition, your NPS report should include competitive information as well. Competitive benchmark surveys will be the subject of my next article. And it is perhaps worth noting that as of May 2019, about 150 of the S&P 500 report NPS scores and trends in their quarterly financial reports.

NPS feedback needs to represent your customer population

Assuming you are not wasting your customers’ time and are going to act, you need to ensure your feedback is representative. The purpose of the whole system is to improve loyalty and encourage customers to promote you to others. You need to be clear which customers this means and to target them correctly in any feedback system. To pick an example, if you want to get feedback about your product, but only use data about people who contact you with problems, your feedback system is incomplete at best.

Let me pick an extreme example in another domain to illustrate this. For many years, homosexuality was on the formal US list of psychiatric illnesses. Research by psychiatrists showed that gay people had a truly amazing set of symptoms. The root cause of this classification problem was that the only homosexuals psychiatrists studied were people who had been referred to them because they had psychiatric illnesses. It took many years before this was understood and corrected.

Types of NPS research

The conventional types of feedback scores and input you need to understand and be able to use are benchmark and supplier surveys, relationship feedback, product / service / project feedback, transactional feedback and the feedback from your own employees, known as eNPS or Net Promoter for People. Each can help you to understand and predict different things. They should never be mixed together or compared to each other. Each needs to be reliable and trusted in its own way.

Introducing a new type of research: ‘ceNPS’

I will cover eNPS in a future article about ensuring you have an Employee / Team environment focused on loyalty. At that point, I will introduce a new type of research, which is called customer-employee NPS. The principle is that you ask your own employees how likely customers are to recommend your company, why, and what should be improved. This can be valuable in addition to customer research because employees are better positioned than customers to understand which improvements are easy to implement.



Looking forward

The next article in this series will cover in important aspect of ensuring you have a trusted and reliable metric: competitive benchmark research.

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

This series supposes the reader is already familiar with the basics of the Net Promoter System, and ideally has read The Ultimate Question 2.0 by Fred Reichheld and Rob Markey. There is quite a lot of mis-information about what they wrote. Some comes from people who have not read the book. Some comes from people who have read it, but had difficulty understanding some points. I have had the opportunity to clarify a few tricky points directly with the source. Let’s start with the metric, rather than the overall system. The image above shows the rating question and how the scoring works. That’s the easy part.

Open questions

After the rating question, the current standard is to use two open questions:

1. Why?
2. What could we do better?

Additional questions may be used for touchpoint surveys, also called ‘episode’,  ‘experience’ or ‘interaction surveys’. Bain recommends simply adding a second rating question, asking “To what extent has your latest interaction changed your willingness to recommend?”

Which direction for the scale?

Fred and Rob have used the scale in both directions, from zero to 10 and the opposite. There are views, supported by psychology experiments, that the order biases the results in one direction or the other. It does not matter which you use, as long as you are consistent. Personally, I have always used it as shown here, though I agreed to requests to reverse the scale from my HP colleagues in countries that write from right to left to reverse the scale.

What about using a different scale?

Some companies use a 1 to 5 scale, with 5 as the best score and the only Promoter score. 4 is for Passives and 1 to 3 for Detractors. This has two disadvantages. First, all too many people believe they should strive to be number 1, and therefore think that 1 is the top score, no matter how it is labeled. This is quite easy to spot if you also have an open text question and the answer seems opposite to the rating, though the respondent could be trying to be sarcastic. Second, there are cultures that just don’t want to give you the top score. On a scale from 0 to 10, the happiest people in these cultures may give you a 9, but on a 1 to 5 scale they give you a 4.

Again, all this does not matter much if you are consistent, only comparing yourself with others using the same scale. When HP moved from the short to the long scale for competitive benchmark surveys, Satmetrix advised that it would have a small positive effect on NPS scores for HP and for all competitors. That is indeed what happened.

Your ultimate question may not be the recommendation question

This may seem subtle: when Fred wrote the original Harvard Business Review paper[1], he found that the recommendation question trend was the best revenue predictor for most but not all industries. This matters. Your personal ‘Ultimate question’ should be whatever single question best predicts your revenue and market share trends relative to your competition. You should start by using the recommendation question in the absence of any contrary data. Even in industries where it was not the best question, it was close to the top.

I realize this statement is somewhat controversial. Fred and Rob have stated it publicly in Episode 82 of the Net Promoter System Podcast series[2]. Some of their early work, notably with Enterprise Rent-A-Car, did not use the recommendation question. There are limits to what you should use, to avoid confusion. First, you will cause confusion if you let each business and function in your company select their own question and all refer to it as NPS. You can only have a single question formulation in your company if you want to be able to communicate effectively. Second, don’t allow composite metrics made up from multiple questions to be represented as NPS. That is opposite to the whole principle of a single simple question.

Categories and behaviors

From the start, the Bain teams were able to observe and quantify behavior by response category. I suppose the most important observation is that a 5 is absolutely not a middle-of-the-road average score. People who give you a 5 (on the 0 to 10 scale) do not like you and may well speak negatively about you to others. The same applies to those who give you a 3 on a 1-to-5 scale. This is an important observation when thinking about how people use Customer Satisfaction (CSAT) scores on a five-point scale. CSAT practitioners consider people who give a 3 to be satisfied, and they are not. Their actions say otherwise.

Early research with Satmetrix

Bain asked Satmetrix to provide additional proof that NPS trends relative to competitors correspond to relative revenue growth, expressed as a percentage. This applied once again to most but not all industries studied. The main exceptions were industries that tend to have local monopolies, such as cable television companies. Fred found the relationship between NPS trends and relative growth to be particularly striking in the airline industry. In the HBR article referred to above, he says “… no airline has found a way to increase growth without improving its ratio of promoters to detractors.”

Don’t forget common sense

Accept for a moment that the recommendation question is the best one-question growth and share predictor for your company. Imagine you have seen multiple quarters of positive NPS trends relative to your competition. Growth ahead! What could possibly go wrong? Many things. Let’s suppose you run a software business that is entirely dependent on sales people that go to visit customers and that you have no intention of changing that. Each sales person has an average quota for the industry of about $2 million per year. You decide that you can’t afford to hire any additional sales people. Happier customers may make it easier for the existing sales people to make their quotas and will renew more of their existing annual contracts, but you will be throwing away the full growth opportunity if you don’t hire more sales people.

To use a non-NPS example, warmer weather predicts improved ice cream sales. If you don’t have additional ice cream in stock, you won’t sell any more. A Net Promoter System that functions well and produces better scores is simply a business enabler. You still have to convert the opportunity it creates.

Analyzing text responses

Humans are unfortunately subject to all sorts of cognitive biases. These are problematic when it comes to the search for insights in customer responses to the open Why and Improve questions. Consciously or unconsciously, we all tend to see things we already believe or agree with more quickly than we recognize things we don’t particularly want to see. Unless you have only a small number of verbatim responses, software will be an invaluable aid to your search for new insights. There were no good solutions available when Fred and Rob wrote their book. That has changed, and I will cover how to evaluate survey analysis software. As it exists today, software can do an excellent job of initial screening and categorization, and humans should still control the final decisions.

The books

Fred Reichheld published The Ultimate Question in March 2006. Rob Markey co-wrote the ‘revised and expanded’ v2.0 of the book and published it in September 2011. As of May 2019, they do not yet plan to write v3.0. Nevertheless, their methodology has been improving since 2011. The updates are reasonably clear in certain Bain Net Promoter System Podcasts, specifically the ones where Rob Markey speaks on his own, or together with Fred. Rob and I also recorded the first of many podcasts together in July 2017 and clarified many of the improvements to the system.

To me, the single most important change is the addition of a third question to the prior standard set of two, “What would you like us to improve?” The reason it matters is that the old standard meant that Detractors gave clear suggestions for improvement, but Promoters did not get that opportunity. Promoters generally love you and want to help. They give longer and more helpful improvement suggestions than Detractors.

Learning check

Decide whether each of these statements is true or false. Answers are at the bottom of this article.

1. The answers to “How likely are you to recommend [Company] to a friend?” were found to be the best predictors of future revenue trends among all the questions tested by Fred Reichheld in his 2003 Harvard Business Review article The One Number You Need to Grow.
2. The answers to the “How likely are you to recommend [Company] to a friend?” were found to be the best growth predictors for all industries tested, in the same HBR article.
3. The Net Promoter Score scale must start at zero and go to ten. All other scales are invalid.
4. Reichheld’s research showed that customer behaviors really do change depending on their classification as Promoters, Passives or Detractors.
5. The definitive reference source for the Net Promoter System is The Ultimate Question 2.0, and no other information on the design and implementation is available from Reichheld and Markey.

Looking forward

The next article in this series will cover the Bain Net Promoter System Framework.

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

[1] The “One Number You Need to Grow” HRB article can be found here:
https://hbr.org/2003/12/the-one-number-you-need-to-grow

[2] All podcasts are at http://www.netpromotersystem.com/resources/podcast.aspx

Learning check answers

1. True.
2. False. Other questions finished at the top of the list for a few industries. The recommendation question finished just behind in those cases.
3. False. The scale can also go from ten to zero, or indeed one to five. It is important to be consistent, using a single scale for your company and keeping it the same from survey to survey.
4. True.
5. False. While The Ultimate Question 2.0 is a reference source, it has been updated extensively. The Bain updates to the system are covered in the Net Promoter System Podcasts. Satmetrix has independently developed what it calls NPS2.