Net Promoter System

Raining surveys wide

NPS (7) – 7th in the series – Transactional / Episode surveys – Best avoided? Use very selectively!

/ , , , , , / , , , , , /

Transactional surveys are by far the most common type of customer research. It is helpful to think of them as feedback about ‘episodes’ or ‘jobs to be done’. Customers interact with you to try to accomplish some individual thing as part of their overall experience with you. An overall season of a TV series, for example, consists of a series of episodes. A customer may phone your support center to try to resolve a problem. Resolving the problem is the job they want to get done. Your research should be about that entire job, and not about a sub-component. Customer feedback about the components is only useful in the context of the overall ‘job’ they need you to do.

Three main situations where such research is useful

Transactional / episode research is most appropriate in three situations:

  1. Use it to periodically validate that improvements in your customer-centric operational metrics do indeed improve customer satisfaction. For example, your customer complaint handling metrics may show that all complaints are fully resolved within 48 hours. Do your customers agree that this is the case? Perhaps your agents have incentives that push them to close cases that are not actually resolved.
  2. Obtaining customer feedback that cannot easily be derived from operational metrics. Let’s suppose you have done a metric validation survey and have discovered that there is a disconnect between your operational metrics and what customers think about complaint handling. You know there is a problem, but don’t have the information necessary to fix it. What exactly is going on, as perceived by customers? Resist the temptation to ask about the politeness of the person on the phone and a long list of items you might think of. Just use the NPS format and ask the customers why, and what should be improved.
  3. As quality indicators for subcontracting agreements. Let’s suppose you have contracted with a trucking company to deliver goods, or have contracted with a third party to process your employees’ travel expense claims. You may have specified certain quality or satisfaction levels in the contract, and need to check that the company you are using meets your standards.

You may be surprised that I do not list something like “100% validation that customers are happy with telephone support”. You will never get 100% of the customers to answer, and your operational metrics and responses to the scripts you use should tell you this in any case. Over the years, I have seen that many call center scripts do not actually ask customers whether their issue has been resolved. I find this odd and recommend always asking, if you don’t already know for certain. If you work in software, please resist the temptation to consider a software defect as resolved if it has been agreed that it will be fixed in the next version. The customer’s issue is only resolved when the software has been released and the customer has installed it. You should leave the case open until then, to ensure correct follow up. Customers love to be remembered, and will appreciate the call at the appropriate time.

NPS may not be the solution for everything

There are types of transactional surveys where I feel NPS is not the best solution. You may still like to use it, simply to avoid having to communicate an additional measurement system within your company. Willingness to recommend a company or product is mainly an overall brand metric. While an individual transaction can destroy your standing with a customer, it is really unlikely that an individual transaction can cause a customer to recommend you all on its own. NPS does not have predictive value for the revenue of telephone support businesses, for example. If you can deal with the communication challenges, you may be better off using Customer Effort Score for many types of transactional surveys. Customer Effort Score measures how hard customers found it to do certain things, like solve a problem. The authors of The Effortless Experience (see below) have demonstrated that it predicts revenue for support businesses.

NPS has additional questions for transactional surveys

In the current version of the Net Promoter System for transactional surveys, the usual Recommendation and Why questions are followed by additional questions: “To what extent has your latest experience made you more or less likely to recommend [company]?” followed by an additional Why question. While this makes the format longer, it provides information on the positive or negative impact individual transactions may have on your overall brand perception.

Difficulties for service center research

It is challenging to use NPS, or indeed any other research system to gain insights about telephone service centers. The reason is simple. As documented in The Effortless Experience[1] by Dixon, Toman and DeLisi, most customers who contact you by phone are simultaneously trying to answer their questions or resolve their issues on your website or elsewhere. It is therefore difficult to know precisely what resolution process their survey responses refer to. The customer could have found 90% of the information they needed from a third-party website, and just received confirmation from you by phone. Think of this as you choose the wording of your questionnaire.

Pop-ups

Personally, I find pop-up web surveys for transactions irritating. I suppose they must work for some companies, or we would not see them anymore. Best practice is to have at least the rating scale directly in the pop-up, like the Windows 10 satisfaction pop-up you may have seen if you are a Windows user. If you are not able to work out how to capture text feedback within the pop-up, clicking on one of the numbers should take you to a web page to answer the remaining questions.

Conclusion

Transactional surveys are over-used in my opinion. Customers are almost never informed about what you have learned or what you are doing with the results. Indeed, even where I have been responsible for the implementation of transactional surveys, I never included a process for summarizing what we had learned and informing customers, so shame on me too. In most cases, the surveys are used to generate metrics, not action. Surveys are sometimes implemented when neither people nor funding are available to work on the improvements that might be required. Yes, there may well be a ‘Hawthorne effect'[2], in that many studies have shown that the simple fact of demonstrating you are paying attention to something does indeed improve its performance. I liken that to the placebo effect of ‘alternative’ medicines. These effects are worthwhile, but are not a great substitute for intervening and fixing what ails you.


[1] Matthew Dixon, Nick Toman and Rick DeLisi: The Effortless Experience, Portfolio, ISBN1591845815
[2] The psychological experiments at the GE Hawthorne plant are described in the Wikipedia article at https://en.wikipedia.org/wiki/Hawthorne_effect



Looking forward

The next article in this series will cover will answer a frequently-asked question: what is a good score?’

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

Bain NPS emoticons

NPS (6) – Getting useful feedback about one of your products, services, or customer projects

/ , , , , , / , , , , , /

Unless your company only sells a single product or service, you need a separate process to gather feedback about each product or service you sell. While benchmark and relationship surveys cover a variety of people and job roles at the customer end, product and service feedback should come from the end users. To get reliable and credible information, you should ask for it when you are explicitly planning to use it. For a service contract that you are going to renegotiate (for example), get feedback from the end users before the negotiation starts and use it to present a more convincing case for renewal.

For products, tell the end users that you are starting a new design cycle, solicit their suggestions, and tell them which suggestions were most common and will be included in the next product or version. If you have customers who are references for your product, pay particular attention to what they say. Your most reliable feedback will come from customers who actively use the product or service. For this type of survey, I don’t think it matters whether they have purchased it recently or not. Indeed, an end user may have no idea when the purchase decision was made. If your survey is about something like software that has versions, you will need to ask the users what version they are using if you don’t already know. If so, make sure you tell the users how to determine the version number.

Project surveys

Assuming you want to improve the way you plan and deliver customer projects, a formal feedback process is good practice. I have never seen these done well and they are indeed tricky. I believe you should consider project surveys as the least reliable in your portfolio. The reasons are quite complex, and change somewhat between shorter (up to three months) and longer projects:

  • The people who will answer the survey are most often chosen when the project is complete (for shorter projects) or when a major milestone has been achieved. Since the project team normally chooses the people who should provide feedback, they have an understandable tendency to select only those who will give favorable answers. Ideally, this can be offset by agreeing the names or job titles of those who will respond in the project contract. Unfortunately, I have never seen this done, both because people want to use standard contracts, and because they do not want to “waste time negotiating details.”
  • Another possibility is to be clear, both in theory and in practice, that you are not going to measure the project team on the customer feedback. Emphasize that you will use the feedback to improve the processes and tools for all projects, as well as to plan the development of project team members. It is almost impossible to communicate this credibly. Are you really willing to discard customer feedback when putting project team members through their performance reviews? The only thing left is to have a person outside the project team agree the names of those who will provide feedback. This can be the executive sponsor for the customer, if you have one, or the sales leader assigned to the customer.
  • For longer projects, the members of the project team and the overall project manager may change over time. It is quite common to see a first team for a proof of concept, a second team for a pilot and a third team for a comprehensive rollout. In such situations, you should not have a single feedback process, but have at least one for each phase. Working purely by phase is deficient too, as lack of project management continuity across phases is a common customer complaint.

While remaining aware of the aforementioned defects, my recommendation is to track trends by manager of project managers. It seems reasonable to expect that a single project manager will introduce the same conscious or unconscious bias to all project surveys over time. Tracking at the manager level will smooth it out somewhat.

Projects involving Alliances

Most people think of alliances between companies as being two-way affairs. However, it commonly takes three or more companies to implement a customer project, and any research process must take that into account. If a customer receives a survey from only one of the companies involved in implementing a project, they are likely to be surprised. An example may help here.

When I was managing certain software alliances at Compaq, I got a surprise. I had Lotus as one of the services alliances I led at a worldwide level. Lotus had been acquired by IBM about 18 months previously. IBM encouraged them to continue to act as though they were independent. One consequence of this was their love of Compaq. 55% of all the Lotus Notes installations were being sold on Compaq ProLiant servers, and the proportion was increasing slowly, to the irritation of Lotus’s new masters. I was in their Cambridge Mass. HQ, trying to work out how to use services to grow our joint business. It turned out that neither Lotus nor Compaq wanted to be ‘Prime contractor’ for any Notes implementations. A simple two-way alliance was pointless.

Further discussion revealed that the largest Notes implementer on the planet was PricewaterhouseCoopers (PwC), and that this was particularly the case in EMEA. I met repeatedly with the head of the PwC Notes practice in Zurich, and we were successful in some three-way projects with Lotus. (Then HP tried and failed to acquire PwC, ruining the PwC/Compaq work, though this is not directly relevant here.) Two- and three-way alliances are quite common for customer projects. You need to agree a single approach to surveying the whole project and driving improvements together.

Training surveys

Since I have taken many training courses over the years, I have been surveyed many times. Surveys tend to be quite long and to involve rating the teachers. Oddly, I don’t recall ever being asked what any individual teacher could improve. I feel NPS can be used in a hierarchical way, starting with an overall course score, and the usual Why and Improve questions. This can be followed by the same three questions for each teacher. I do not believe you should ask about the course material, facilities, catering, accommodation and so on. If they matter, they will come up in the written comments. Unfortunately, trainers tend to want to train you on how to answer the surveys. They will typically show a slide that gives a sample score (always a 10), and sample comments, which are always positive. This is a misguided approach and means the trainers do not get many of the improvement suggestions they need.

 

Note that the Bain NPS emoticons at the top of this article are registered service marks and are used with permission.



Looking forward

The next article in this series will cover will be about how to handle transactional / episode research.

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

Track relationships with an organization chart

NPS (5) – Relationship research with large B2B customers

/ , , , , , / , , , , , /

A good relationship feedback process could be described as a non-anonymous mono-vendor benchmark survey. You usually only get detailed information about your own company. If you just have a small number of customers, say less than a hundred, you should simply interview them all face-to-face. If you have a large number of customers, you should segment the customers in some way, using increasing depth for your most important customers. Deeper surveys do not mean asking more questions. The Net Promoter System format remains intact. When you interview your most important customers, you go into far greater depth on the ‘Why?’ and ‘What should we improve?’ questions. Clarifying the input right away will make it far easier to prepare a worthwhile improvement plan for the customer to approve.

It is certainly possible to add questions about competitors to your relationship survey, though it would dilute the focus on your own company’s improvement opportunities. In cases where your product and your competitor’s are mutually exclusive, meaning nobody would buy from both companies, adding questions about competition would give you an early indication about whether customers are thinking of changing. For example, “Why do you buy from us instead of [insert key competitor name]?

Resellers and other partners

Companies that help you to sell or implement your products and services are a special case. Since their job is to recommend your products and services, asking them the recommendation question is not appropriate. I recommend asking “Please rate your overall satisfaction as an Acme partner”, followed by the usual “Why?” and “What should we improve?” Since your partners have the choice of working with other suppliers, they are a good source of competitive information. I therefore suggest adding the following questions, using a reseller example, and supposing that the main competitors are called Alpha and Beta:

  • Do you sell products from Alpha or Beta?
  • What can we at Acme do to make it easier for you to recommend us over Alpha and Beta?

Future-facing relationship segmentation

I believe it is more useful to segment your research and improvement process by your future revenue expectations for a customer, rather than by what you achieve now. This means that customers who spend a lot on your type of product or service deserve a lot of attention, no matter who they spend it with. I suggest a two-year time horizon. The segmentation is used to determine the type of feedback process you will use. The top tier should be interviewed face-to-face, the second tier by phone or Skype and the lowest tier by email / web. The depth of your loop-closing and improvement commitments will also vary by tier.

Publish strategic relationship NPS numbers and trends by individual, not by company

Account-level NPS metrics can be counter-productive. If you mainly publish relationship NPS scores for each individual customer and perhaps a table that compares the scores, your metric will never be reliable. Account teams who do not look good on the list will do everything possible to manipulate the process so they do better the next time. To ensure reliability, publish the scores and trends by individual person in the customer relationship map. Your overall relationship with a customer is largely the sum of a number of individual relationships. These are what need to be monitored and improved, or referenced. Improvements should be agreed with each person. Even at the level of the individual, the trend is more important than the absolute number. The chart at the top of this article is a suggestion about how to show the relevant information quite simply.



Looking forward

The next article in this series will cover will be about how to get actionable feedback for individual products, services, and projects.

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

Competitive NPS benchmarks

NPS (4) – Competitive benchmarks and supplier surveys

/ , , , , , / , , , , , , /

Quite a few companies sell NPS benchmark data. Some will do custom surveys to your design. Done properly, benchmark survey trends are the best predictor of revenue and market share trends for your company. Since benchmark surveys provide comparative data, they are actually the only credible predictors. All other survey types cover only your company. Feedback about your product may be improving, but if that for your main competitor’s product improves more, you will lose share. Unfortunately, most vendors only provide annual benchmark information as a standard offering, though Bain’s new Prism offering is an exception. If you are in an industry that does not change quickly, annual updates may be reasonable. For most businesses, and to keep customer experience as visible as financial results, you need benchmark data quarterly, at the very least.

The ‘Why’ matters more than the trends

The score trends for your company and competitors are useless without knowing why. If you buy benchmark reports that do not include reasons why, they are not worthwhile and you should change vendors. The reasons behind positive and negative relative trends need to be explained at the same time as the numbers. The best benchmark vendors also provide improvement suggestions that are customized for your company and your competitors.

Use rolling quarterly reports for benchmark surveys

It can be expensive to get statistically significant NPS volumes for all of your competitors every quarter. Using a rolling four-quarter average is fine. This means your Q1 report will also include the data from the prior Q2, Q3 and Q4. Drop the oldest quarter as soon as you have a new quarter of data. Your reports should show multiple rolling quarters so trends are easy to understand. The graph at the top of this article shows a good way of communicating quarterly rolling trends.

The good and bad of double-blind benchmark surveys

The best benchmark surveys are double-blind. This means the people answering the survey do not know who is funding it. Unfortunately, it also means that you have no way of knowing the names of those who have answered. The positive aspect of this is that neither you nor your competitors can try to influence them. On the negative side, you can’t follow up with anyone you would like to use as a reference or who has serious concerns.

When you receive benchmark data, it is ‘unweighted’. For some businesses, equality of respondents is a good thing. For most businesses equality of all respondents in a bad thing. While you should get data about the sizes of the companies and the job titles of people represented in a business-to-business (B2B) survey panel, I have never seen it used in practice. Customers with large budgets and small budgets are mixed together. Responses from people who are decision-influencers have the same weight as those from decision-makers and indeed from end users. Once you slice and dice the data, you quickly get into issues with the ‘law of small numbers’. By this I mean that you will see outliers that are just due to statistical variability in small samples. The subject is tricky, and I will cover it in another article.

I am not aware of any satisfaction benchmark that compares benchmark survey providers. The quality of the benchmark results depends on the relevance of the people who answer the questions. This is probably where you should spend most of your investigation time when selecting a new vendor. Ask to see the profiles of the panel members that will be used for your work. Ask for samples of the text input the vendor has received. They may only be willing to provide it for a different panel, used for a different industry from your own. The vendor should have no difficulty removing company names to make data anonymous.

Panel members usually receive points or other cash-equivalents that they can redeem for merchandise. I don’t believe anyone could make a living from being on many panels and taking surveys full-time, though you may have seen get-rich-quick schemes advertised on the internet that suggest it could be done.

Supplier surveys

In business-to-business situations, some customers regularly survey internal users of products and services and provide the resulting feedback to their suppliers. This is a form of benchmark survey in that each supplier is given a report that compares them to other suppliers. In some cases, you will see the names of the comparison companies and other cases they are just shown as ‘Company A’, ‘Company B’ and so on. Supplier surveys are my favorite type of B2B research, for a simple reason. Your customer is highly motivated to see you improve.

The first such process I was involved in was Vodafone’s Supplier Performance Management initiative. The person who set it up for Vodafone was subsequently headhunted by BP in Houston to do the same. The survey was run by the procurement department. They were measured on supplier performance improvement from year to year. They wanted us to present our improvement initiatives to them, and were very helpful in providing contacts at their end, and helping us with the work. Companies that operate their own supplier surveys usually decline to participate in other relationship surveys you may be running.

Correlation and regression analysis can be useful in benchmark studies

If you have limited written commentary and multiple questions have been asked in the benchmark survey, simple correlation analysis can be helpful. Here is a good example from the ICT industry. The Research Board is a Gartner-owned entity. Its members are about 260 CIOs and former CIOs from the Fortune Global 500. One of their annual projects is a supplier survey. Members rate each of their major IT and communications providers on many factors. An anonymized version of one of the results tables is shown in the table below. I used Excel to do simple correlation analysis between the overall scores and individual factors. I then used Excel’s conditional formatting to make it more readable. The correlation scores on the right suggest the weight of each question in the overall performance score.

Correlation, of course, is not the same as causality, and the written comments help. The main body of the table contains the scores, where 5.0 is the maximum possible. Account management comes out as the most important factor. Surprisingly and counter-intuitively, ‘understanding of my business’ is the least important item. Note in passing that vendor 14 has finished last in the survey for many years and is one of the most profitable companies on the planet. You don’t have to be loved to make lots of money. Having an effective monopoly in your area is another way of doing it. The Research Board survey does not include the recommendation question and each company covered only receives customer comments and improvement suggestions for their own company. Since there is no charge to the vendors for the report, it is fabulous value.

Research board benchmark results

Syndicated benchmark surveys may be the future

When I have shared benchmark survey results with my equivalents at other high-tech companies, I have been pleased to see that the NPS numbers and trends look similar in their surveys. This simply suggests that there are good NPS benchmark vendors covering high-tech. All customer experience leaders I have met have been highly irritated by competitors that publish non-benchmark scores, representing them as fair comparisons. This makes me feel that a good solution for many industries would be for the leading vendors to get together and agree to use a single benchmark provider as their reference. I believe it would be easy to get competitors to agree on the core NPS questions. Each would be free to pay the survey provider for additional questions if they so wished. Additional companies would be free to pay a fee to join the consortium. I suppose a challenge is that non-customer-experience executives from companies that do poorly would want to cut off the vendor, rather than address their issues. A first such offering exists. Bain and Qualtrics announced a joint syndicated benchmarking offering in March 2019. They call it Prism, and initial demand has been strong. Their announcement press release describes the service as ‘continuously updated’’.



Looking forward

The next article in this series will cover will be about how to understand your relationship and improvement opportunities with large B2B customers.

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

Bain Net Promoter System Framework

NPS (3) – Net Promoter System Framework, ensuring metric is trusted

/ , , , , , / , , , , , , , /

The NPS metric is just one part of a much larger system. The Bain & Company Net Promoter System Framework has five elements, as shown in the diagram at the top of this article. It comes from the Bain www.netpromotersystem.com website. The website describes the elements in some detail. This series of articles discusses how to implement each element. The material on some parts of the framework is more detailed than others, notably everything that is involved in ensuring NPS is a reliable and trusted metric in your organization.

Leadership matters

Leadership is important. It is unlikely that the system can be implemented by consensus. Attempts to get everyone possible to agree to what needs to be done will lead to excessive complexity, and lack of clarity about the priorities. Personal leadership is required, and it can only be completely successful in the context of sustained commitment from senior leaders. The articles on each part of the framework will include examples of good practice. Real-world counter-examples are provided between framework discussions. There is somewhat of a theme in the counter-examples: excessive focus on metrics, rather than improvement. We will start by discussing how to ensure NPS is a ‘reliable, trusted currency/metric’, work down the framework, and finish by covering leadership commitment.

Reliable trusted metric – Introduction

The Net Promoter System relies on a metric that is easy to understand, and perhaps paradoxically, easy to misunderstand or even deliberately misuse. There are several different ways of generating a Net Promoter Score. Each has its own specific use. Each is reliable in a different way. It is quite common to see companies advance marketing statements like “Our NPS is 92, with a 30-point lead over our nearest competitor.” No explanation is typically provided about where the number comes from. I have investigated a number of such claims in detail. Typically, the two numbers being compared are from different types of surveys, and are not actually comparable.

This part of the blog series is about how to ensure both a reliable score and a trusted feedback-gathering process. The trust has internal and external aspects. Internally, your colleagues must believe that the feedback is indeed representative of the customer and competitor base it claims to represent. Externally, your feedback and improvement process must provide more value to customers than it extracts. If not, fewer customers will respond to your next feedback request, decreasing reliability.

Making it matter as much as financial metrics

You want NPS to be included in leadership team discussions and to have similar status to any other robust metric, such as your P&L. You need to be able to report your key research results with the same frequency and reliability. If your main financial discussions are quarterly, your main NPS reports should be quarterly too. Just as your P&L is compared to that of your competition, your NPS report should include competitive information as well. Competitive benchmark surveys will be the subject of my next article. And it is perhaps worth noting that as of May 2019, about 150 of the S&P 500 report NPS scores and trends in their quarterly financial reports.

NPS feedback needs to represent your customer population

Assuming you are not wasting your customers’ time and are going to act, you need to ensure your feedback is representative. The purpose of the whole system is to improve loyalty and encourage customers to promote you to others. You need to be clear which customers this means and to target them correctly in any feedback system. To pick an example, if you want to get feedback about your product, but only use data about people who contact you with problems, your feedback system is incomplete at best.

Let me pick an extreme example in another domain to illustrate this. For many years, homosexuality was on the formal US list of psychiatric illnesses. Research by psychiatrists showed that gay people had a truly amazing set of symptoms. The root cause of this classification problem was that the only homosexuals psychiatrists studied were people who had been referred to them because they had psychiatric illnesses. It took many years before this was understood and corrected.

Types of NPS research

The conventional types of feedback scores and input you need to understand and be able to use are benchmark and supplier surveys, relationship feedback, product / service / project feedback, transactional feedback and the feedback from your own employees, known as eNPS or Net Promoter for People. Each can help you to understand and predict different things. They should never be mixed together or compared to each other. Each needs to be reliable and trusted in its own way.

Introducing a new type of research: ‘ceNPS’

I will cover eNPS in a future article about ensuring you have an Employee / Team environment focused on loyalty. At that point, I will introduce a new type of research, which is called customer-employee NPS. The principle is that you ask your own employees how likely customers are to recommend your company, why, and what should be improved. This can be valuable in addition to customer research because employees are better positioned than customers to understand which improvements are easy to implement.



Looking forward

The next article in this series will cover in important aspect of ensuring you have a trusted and reliable metric: competitive benchmark research.

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

NPS rating question and scale

NPS (2) – Basics and history of the Net Promoter Score and System

/ , , , , , / , , , , , , , /

This series supposes the reader is already familiar with the basics of the Net Promoter System, and ideally has read The Ultimate Question 2.0 by Fred Reichheld and Rob Markey. There is quite a lot of mis-information about what they wrote. Some comes from people who have not read the book. Some comes from people who have read it, but had difficulty understanding some points. I have had the opportunity to clarify a few tricky points directly with the source. Let’s start with the metric, rather than the overall system. The image above shows the rating question and how the scoring works. That’s the easy part.

Open questions

After the rating question, the current standard is to use two open questions:

  1. Why?
  2. What could we do better?

Additional questions may be used for touchpoint surveys, also called ‘episode’,  ‘experience’ or ‘interaction surveys’. Bain recommends simply adding a second rating question, asking “To what extent has your latest interaction changed your willingness to recommend?”

Which direction for the scale?

Fred and Rob have used the scale in both directions, from zero to 10 and the opposite. There are views, supported by psychology experiments, that the order biases the results in one direction or the other. It does not matter which you use, as long as you are consistent. Personally, I have always used it as shown here, though I agreed to requests to reverse the scale from my HP colleagues in countries that write from right to left to reverse the scale.

What about using a different scale?

Some companies use a 1 to 5 scale, with 5 as the best score and the only Promoter score. 4 is for Passives and 1 to 3 for Detractors. This has two disadvantages. First, all too many people believe they should strive to be number 1, and therefore think that 1 is the top score, no matter how it is labeled. This is quite easy to spot if you also have an open text question and the answer seems opposite to the rating, though the respondent could be trying to be sarcastic. Second, there are cultures that just don’t want to give you the top score. On a scale from 0 to 10, the happiest people in these cultures may give you a 9, but on a 1 to 5 scale they give you a 4.

Again, all this does not matter much if you are consistent, only comparing yourself with others using the same scale. When HP moved from the short to the long scale for competitive benchmark surveys, Satmetrix advised that it would have a small positive effect on NPS scores for HP and for all competitors. That is indeed what happened.

Your ultimate question may not be the recommendation question

This may seem subtle: when Fred wrote the original Harvard Business Review paper[1], he found that the recommendation question trend was the best revenue predictor for most but not all industries. This matters. Your personal ‘Ultimate question’ should be whatever single question best predicts your revenue and market share trends relative to your competition. You should start by using the recommendation question in the absence of any contrary data. Even in industries where it was not the best question, it was close to the top.

I realize this statement is somewhat controversial. Fred and Rob have stated it publicly in Episode 82 of the Net Promoter System Podcast series[2]. Some of their early work, notably with Enterprise Rent-A-Car, did not use the recommendation question. There are limits to what you should use, to avoid confusion. First, you will cause confusion if you let each business and function in your company select their own question and all refer to it as NPS. You can only have a single question formulation in your company if you want to be able to communicate effectively. Second, don’t allow composite metrics made up from multiple questions to be represented as NPS. That is opposite to the whole principle of a single simple question.

Categories and behaviors

From the start, the Bain teams were able to observe and quantify behavior by response category. I suppose the most important observation is that a 5 is absolutely not a middle-of-the-road average score. People who give you a 5 (on the 0 to 10 scale) do not like you and may well speak negatively about you to others. The same applies to those who give you a 3 on a 1-to-5 scale. This is an important observation when thinking about how people use Customer Satisfaction (CSAT) scores on a five-point scale. CSAT practitioners consider people who give a 3 to be satisfied, and they are not. Their actions say otherwise.

Early research with Satmetrix

Bain asked Satmetrix to provide additional proof that NPS trends relative to competitors correspond to relative revenue growth, expressed as a percentage. This applied once again to most but not all industries studied. The main exceptions were industries that tend to have local monopolies, such as cable television companies. Fred found the relationship between NPS trends and relative growth to be particularly striking in the airline industry. In the HBR article referred to above, he says “… no airline has found a way to increase growth without improving its ratio of promoters to detractors.”

Don’t forget common sense

Accept for a moment that the recommendation question is the best one-question growth and share predictor for your company. Imagine you have seen multiple quarters of positive NPS trends relative to your competition. Growth ahead! What could possibly go wrong? Many things. Let’s suppose you run a software business that is entirely dependent on sales people that go to visit customers and that you have no intention of changing that. Each sales person has an average quota for the industry of about $2 million per year. You decide that you can’t afford to hire any additional sales people. Happier customers may make it easier for the existing sales people to make their quotas and will renew more of their existing annual contracts, but you will be throwing away the full growth opportunity if you don’t hire more sales people.

To use a non-NPS example, warmer weather predicts improved ice cream sales. If you don’t have additional ice cream in stock, you won’t sell any more. A Net Promoter System that functions well and produces better scores is simply a business enabler. You still have to convert the opportunity it creates.

Analyzing text responses

Humans are unfortunately subject to all sorts of cognitive biases. These are problematic when it comes to the search for insights in customer responses to the open Why and Improve questions. Consciously or unconsciously, we all tend to see things we already believe or agree with more quickly than we recognize things we don’t particularly want to see. Unless you have only a small number of verbatim responses, software will be an invaluable aid to your search for new insights. There were no good solutions available when Fred and Rob wrote their book. That has changed, and I will cover how to evaluate survey analysis software. As it exists today, software can do an excellent job of initial screening and categorization, and humans should still control the final decisions.

The books

Fred Reichheld published The Ultimate Question in March 2006. Rob Markey co-wrote the ‘revised and expanded’ v2.0 of the book and published it in September 2011. As of May 2019, they do not yet plan to write v3.0. Nevertheless, their methodology has been improving since 2011. The updates are reasonably clear in certain Bain Net Promoter System Podcasts, specifically the ones where Rob Markey speaks on his own, or together with Fred. Rob and I also recorded the first of many podcasts together in July 2017 and clarified many of the improvements to the system.

To me, the single most important change is the addition of a third question to the prior standard set of two, “What would you like us to improve?” The reason it matters is that the old standard meant that Detractors gave clear suggestions for improvement, but Promoters did not get that opportunity. Promoters generally love you and want to help. They give longer and more helpful improvement suggestions than Detractors.

Learning check

Decide whether each of these statements is true or false. Answers are at the bottom of this article.

  1. The answers to “How likely are you to recommend [Company] to a friend?” were found to be the best predictors of future revenue trends among all the questions tested by Fred Reichheld in his 2003 Harvard Business Review article The One Number You Need to Grow.
  2. The answers to the “How likely are you to recommend [Company] to a friend?” were found to be the best growth predictors for all industries tested, in the same HBR article.
  3. The Net Promoter Score scale must start at zero and go to ten. All other scales are invalid.
  4. Reichheld’s research showed that customer behaviors really do change depending on their classification as Promoters, Passives or Detractors.
  5. The definitive reference source for the Net Promoter System is The Ultimate Question 2.0, and no other information on the design and implementation is available from Reichheld and Markey.

Looking forward

The next article in this series will cover the Bain Net Promoter System Framework.

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

[1] The “One Number You Need to Grow” HRB article can be found here:
https://hbr.org/2003/12/the-one-number-you-need-to-grow

[2] All podcasts are at http://www.netpromotersystem.com/resources/podcast.aspx

Learning check answers

  1. True.
  2. False. Other questions finished at the top of the list for a few industries. The recommendation question finished just behind in those cases.
  3. False. The scale can also go from ten to zero, or indeed one to five. It is important to be consistent, using a single scale for your company and keeping it the same from survey to survey.
  4. True.
  5. False. While The Ultimate Question 2.0 is a reference source, it has been updated extensively. The Bain updates to the system are covered in the Net Promoter System Podcasts. Satmetrix has independently developed what it calls NPS2.
News flash: Survey giver actually responds to survey taker!

NPS (1) – Do you really need a customer experience measurement and improvement system?

/ , , , , , / , , , , , /

This is the first in a series of blog posts about implementing the Net Promoter System.

The Net Promoter System®[1] is the most widely adopted system for managing customer feedback and improvement. It is used around the world and works well in all cultures and languages. Like most management systems, it has advantages and disadvantages. Its principal advantage is its simplicity. It is easy to understand and easy to communicate. Its disadvantages are two-fold: its lack of sophistication means it is tempting to find more intellectually pleasing solutions, and many implementers confuse the Net Promoter Score® and the Net Promoter System. Before going into detail on NPS®, let’s cover a few important points.

Current state of customer feedback systems on planet Earth

Like most readers, I receive requests to take surveys almost every day. Since I am interested in the topic, I fill out a lot of survey forms. I almost never hear anything from the people who sent the survey after that. What have they learned? What are they doing with my input. I took fifteen minutes to answer their questions, why can’t they take two minutes to keep me informed? Maybe they had no plans to do anything with the input other than report numbers to their management? Six months or a year later, I can observe that none of my improvement suggestions have been put into practice. I wind up feeling that my time has been wasted, and theirs too, since no action seems to have been taken.

My writing about customer experience strategy in general and the Net Promoter System in particular is about eliminating that waste, moving from survey processes to a mutual feedback and improvement process. The content of our books builds on my experience in implementing the Net Promoter System at HP, and helping others to do the same. While the system set out by Fred Reichheld and Rob Markey in The Ultimate Question 2.0[2] serves as the basis for all that follows, the system has been augmented in several ways that should speed up both implementation and results.

Do I actually need a feedback and improvement system?

Maybe you don’t absolutely need one. If you have a monopoly in your market and have government-imposed pricing, there may be nothing to gain in the short term. Why spend money if you already have all the income you can possibly get? Of course, your monopoly may not last. As well-known strategy guru Willie Pietersen told my class at Columbia Business School, “Somewhere in the world, somebody has just had an idea that will destroy your business.”

Cable TV companies discovered this to their cost when telcos started to compete with them. Most businesses are not monopolies and care about winning in their markets. They care about growth. According to Bain and Satmetrix, co-holders of the NPS service mark, NPS trends predict 20 to 60% of business growth trends, depending on the industry. Understanding how happy your customers are compared to your competitors is usually at least as important as comparing your sales coverage models and product features.

Make sure your customers are a company priority

Rather than reacting by saying “Of course we care about customers!” check your intranet, corporate annual reports, press releases and quarterly earnings statements. Do they actually mention customers? Check your CEO’s latest motivational email to all employees. Are customers mentioned in the top three priorities? Is cost reduction above customers on the priority list? Are things like “It is critical that we deliver the current quarter” at the top of the list? Checking the evidence is easy enough. It allows you to determine whether your company cares about customers from a strategic perspective. If customers are close to the top of the formal priority list in your company at the moment, get formal sponsorship for your NPS work and move ahead. More on how to get sponsorship this in another blog post. If customers are not on the list, you have some persuasion work to do first, and we will discuss how to do it.

Use operational data, not NPS, to run your company daily

All companies use operational data to ensure their processes are working as designed. The processes are in turn designed to maximize revenue, optimize cost, and respect defined service levels. Many operational metrics are delivered in real time. NPS has a time lag. Customer research and feedback needs to be used to adjust, prioritize and improve operational metrics. It is also particularly useful for the study of the relationship between cost and growth. For example, you may be able to show that pursuit of a particular cost metric has a measurable positive or negative effect on NPS, and therefore on future revenue.

In many cases, business processes that are cheap and simple are better for customers, as they may be faster and more efficient for them too. Operational data often has gaps or “white space”. This means your operational data may look good while the customer feedback may be bad. The most common example of this is the difference between the way customers measure delivery time for their orders compared to the way companies measure it. I will cover this in more detail another time. NPS should be used to supplement, correct and prioritize operational metrics, not as a substitute.

Avoid common mistakes

Customers are important to your company. Implementing a feedback system will improve their lives and your financial results. This series of articles will speed up your NPS work and will help you avoid many common implementation mistakes. You will benefit from understanding mistakes I have made and seen others make, and you will learn how they can be addressed or avoided.

Looking forward

My NPS blog series will cover the following subject areas:

  • Refresher on the history and main aspects of the Net Promoter System.
  • Ensuring NPS is a reliable, trusted metric.
  • Feedback, learning and improvement processes — the heart of the system.
  • Implementing a robust operational and analytic infrastructure.
  • An employee and team environment focused on loyalty.
  • Ensuring sustained leadership commitment.

Real-life examples of how to implement well and how to fail will be provided throughout the series.

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

[1] Net Promoter, Net Promoter System, Net Promoter Score, NPS and the NPS-related emoticons are registered trademarks of Bain & Company, Inc., Fred Reichheld and Satmetrix Systems, Inc.

[2] Fred Reichheld with Rob Markey: The Ultimate Question 2.0, Harvard Business Review Press, 2011, ISBN 978-1-4221-7335-0

 

1 2 3 4

↑

Net Promoter™, Net Promoter System™, Net Promoter Score™, NPS™ and the NPS™-related emoticons are registered trademarks of Bain & Company, Inc., Fred Reichheld and Satmetrix Systems, Inc.
CustomerStrategy.net is owned by Maurice FitzGerald Consulting, Geneva, Switzerland • Site developed in conjunction with iCulture and iBusiness

Show Buttons
Hide Buttons