NPS

World map used for blog on cultural differences

NPS (17) – Does culture affect NPS / customer survey outcomes? – 17th in the series on NPS

/ , , / , , , /

This is the 17th article in a series on the Net Promoter Score and Net Promoter System. This time I cover cultural differences in the way people respond to surveys.

As a person who has lived and worked in seven countries, I believe cultural differences exist and matter. My career has taken me regularly to countries in Africa, Asia, Europe and North America and I have directly observed that people behave differently, especially in business situations. A lot has been written on the subject. My favorite book on the subject is When Cultures Collide by Richard D. Lewis [1]. Cultures and Organizations by Hofstede and Hofstede[2] is also excellent. I got into the habit of studying the When Cultures Collide chapter on a new country before visiting for the first time. I suppose my most educational experience was in Jordan.

Comfort zones in different countries

Along with all the science, Lewis tells entertaining stories about the effect of comfort zones on behavior in groups. In Northern European and North American culture, the ‘comfort zone’ extends to arm’s length. What this means is that people from these cultures become uncomfortable when someone is less than a full arm’s length away from them. They will tend to back away from someone the feel has come too close. In Southern Europe, the comfort zone is half that, meaning elbow’s length.

Lewis describes multinational meetings, for example European Union social gatherings, where the Southern Europeans back all the Northern Europeans into the corners of the room by the end of the evening. His chapter covering Jordan mentions that the local comfort zone is half again what you find in Italy. He also mentions what happens when you are being successful in your discussions. He was not wrong.

Opening HP’s office in Jordan

I went to meet the Minister for Information and Communication Technology together with an American colleague. We wanted to negotiate a series of things to do with opening an office and potentially a software center in Amman. My American colleague had not read the Lewis book, and I did not say anything about what I was going to do in advance. I sat just left of the minister. When the time came to have the critical discussion, I moved to about 20 cm/8 inches from his face, and …this is critical… did not lose eye contact at any time while we spoke. I could see my American colleague getting twitchy while this was going on. And yes, the sign of success then happened. The minister put his arm around my shoulders. We had never met before. My colleague’s eyes were popping out. Success! I had learned how to do the male cheek kissing version that is specific to Jordan, but did not get to try it out as we did not socialize beyond the meeting. On a side note, the best coffee I have ever had in my life was in that office. Something to do with flavoring it with cardamom seeds, I think.

Culture matters, but does it affect customer surveys?

After many years using double-blind surveys at HP and seeing responses that were consistently different by country, I became certain that the sole explanation for the differences was culture. Exhibits 3.4 and 3.5 show what the NPS scores looked like by country for the combination HP and its major enterprise hardware and software competitors respectively, by country. This excludes consumer PCs and printers. The data is for a three-year period and the sample sizes are absolutely massive.

 

Enterprise hardware NPS by country for all major vendors

Enterprise software NPS by country for all major vendors

So therefore

The only possible conclusion from this data is that the Japanese, Koreans, Australians and Dutch are grumpy people and don’t want to give top grades to anyone. Right? The only possible conclusion? That is certainly what I thought before going to meet Rob Markey for the first time in August 2015. Before getting into what he told me, think about it this way. The data above represents customer views on the major IT suppliers around the world. Almost all are US companies. Almost all try to do business in other countries exactly the way they do it at home in the USA. Almost all believe that this is the best and only way of doing business and that any other way of doing business is inferior, or perhaps just costs more.

The perception is widespread

Do a web search on cultural differences in Net Promoter Scores, and you will find a consensus that the cultural differences in scoring are real and look something like the graph above. Even Bain consultants used to agree. Back in 2006, Jayne Hrdlicka, Edmund Lin, Gary Turner, Bruno Lannes, YeonHee Kim and Shintaro Hori published How to win—and retain—loyal customers in Asia and Australia . It includes many examples of NPS score ranges from different industries. Most of them have negative average scores. There are also some ‘normal’ scores in the mix, which should not be possible if the cultural paradigms are 100% correct. By this I mean that either ‘all Australians are grumpy when responding to surveys’ or they are not. Both cannot simultaneously be true.

There is data that supports the existence of cultural differences

Per.ceptive (Customer Monitor) is an Auckland-based company that publishes consumer NPS benchmarks for Australia and New Zealand, among other work. Their work on consumer benchmarks seems to support the ‘grumpy Australian’ viewpoint. The only other possible explanation would seem to be that all types of Australian companies don’t care much about their customers. The report in Exhibit 3.6 is from August 2016.

There is limited though reliable data that says the opposite

If the bar graphs really reflected cultural differences, then there would be no benchmark NPS scores in these countries that would show anything different in any industry. Not only is there in fact benchmark data from the ‘bottom four countries’ that has similar scores to those observed in the United States, but some of the data is for high-tech. ING Direct Australia scores a 38 in double-blind sampling. Local IT outsourcing suppliers working in Dutch score in the 30s in the Netherlands, though Dutch Telco KPN has an 11.

OK, so these are cultural anecdotes

Here are a few short examples that gradually changed my views about what was going on:

  • You should adjust to local conditions if you are not from the country. I remember being beside Francesco Serafini, then the HP Managing Director for Europe, Middle East and Africa, when the CEO of Philips Electronics phoned him. They had recently transferred their IT help desk operations to us. Nobody had told the CEO that one way cost had been reduced was by eliminating Dutch as a language for telephone support. He was angry, to say the least. Angry with us and angry with his CFO who had signed the contract and had not told him about the change.
  • According to a former Apple employee I met socially, the Tokyo and Beijing Apple stores had the lowest NPS scores among all stores worldwide, at one point in time. They did deep research into local habits after asking themselves whether the way they did business in New York and San Francisco was appropriate in China and Japan. The first thing to change was the introduction of ‘fast checkout’ for a set of commonly purchased items that were put against one wall. The customers could avoid the friendly-but-lengthy typical Apple experience and get out of the store quickly. NPS scores rose to and even above the company average quite quickly.
  • I met Bill Thomas of EquaTerra at an IT event. EquaTerra had just published the customer satisfaction benchmark numbers for the Benelux for the IT Outsourcing industry. All multinationals (HP, IBM, Accenture) did badly, and many local providers did very well. I asked him to tell me the top reason why. He said it was because all the local providers had their support teams physically in the Benelux and they all spoke to customers in their languages and accents. The multinationals all operated offshore, providing support in English for the Netherlands and Luxembourg, and non-Belgian-accented French for the Belgian customers. He said this was a pattern in other countries too. The multinationals may have had lower costs, but the costs were quickly forgotten after the contracts were signed.
  • A local radio station reported on an esoteric study on how people wanted to be addressed by doctors in hospitals. In the United States and the UK, the vast majority of patients wanted doctors to use their first names. In France none did, preferring their family names. I suspect the answers would be even more extreme here in the French-speaking part of Switzerland where you are expected to know someone before going further than “Sir” or “Madam.” Indeed, in a recent newspaper article a number of Swiss rail passengers were complaining that the staff inspecting their annual rail passes thanked them and used their family names, which they considered totally unacceptable.
  • During a discussion about NPS scores in Australia, a person I met at a Satmetrix conference asked me from where we provided telephone support for Australian customers. I told him it was done from India for the business in question. He told me that many Australian IT companies had tried that, but had moved support back onshore to have people on the phone with Australian accents, despite the cost. HP’s consumer support people did something similar when moving UK telephone support back to the UK from South Africa.

More Bain experience

I went to my meeting with Rob Markey armed with my data and convictions. He persuaded me that I was wrong, at least partly. He described his experience working with an unnamed American company’s operations in Japan. The Japanese operation had had the worst NPS scores in the company for years. To get a new answer, they asked themselves a new question, “What if the problem is not the supposedly grumpy Japanese? What if the problem is us, and the way we do business in Japan?” That was a transformative question. Bain and their client did deep research on the way Japanese customers wanted to be served. They adjusted the way they did business from top to bottom, and it does not look like any other country. Japan now consistently scores in the top three countries on the corporate NPS chart. Going back to the anecdotes, the company in question had had a policy of always addressing customers by their first names, even in Japan. Totally unacceptable. That was one of the first things they changed.

Suggestions

What all this means is that cultural differences exist in responses to NPS surveys, and the reasons may not be entirely what you think. At a pure survey level, yes there are cultures that are reluctant to give you the top score. This is a good reason to use the zero to 10 scale for NPS ratings rather than 1 to 5. People who have a cultural reluctance to give you a ten may still give you a nine. There are cultures where customer-centricity has not evolved to the extent it has in the United States. An American company offering US-style service there may do very well compared to local expectations. This may be what we see happening in Mexico and Brazil in the IT industry, for example. There are other countries where behaving in the excessively friendly American way may be totally inappropriate. If I were to suggest remembering one thing when you see low scores in a country, ask yourself, “What if the problem is not the customers? What if the problem is us?” Answer that question before your competitors and you will win.

Communicating multi-country competitive customer experience scores

If you have competitive benchmark scores and graphs like the ones shown above, you should never present them in the way I have. For most companies, competition is local. It does not actually matter much that you and your competitors average an 8 in Australia and a 50 in India. I suggest showing individual country competitive comparisons without showing the rating scale at all. Show the tick marks on the graph but no numbers, or just relative numbers compared to your score. What you want in Australia is for your NPS trend to be better than that of your competitors, no matter what the numbers. Comparing country scores to each other mainly wastes time and delays getting on with improvements. Only removing the numbers from the scoring scales can help avoid your audience making irrelevant comparisons.

[1] Richard D Lewis: When Cultures Collide, 2005, Nicholas Brealey Publishing, ISBN 978-1904838029

[2] Geert Hofstede, Gert Jan Hofstede and Michael Minkov: Cultures and Organizations, 2010, McGraw Hill Education, ISBN 978-0071664189

 

Learning check

Decide whether each of these statements is true or false. Answers below.

1. Even the font you use in an email asking customers to take a survey can make a big difference.
2. To make sure you are doing a good job, you should always send a survey request to every customer who phones your service center.
3. Embedding at least the rating question in a survey request email improves response rates by about 2.5 times.
4. Culture makes no difference to survey responses, so it is always fair to compare countries’ NPS performance with each other.
5. Low survey response rates tend to give overly optimistic NPS scores.
6. The more questions there are in your survey, the more likely people are to complete the survey, as it is clearer that you value their opinions.

[Answers below.]

Looking forward

The next article in the series will be about the all-important question of whether or not NPS trends predict revenue and market share trends. Deep stuff.

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

Answers to the learning check questions

  1. True. Using a serif font like Times New Roman in emails improves response rates compared to using a non-serif font like Arial.
  2. False. You should only survey if you have the people, funding and plans to act on the input.
  3. True, at least in the author’s experiments. There are software vendors who say that it triples response rates.
  4. False. Do your best to make all competitive comparisons local. Comparing NPS scores between countries is not helpful.
  5. True, and the effect is substantial. You can only consider an NPS survey to be truly representative of the customer population when you have response rates above about 50%. Such numbers are rare for transactional surveys, another reason to be selective when using them.
  6. False. The longer your survey questionnaire, the more people will drop out before completing it. Compared to a 3-question survey, about 6% of the people who start a 20-question survey will not complete it, rising to 10% for a 40-question survey.
Upwards arrow

NPS (16) – Five more survey response rate improvement suggestions – 16th in the series on NPS

/ , , / , , , /

This is the 16th article in a series on the Net Promoter Score and Net Promoter System.

In addition to the direct marketing techniques mentioned in my last article, here are some additional ways of improving response rates to email-based surveys, in priority sequence:

  1. By far the most important thing you can do is allow respondents to respond to the survey directly in the email they receive. Unfortunately, this is technically difficult. The best you can do is to embed the first question in the email, usually the overall rating question. Use HTML widgets to do this. They do not get blocked by the common email clients at least at the time this is written. Embedding the first question improves response rates by a factor of 2.5x, in my personal experience. Promoter.io and other software providers have found that it triples response rates. My suspicion is that embedding the first question makes it far less likely that people believe the survey link could be a phishing attack or other security risk.
  2. Provide feedback to everyone, not just the respondents. When you have completed the survey and determined the actions, write back to the entire mailing list to let them know their voices have been heard and telling them what you are going to do. Those who did not respond are more likely to answer the next time and could also be given a way of adding their voices immediately if they do not feel represented.
  3. Ensure you have an email contact list that you maintain and use regularly for customer communications. That has two main benefits, one obvious and one less so. First, you will have far fewer emails ‘bounce’ due to unknown addresses. The second benefit is that customers who read your emails will almost certainly have authorized the displaying of ‘pictures’ for anything received from your email domain. Outlook and other email clients default to blocking pictures from unknown sources. You should minimize the use of pictures and other graphics in outbound emails in any case, as they present consistency challenges across phone, tablet and PC platforms.
  4. Test your outbound email with a set of internal users first, to detect problems. Send some additional test emails to internal users from a non-company email address, to see how they display if an email client blocks anything or automatically puts the email in a spam folder.
  5. While I have suggested not using strong incentives, simply thanking people in advance, promising to act and to inform them of the results should be a sufficient incentive for many, and should have a positive effect on response rates.

Risk of over-surveying

How do you know whether you are over-surveying? There are two main principles. First, you need to provide more value than you extract, which should push you towards short surveys. Second, you need to be planning to act on the feedback, including informing customers about your progress. Remember that over-surveying is about individuals, not companies. It does not matter if you send surveys to multiple people in a company. What you want to avoid is sending one individual multiple surveys in a short period of time. When thinking of this, bear in mind that your competitors may be sending surveys to the same person too.

Over-surveying may be over-rated as a problem

Let’s face it, if a customer has not responded to your survey request, they have not wasted any time, but you do not know what they are thinking. Most transactional surveys will give you a response rate between 5 and 15%. Product survey response rates can be a little higher. The people you need to avoid asking to respond again in a short period of time are those who already answered your survey. Most of your customers will not have answered. These should be the focus for the following research round, assuming it happens reasonably soon after the previous one. Write something that starts like this:

“Dear Mrs. Smith,

I realize you did not have time to respond to my request to help us improve our product when I wrote to you six weeks ago. The people who did answer helped us to determine the two items below as our top two improvement priorities. I really want to have a third project and would like to once again ask for your help. There are just three questions and this won’t take you more than about 3 minutes…”

Taking this approach improves the overall response rate, and therefore improves reliability. Make sure you don’t send these emails to people who have already answered.

More generally

If you have listened to customers, made improvements, then reported the progress back to customers, you cannot be accused of over-surveying if you start the process again. Customers will be able to see that you have provided at least as much value as you have extracted and will cooperate. It is important to communicate your progress to all customers, not just those who responded.

Looking forward

A big subject the next time: cultural differences. Do they affect survey outcomes? (Yes, they do.) How should you deal with this? It’s a deep subject and will be the longest article so far in the series. And I will include a learning check on survey design and response rate improvement techniques.

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

Raining surveys

NPS (15) – Use direct marketing techniques to improve survey response rates – 15th in the series on NPS

/ , , / , , , /

This is the 15th article in a series on the Net Promoter Score and Net Promoter System. The subject this time is the use of classical direct marketing techniques to improve response rates. By classical, well… yes, I do mean ‘old’.

Like most people, I receive lots of survey requests by email. Unlike most people, I don’t just delete them all. I want to learn. What I usually learn is that most survey designers don’t understand even the most basic direct marketing techniques. Here are some that you may want to consider.

Direct marketing reference source

My direct marketing guru is Drayton Bird, author of Commonsense Direct & Digital Marketing. Most of the techniques for getting people to do what you want them to do have not changed since direct marketing was done by sending people things by snail mail. Here are the most critical points:

Personalize

Make your invitation as personal as conceivably possible. Use the person’s first and last name. (It is better to err on the side of formality. There are cultures where using an adult person’s first name only will turn them off. France is a good example.) If you have additional information about them, like the city where they live, the industry in which they work and so on, use that too. Personalization applies at the outbound source as well. If you are doing an overall brand image/relationship survey that is being sent by email to smaller customers and partners, it should go out from the CEO’s mailbox. Product-specific surveys should go out from the product manager’s mailbox and should start with “Good morning Mr. Doe, I am the product manager for the DoubleBubble Jacuzzi you bought recently at Home Depot…”

Single action request

Only ask the person to take a single action. When I was taught direct marketing, I was taught to think of the recipient holding a stack of letters and standing over a waste bin. The first letter asks the person to do A. She puts it in an action stack on her desk. The second one asks her to do B. She puts it in the bin. The third one asks her to do A or B… She thinks “Wow, A or B, which should I do? I can’t decide.” So she drops it in the bin. Like most aspects of direct marketing, this has been studied using A / B testing; the technique where you change only one thing in a marketing piece and measure what happens. While intuition would tell you that giving a person two choices would double the overall response rate, the opposite actually happens. Your overall response rate halves if you give a person two choices rather than asking them to do a single thing. If you give them three choices, response rates drop to a quarter.

Herd instinct

Appeal to herd instinct. The UK government has a partially privatized unit called the ‘Behavioural Insights Team’, generally known as the ‘Nudge unit’. They have convincingly shown that writing to people who are late paying their taxes in a particular way works. Something like, “Almost every company here in the greater Homeville area has paid their taxes on time. In the last year, these taxes have paid for a new playground on Middle St. and the new recycling centre behind the farmers’ market. Please join your fellow citizens in making Homeville a better place to live.” Of course, all that has to be true.

Since you are unlikely to get half the recipients of an email to respond to a survey before sending a reminder, the reminder could say something like, “Acme customers from all over the greater Chicago area have already answered our three-minute survey and are helping us to improve our lawnmowers. Please add your views to theirs to ensure all of Chicago is represented.” Naturally, this requires knowing that that the person does indeed live in that area. Again, the more personal you can make it, the better.

The correct choice of fonts

For some reason, this one is controversial and when I talk about it, I am usually told I am wrong, at least by the people who are being polite. The others use stronger words. The font you use makes a big difference to response rates. You should always use a serif font for body text in your emails or paper forms. Serif fonts are the ones with little hooks on them and with bases on all the vertical strokes. These extra hooks and bases are the serifs.There is a reason 99.999% of all books are written with serif fonts. These fonts are much easier to read.

Most people will recognize Arial as a ‘sans-serif’ font and Times New Roman as a serif font. The serifs give the eye a natural line to follow. Comprehension is much better than with sans-serif fonts because your eye never accidentally skips to the previous or following line. This is not an issue with short headings. Look around you and you will see that most books and magazines have sans-serif fonts for the short headings, and serif fonts for body text. This has been tested repeatedly in direct marketing, especially by Drayton Bird. The common view that this is no longer the case because so many sans-serif fonts are used on web pages is simply wrong.

Place pictures carefully, if at all

Don’t break up your text with pictures. Drayton Bird’s research shows that a high proportion of people won’t read anything below your picture. If you have to have a picture, it should be at the top or bottom, or perhaps to one side. Note that if you are sending emails to a new distribution list, the customers’ email clients may block the display of images.

The image below is an example of a survey request I received from Thai Airways. My Outlook client blocked the images, though not the button image, while my iPad mail client displayed it correctly. I can’t resist noting that while they know that I took the flight, they don’t know my name. Did you know that airline slang for passengers is ‘Self-loading freight’? Should I be consoled by the fact that they don’t know the name of the EVP who sent the email either? I suppose I should complete my pickiness by adding that you should spell out dates to avoid confusion. Thai has used the US date format for a European customer.

Thai airways survey for blog

Reminders

Send a reminder after two days. When you send a survey invitation by email, you will typically get 90% of the responses within 24 hours. Almost nobody files this sort of thing away for later consideration. They either handle it when they see it, or not at all. Only send the reminder to people who have not already responded. If 100 people responded to the first email, my experience suggests that about 65 will answer a second email telling them there are only 24 hours left to provide input.

Incentives

I suppose I should add a word about incentives. Incentives can improve response rates, but should not be used. Way back in early 1994, I ran a project to launch a Digital-Microsoft software subscription service across Europe. It was a precursor to Microsoft’s launch of a volume licensing program. The main technique used was direct-response advertising. One country project leader (OK, it was Norway) ignored my advice and included an incentive. The incentive was to be entered in a draw for two tickets to the soccer World Cup in the USA that year. Norway had qualified, to their surprise, and the promotion was a spectacular success. Surprisingly, it almost got me fired. A certain Steve Ballmer was leading sales at Microsoft at the time, and he apparently became irate that we had taken 20% market share in Norway with the promotion. The incumbent Microsoft resellers were unhappy and complained bitterly. My boss’s boss’s boss (John Rando at the time) had signed the agreement with Microsoft, came to see me, determined that everything was OK and defended me. Whew! In any case, it proved that incentives work in direct marketing, and have consequences.

Incentives produce positive bias

While that is an extreme case of an incentive, there is a more general issue that has to do with human nature. Providing incentives seems to positively bias responses to surveys. Humans have difficulty believing they can win something in a random prize draw if they give very negative feedback. So, if your intent is just to improve the metrics but not to gain accurate insights, go ahead and use incentives.

A/B testing

Direct marketing introduced the concept of A/B testing. This is where you change just one thing in an email (for example) and compare the results randomly with emails that do not have the change. If you want to test the effect of using a person’s name on response rates, randomly pick half your email list and start with “Dear customer” while starting the other half with “Dear Mr. Doe”. Try another experiment the following time, like using the person’s name in the subject line of the email.

Conclusion

This is not a complete list of everything you can do to improve response rates. It is just the subset that has to do with direct marketing techniques. Feel free to disagree with the above, or add your own suggestions below.

Looking forward

The next article will be the last one on improving response rates. It includes a number of proven ideas that don’t fit into a single category. One of the five is the one that I have personally found to have the strongest positive effect.

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

Normal curve for NPS blog on statistical significance

NPS (14) – 14th in the series – Customer survey response rates and statistical significance

/ , , , , , / , , , , , , /

Welcome to the 14th article in my series about NPS and the Net Promoter System. This time I cover response rates, sample sizes and statistical significance. Perhaps the most important points are about how to communicate such technical concepts. (Hint: Don’t!)

All survey types are sensitive to response rates. The lower your response rate, the less accurate your survey. Bain analysis shows that surveys with low response rates give over-optimistic results, as the people who like you tend to respond more easily than those who do not. Getting high response rates is primarily a function of the nature of the relationship you have with the people you are surveying. If you are using email to try to survey people you have never met, and with whom you have never done business, you should expect a response rate of 3% or less. At the other extreme, if you have a practice of formally interviewing ten executives from your largest client face-to-face every six months, have been doing so for the last five years, demonstrating that you provide the improvements they request, a 100% response rate is a reasonable expectation.

Perverse effect of high response rates

High response rates drive lower scores on a constant scale. In my experience with HP, this meant that our deep face-to-face relationship surveys with 80%+ response rates usually produced Net Promoter Scores that were considerably lower than the scores we saw in benchmark surveys or indeed in any other type of survey. Avoid the temptation to compare scores from different types of survey, or at least from surveys with very different response rates.

Here is what Reichheld and Markey say

My own experience is not an isolated case and others have found similar trends. Based on their many years of experience, Fred and Rob provide some numbers, and I believe the numbers should surprise most people. They show a far greater impact of low response rates than expected. I don’t believe the phenomenon is specific to NPS. The principles should apply to all rating systems. Let’s suppose you have carried out a survey and the results are as shown below.

What about the non-respondents?

The question is, what do the people on the right think? The diagram below is what is shown in The Ultimate Question 2.0. It represents what would have happened had 100% of the recipients responded to your survey.

The theory is that Promoters are the ones most likely to answer your surveys. They like you and want to help. Passives and Detractors are more likely to think you have already wasted enough of their time. Extreme Detractors will still tend to answer. In a conversation with Rob Markey I asked him how they had determined the numbers on the right. He said it was done by observing the actual behaviors of the non-respondents in a large number of studies in a variety of industries.

Surprise!

This surprises most people. It is a very difficult subject to study as you can’t simultaneously get different response rates for the same survey and same audience. What I have seen is that the results from people who take the survey when they receive a follow-up email tend to be lower than for the initial respondents, though I have also seen exceptions. My brother (and this book’s illustrator) has a doctorate in Cognitive Psychology and has not been able to design an experiment that would prove the theory. Nonetheless, the psychology suggested by Fred and Rob makes sense to me. (Feel free to accuse me of cognitive bias!)

Statistical significance

Analysis and determination of statistical significance of results can be challenging. The following statement may seem counter-intuitive: If you want people to trust your metrics, don’t include any information about statistical significance in your main communication vehicles. There are two main reasons for this:

  • There is no single way to explain significance across all feedback systems. Correctly explaining the significance for each type of survey is long and complex and will make people believe you are manipulating the numbers.
  • There are survey types where intuitive significance rules do not apply. For example, if you have 20 customers that represent 60% of your revenue and you have high-quality relationship survey results from 15 of them, you are in great shape. If you try to explain the statistical significance range correctly, some amateur statistician in your audience is bound to say, “There are less than 30 companies in your sample so it can’t possibly be significant.” If you don’t mention significance, there will be fewer opportunities for such people to embarrass themselves. If the question is asked, well, good luck, and be diplomatic.

Improving response rates

Next, let’s look at the key techniques for improving response rates:

  • Application of classical direct marketing practices.
  • Use of technology when email is used to send survey invitations.
  • Work on the quality of your contact list.

Looking forward

Next time I will cover the use of classical direct marketing techniques to improve response rates.

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

 

Survey dramatic interpretation

NPS (13) – 13th in the series – Whether and when to survey

/ , , , , , / , , , , , /

This is the 13th episode in my series on the Net Promoter Score and the Net Promoter System. This one is a bit longer than the average and covers some important topics, including whether you should bother surveying your customers at all. If you have missed some prior articles in the series you will find them on our blog page here.

The first step in designing surveys is being selective about what research you are going to do. Make sure you are going to provide customers with more value than you will extract from them.

Sanity check on whether you should gather feedback at all

Here are some considerations that should help you decide where to start:

  1. Are you going to use any new input you get? If you have already established your top five improvement priorities, have staffed them, are implementing, and can’t add a new project in the next six months, don’t do any surveys other than those that provide progress feedback on your improvement work.
  2. Taking a software example, if you are planning a new major release and want user input on what should be in it, ask them in a formal way. After 30 years in and out of the software business, as a leader and as a user, I have never once been asked for formal input about what should be in the next version. Product managers generally feel they know what users want, based on informal and anecdotal conversations, rather than research. This of course applies beyond software.
  3. Not every customer touchpoint is important. Use your overall brand or benchmark surveys to establish which touchpoints customers mention as competitive advantages or disadvantages, and don’t survey anyone about any of the others. To pick a B2B example, surveying people about the friendliness, competence and efficiency of the person who handles a query about the accuracy of an invoice is certain to be a waste of resources that could better be spent elsewhere.

Timing is everything

Survey timing is an important part of research reliability. Seeking feedback about things that are in the distant past is unlikely to be of much value for example. Here are some appropriate times to consider collecting customer input, by type of survey. What follows presupposes that you are going to use the feedback to make improvements for your customers. If not, there is no good timing.

Product surveys part 1 – Unboxing and installing your product

Here we are talking about an individual product, not a generic purchase or other touchpoint experience that applies to all your products. Some products are sophisticated (the marketing word for complex) to install. Others are not. If your product takes time and effort to unbox or download, install and use for the first time, it is worthwhile understanding what your customers think of the experience and how it can be improved. The research needs to be done immediately after the experience. Waiting a week or more will mean losing the positive and negative emotional impact.

If yours is a technology product, I suggest sending the survey request or phoning the customer within 48 hours of delivery. While this will be easy to do in a direct business that sells to consumers, it is more difficult in other circumstances. You want to survey the end user of the product, and they may not be the buyer in a B2B situation, so you might not know whom to contact.

If you use resellers, they may not want to provide the customer data, and you should offer to run the survey on their behalf, providing them the data for initial screening. If yours is a software product, the installation routine itself should generate the survey request 24 hours after the installation takes place. Requiring or at least suggesting product registration on a website will give you additional sources of end-user contact details.

Product surveys part 2 – Usage

In most situations, you won’t hear anything from a customer who does not have any problems. Unfortunately, the reason for not having problems could be that the customer no longer uses your product. Perhaps they never started using it in the first place. You need to understand the views of customers who have been using your product for some time, and I suggest six months after purchase as a reasonable target for your research. The purpose, as explained to the customer, should be to get feedback for improving the product itself, as well as future products. In my own experience of software product usage surveys, it is common to get suggestions for additional features that the product already has, but the customer has not been able to find. Usage surveys have the same challenge as installation surveys, meaning you may not know whom to contact.

SaaS (Cloud) software is a completely different in that you do know who is using your software and indeed how much they use it. If a customer who has been using your software suddenly stops, contact them immediately. SaaS software usually allows you to see what functions a customer uses. If you can see that they are not using something they really should be using, you can send them a link to a training video or send over someone who can talk them through it.

Service contracts

The ideal time to get feedback on a service contract is a little before the customer is considering renewing it. This is a bit more subtle than it might seem. If you have a large multi-year contract (for example to provide security services to a company) your competitors will probably know about it and will start to contact your customer well before the contract renewal date. Since your competitors will be saying how wonderful they are, and how awful you are, you need to have acted well before this point. I suggest an annual survey as a minimum for multi-year contracts, taking place half way through each year. If you have a single-year contract, I suggest doing the research about three months in, so that all improvement suggestions can be implemented before the contract comes up for renewal consideration, probably about nine months in.

Hybrid product / service

An airline flight is a hybrid product & service. This sort of offer should be researched within 24 hours of completion. Most airlines seem to miss the point. Some give you the survey while you are still in the air, not knowing essential things like whether your luggage will show up on the carousel. Others send you the survey after a week or two. Frequent fliers will have completely forgotten about the experience by then or will confuse it with another flight. This happened to me regularly, as I have been taking over 100 flights per year for nearly as long as I can remember. What did I think of my flight from Atlanta to Paris two weeks ago? Who knows? Since product and service surveys are not mentioned in The Ultimate Question 2.0, this may be a good time to reinforce the definition. An entire flight experience, from awareness, to reservation, to the actual trip itself, involves a sequence of touchpoints. Each touchpoint could be studied by a transactional survey. The airline offers many different products and services, the sum of which might be studied in their brand survey.

Relationship surveys and Brand surveys

Twice a year is a good frequency for your most important customer relationships and may work best for your brand surveys. You may like to run them both on a continuous basis so you can see the effects of things like changes in the economy and your financial and product announcements. This would mean, for example, sampling one sixth of your customers every month. Remember that you need to report results and improvement projects to customers as well as to your own team.

Transactional surveys

While it might seem obvious that they should happen shortly after the transaction takes place, it is surprising how rarely this happens. All too often, I get pop-up windows when I have just started an online transaction. I dismiss the window, and then don’t think about it anymore. For the simplest transactions, surveying just after they are complete seems appropriate. For more complex things, like telephone support for a technology product, it seems appropriate to wait for 24 hours, as it may take a little time for the customer to know whether the issue has been completely resolved.

Conclusion

I hope this helps you to decide whether you should survey customers at all, and if you decide to do so, when it should be done. It is tricky to get it right.

Looking forward

Next time I will cover response rates and statistical significance.

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

 

#100 – 100th issue – When is it OK not to invest in CX? – NPS blogs – Infographics – 100 ideas

/ , , , , / , , , , , , /

 

Welcome to the 100th Customer Strategy newsletter! The five topics this week are:

  1. 100 newsletters already?
  2. When is it OK not to bother with improving customer experience?
  3. Latest blog posts – New series on NPS continues
  4. Notable items from other sites – CXO Infographics, 100 customer-centric companies
  5. Looking forward

100 newsletters already?

Yes, this is indeed the 100th Customer Strategy newsletter. I started the newsletter, blog, and books with a simple idea in mind: to do a sort of ‘mind dump’, meaning to transfer a lot of knowledge and experience from my brain to electronic and physical media. I imagined that I would then be able to forget about it all and concentrate on being retired.

It has not worked out that way.

I thought the principle was sound. I started my career in the clothing industry. When I left it, I felt I was an expert in two areas: logistics and stonewashing. Digital Equipment Corporation (DEC) hired me for my logistics skills, so no mind dump was necessary there. However, I felt I had innovated considerably with the various chemical and physical processes involved in washing jeans. When I stopped working for the Wrangler jeans parent company I felt a need to preserve that knowledge. So I wrote it all up, photocopied it, and sent it to my colleagues and to various internal and external laundries and chemical companies with which I had worked. That was in 1985. I can say that I never thought about it all again from that point onwards.

My latest mind dump worked as far as my work on cost reduction is concerned. It has not worked out that way for customer experience and I think the reason is clear: we are all constantly surrounded by customer experiences every day. As a professional, it is hard not to observe what is happening and to think about what companies and government institutions could do better. So I keep on writing and posting content, though I feel I have already offered all the original thought that was in my head. For the last six months I believe I mainly reinforce points I have already made, sometimes by commenting on the work of others.

Our books continue to do well. June 2019 was the best of the three months of June we have had since launch. I have also been fortunate enough to be asked to speak to live audiences from time to time, and that is quite rewarding. The only work I do from which I don’t get any particular satisfaction is managing the 104,000-member LinkedIn Customer Experience and Artificial Intelligence group. Ever since LinkedIn removed certain management features about 18 months ago it has been impossible to stimulate interesting discussions or indeed any discussions at all. I think I will probably stop working on it.

I feel comfortable continuing with the books, newsletters, blogs, and speaking for now, and hope you feel the same way, faithful readers.

When is it OK not to bother with improving customer experience?

Those of you who expect me to answer “Never!” are going to be disappointed. Business strategy is about winning. Winning is about deciding how to deploy your scarce resources so that you can win. Yes, you need to decide whether your plan is to win in the short term or over time. Whichever time horizon you choose, the nature of your decisions is the same: what investments will give you the ROI you need in the timeframe you need it?

Try to think about the question this way: Customer experience investments are based on the principle that if you make your customers happier than those of your competitors your business will grow at a faster rate than your competitors’ growth rates. So, if you are profitable and growing faster than your competitors in your industry you are by definition making your customers happier than the competition is. This is true whether or not you have any formal customer experience initiatives or a CX team or indeed any processes that are explicitly designed to improve customer experience. So… if everything is going just fine, other investments may make more sense in your current business situation.

While my point is a general one, I think the situation is most likely to arise with smaller companies. Larger companies tend to become more ‘average’, and almost all will probably need to invest in customer experience improvement projects.

Yes, I enjoy being provocative from time to time. The above is my attempt for this 100th newsletter.

 

Our latest blog posts

The posts on this list are part of my extensive and deep series about the Net Promoter System. Older posts are still available on the blog page.

image NPS (12) – 12th in the series – Should I weight my survey results?

This is the 12th episode in my series on the Net Promoter Score and the Net Promoter System. There is a learning check / quiz at the end of the post. Suppose your research population does not match the actual profile of your customers. What should you do, if anything?
image NPS (11) – 11th in the series – The law of small numbers

Continuing the discussion on Daniel Kahneman’s System 1 (intuitive fast thinking) and System 2 (rational slow thinking), some curious things happen when people study surprising data…
image How and why you should make it as easy as possible for customers to complain

Complaints are bad, right? Well… sort of. The complaints you know about are valuable feedback and give you an opportunity to recover and retain your customers. It seems logical that you should do everything to minimize the number of complaints you receive. I disagree…
image NPS (10) – 10th in the series – How to avoid confirmation bias when analyzing research results

In his book Thinking, Fast and Slow [1], Daniel Kahneman describes two ways our brains function. He refers to them as ‘System 1’ and ‘System 2’. In this post I cover the way our minds mess us up when we analyze research results.
image NPS (9) – 9th in the series – How to use the answers to the open questions

While our rational minds like to see scores and trends, the primary value of Net Promoter System surveys lies in the answers to the “Why?” and “What should we improve?” questions…
image NPS (8) – 8th in the series – I have my NPS number. Is it good or bad?

“OK, I have my NPS number. Is it good or bad?” There are two parts to the answer. First, the score does not tell you much on its own…

Notable customer experience items from other sites

CXO infographics by Annette Franz

Infographics can be a powerful communication tool. When I see them in office settings they often make me stop, look, read, and sometimes understand something new. Annette Franz is one of the world’s leading customer experience experts and she has collected 263 infographics on Pinterest. There are a few duplicates, and I expect that the number of unique entries is about 10% fewer. One thing she did not explain is what the ‘O’ in CXO means. Optimization? Organization? Any guesses?

Have a look here.

Habits of 100 customer-centric companies – Blake Morgan for Forbes

Blake Morgan contributed this interesting article to Forbes. She picked 100 companies that have done well in the American Customer Satisfaction Index reports, in the Temkin Experience ratings and elsewhere. Blake then tries to pick out something interesting and relevant to say about each. I have to admit that about a quarter of the statements are too generic to be useful, though there are some good ones on the list as well. If a reader would like to put the article through their text analytics software I would be interested to learn about any consistent messages. Please let me know.

Read Blake’s article here.

 

Looking forward

100 and counting. Will we make it to 200? Who knows?

Here are links to all of our books on Amazon.com. Kindle versions are available in all stores. Print versions are available from the major stores only. And as of two weeks ago, you can find the books, or at least order them in many bookstores. If you have already read any of our books, please write reviews on Amazon.

Customer Experience Strategy – Design & Implementation

Net Promoter – Implement the System

Customer-centric Cost Reduction

“So Happy Here”: The Absurdist but Essential Guide to Better Business (Color edition)

“So Happy Here”: The Absurdist but Essential Guide to Better Business (Black & White edition)

Please share this newsletter with your friends and colleagues and encourage them to sign up for it here. I have put links to past newsletters on the subscription page. Finally, please feel free to change or cancel your subscription using the link below.

You can also email me, Maurice FitzGerald, at mfg@customerstrategy.net.

To change your subscription, click here.

 

Scales for article on weighting research results

NPS (12) – 12th in the series – Should I weight my survey results?

/ , , , , , / , , , , , /

This is the 12th episode in my series on the Net Promoter Score and the Net Promoter System. There is a learning check / quiz at the end of this post.

Suppose you have performed an NPS survey and feel the data set is not representative of your customers. What should you do?

A common issue, rarely addressed

Throughout my customer experience career, I have never adjusted my data sets before or after analyzing them. This has always made me somewhat uncomfortable. Think about these examples:

  • Your competitive benchmark survey provider uses a panel of people in countries around the world. You do 60% of your business in the USA, 15% in Canada, 10% in the UK, and smaller percentages in over thirty countries. The survey provider panel is 40% in the USA, 10% in Canada, 15% in the UK, 10% each in France, Germany and Australia, and 5% in another twenty countries. Should you take the survey provider data and re-weight it to reflect the profile of your company? At HP, we did not have any data for Japan the year of the earthquake and tsunami. That had a material impact on the worldwide averages, but we did not call it out in the reports. Was this some subconscious decision because leaving out the Japanese data made the overall results slightly better?
  • You run project completion surveys for your consulting company. Customers pay vastly different prices for different types of projects, varying by a factor of 100:1. Should you weight the survey responses by revenue, giving 100 times the weight to a response for an expensive project?
  • Your relationship survey covers two CxOs, three other decision-makers, five decision-influencers and ten end users. Should you weight the survey so the responses from CxOs count more?
  • You run transactional surveys after calls are closed in your service center. You code calls by their level of severity, into critical, high, medium and low. Should critical calls have more weight in your survey? If 10% of your calls are in the critical category, but only 5% of your survey responses, should you double the weight of those survey responses?

No consensus, major consequences

There is no consensus on the topic. The reason is that it is difficult to take an unbiased decision when adjusting data sets. This is of course not just the case for customer experience data. The New York Times published an article on the subject in September 2016, while the Trump-Clinton contest was in its final stages. The Upshot gave 867 voter intention responses to four different highly respected pollsters and studied the data themselves. Each person or team studying the data adjusted it using their own methodology. Adjustments were made to reflect race, gender and age to reflect the population of Florida, source of the data. Further adjustments were made to reflect likely voting intention. For example, if Hispanics are less likely to vote than Whites, some adjusted the poll results to reflect it. Others considered whether the people polled said they had voted in prior elections. The variations in methodology produced a greater variation in results than would be expected only from sampling error. The difference in views varied by five points. One showed Trump winning by one point; another showed Clinton winning by three points. History tells us that Clinton won the popular vote by two points.

Conclusion

I am not against adjusting an analysis to force data to better match your customer profile. I am against doing such adjustments differently each time, or having different principles for different types of surveys. Either you adjust to reflect customer revenue profiles, geography and other factors, or you don’t. You need to be consistent. If you have multiple businesses in your company, the same principles should be used by all, all the time. If you use a benchmark survey provider that customizes a solution specifically for you, you should agree response quotas in your most important customer dimensions. Do it once, then avoid the temptation to change it for a reasonable period, say three years or so.

Learning check

Decide whether the following statements are true or false. Answers at the bottom of the page. The questions refer to topics that I have covered at some point between the third article in the series and this twelfth article.

  1. If your competitive benchmark NPS number is better than that of your main competitor, you will take market share from them moving forward.
  2. Good NPS scores start at 30, great scores at 40, and you are world-class at 50.
  3. Consistent trends in numbers that are based on small sample sizes are always useful. You do not need sample sizes for all competitors to be the same, for example.
  4. All customers are considered equal by benchmark survey suppliers, so customers who spend a lot have the same weight as those who do not spend much.

Looking forward

Next time I will start the general topic of survey design with an article about when and whether to survey.

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

Learning check answers

  1. False. What matters is the trend relative to competition. If your number is higher and has stayed flat, but your competitor’s number has improved, they will take share from you in the future.
  2. False. A good NPS score is one whose trend is better than the trend of your main competitor. A world-class score is one that is better than the competition and improving faster. The absolute number has no importance.
  3. True. You may want to treat a single outlier result as suspicious if the sample size is smaller than for your other competitors or countries, for example. However, if you see similar numbers in sequential sampling periods, you should trust the trend.
  4. True, though the survey vendors also supply demographic data so you can do your own weighting if you so wish. However, when vendors such as Temkin publish tables on their websites, there is no weighting.
Law of small numbers illustration for blog

NPS (11) – 11th in the series – The law of small numbers

/ , , , , , / , , , , , , /

Continuing the discussion on Daniel Kahneman’s System 1 (intuitive fast thinking) and System 2 (rational slow thinking), some curious things happen when people study surprising data. As I write this article in June 2019 there are media reports in France of exactly what I am about to describe, a surprising ‘cluster’ of cancer cases in a relatively small area. This just reinforces the message I wrote a couple of years ago (see below) about similar American and Swiss studies.

Kidney cancer in the United States

Daniel Kahneman asked his readers to consider the example of the data for the incidence of kidney cancer in the 3,141 counties of the United States. “The counties in which the incidence of kidney cancer is lowest are mostly rural, sparsely populated, and located in traditional Republican states in the Midwest, the South and the West. What do you make of this?” I have recently seen a similar type of study here in Switzerland. Our rational minds immediately get busy trying to analyze why the geography or being Republican might cause cancer. For the Swiss study, I worked out that it was probably the healthy rural lifestyles that provided the benefit. Fresh air, fresh vegetables, plenty of exercise. How could that not be the reason?

What about the worst places?

However, the counties with the highest incidence of kidney cancer had exactly the same profile. The somewhat similar Swiss study concentrated on this aspect. The TV news report interviewed a person who talked about the use of pesticides in agriculture, various treatments in the vineyards and so on. Seems perfectly logical to think that the rural population would be far more exposed to them. In addition, the rural populations are poorer and probably have more difficulty getting high-quality health care. There were discussions of ‘cancer clusters’ in areas with the highest rates.

The key words were…

In reality, the most important words in the description of the counties with high rates of kidney cancer were ‘sparsely populated’. Counties in the United States have populations that range from under 500 to over 10 million. According to www.kidneycancer.org, the lifetime likelihood of anyone contracting kidney cancer is 1 in 63. In a small county, it does not take many cases for the percentage to be unusually high. The general randomness of cancer over the entire country means that there will of course be some small counties, with quite small numbers of cancers, but with relatively high or relatively low incidence percentages. The extremes are what grab the headlines. Next time you see a story about something like scientists studying why people in some small village live such long lives, treat it with suspicion. A similar village just down the road may have a population with an unusually short average lifespan. That’s statistics!

Use care when presenting your data

Customer experience data commonly has small numbers. If, for example, you are presenting NPS scores for 40 countries, it is very likely that both the highest and lowest scores will be from countries with relatively small sample sizes. This has little to do with statistical significance. If you are looking at enough items, each with 90% confidence, one in ten is just going to be a relatively poor reflection of reality. I used to present NPS benchmark scores for 45 enterprise software companies each quarter. It was very common to see a new small company top the list for a single quarter. If trends from prior quarters did not support this, I told my audience we should wait until the following quarter to react. If the prior quarterly trends did support it (as was the case with ServiceNow’s gradual move from awful NPS scores to great ones), I proposed immediate action.

Nobel laureates by per capita

Another entertaining example and research question would be the following: Why do the Faroe Islands have the highest number of Nobel Prizes by capita? Let’s send out a research team to study their education system and copy it everywhere! In fact, the reason is that they have had one Nobel, and have a population of just over 48,000.

Regression to the mean

If you have loads of data and a consistent trend for something, such as transactional NPS scores for a call center, be careful how you react when you see a substantial short-term change from the trend. To give an analogy, behavioral psychologists generally agree that punishing children for unusually bad behavior does not work, and can even have negative long-term effects. (These psychologists believe that rewarding good behavior works far better.) However, most parents believe punishment does work, and works well. The reason for this belief is regression to the mean. The child has a certain average behavior, trending in a certain direction. They then do something that is unacceptable, and outside the prior average. The parent punishes them and the behavior returns to the prior average. That would have happened in any case, even without any intervention. That is, it will regress to the mean. The only time you have act in new ways is when the trend has actually changed (as it does in fact do if you reward good behavior over time). If you are constantly seen to reward or punish people for what are really single random deviations from their normal trend, you will lose credibility.

Looking forward

Next time I will discuss whether you should adjust or ‘weight’ your research results in any way before you report them. It’s a tricky subject without simple answers. I hope I can reduce the complexity for you. And that post will also include a ‘true or false’ learning test.

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

Feature image for No No No complaints blog

How and why you should make it as easy as possible for customers to complain

/ , , , , , / , , , , , /

Complaints are bad, right? Well… sort of. The complaints you know about are valuable feedback and give you an opportunity to recover and retain your customers. It seems logical that you should do everything possible to minimize the number of complaints you receive.

Are you hearing all complaints?

But what about the customers who have a complaint and don’t tell you about it? But they tell their families, friends and colleagues at work. You have no opportunity to recover because you don’t know anything about the complaint.

Maybe this is such a small problem that you should not worry about it. Or maybe it is huge. I would like to suggest the latter is true, based on Bain research. The key points are shown in a graph on page 111 of the hardcover version of The Ultimate Question 2.0 by Reichheld and Markey, and explained in the text. The message is that the vast majority of people who do not respond to your survey requests are either Detractors or Passives. If your survey response rates are less than 100% your Detractors miss out on the opportunity to tell you what is making them so unhappy. They probably think you have wasted enough of their time already and just leave you for one of your competitors.

That’s not good.

Make it easy to complain

In short, I believe you should make it as easy as possible for your customers to complain. I have recently come across a startup that is doing just this. Jaakko Timonen (with whom I have no commercial relationship of any kind) founded his company No No No with the objective of ensuring that consumers can complain easily and get a response. Customers can log a public complaint about any company that is active in the United States here. The complaints are routed to the companies concerned. Their incentive to act is the simple fact that customers have the opportunity to publicly rate and comment on the issue resolution.

Think of it as an online version of the ombudsman function that exists in certain industries. An ombudsman is a person or office that is supposed to be an agreed neutral party in a dispute. They are common in the insurance industry, for example, and their role is to facilitate an agreement between the two sides. No No No does the same thing, at scale. At first sight, the Better Business Bureau website seems to offer a similar service. The difference between the two approaches may seem subtle, but I find it significant. The Better Business Bureau site allows you to log a complaint about any business but does not provide any way for you to rate the resolution of your complaint. I feel that making customer views about the effectiveness of a resolution is important. How else can companies compare themselves with their competitors?

At a more personal level – what are some tips to ensure your own complaints about a company get resolved?

A fascinating side-benefit of Jaakko’s work is that he has been able to study the keywords in a complaint that seem to most stimulate rapid resolution. The two that seem to have the greatest impact are the use of the word ‘you’ in the subject line, and mentioning a competitor in the main text of the complaint. Both improve resolution rates by a factor of five.

The technology is also available to individual companies. Bearing in mind what I have written above, if you owned the Penn Hotel, would you prefer the upper or lower customer feedback solution below?

 

For more about the excellent work Jakko and No No No are doing, please have a look here.

 

Images above are from the No No No website and are included with permission.

If you like this style of writing, think about buying one of our books. All are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

Elephant to illustrate confirmation bias

NPS (10) – 10th in the series – How to avoid confirmation bias when analyzing research results

/ , , , , , / , , , , , , /

In his book Thinking, Fast and Slow [1], Daniel Kahneman describes two ways our brains function. He refers to them as ‘System 1’ and ‘System 2’. At their most superficial, you can think of System 1 as intuition, and System 2 as rational thought. He offers persuasive arguments that System 1 acts first, and that System 2 is lazy, and does its best to avoid work.

Relevance to NPS

For NPS to be a trusted metric, it must work for both System 1 and System 2. The Ultimate Question 2.0 concentrates on System 2. This book’s content on survey design, improving response rates and things like the automated analysis of verbatim responses all appeal to System 2. I consider the System 2 work to be necessary but insufficient for success. You also have to appeal to the fast-acting intuitive reactions of System 1 thinking. The following are some items for you to consider.

Sequence of presentation

Kahneman describes an experiment set up by Solomon Asch [2]: subjects were given descriptions of two people and asked for comments on their personalities. Here they are. What do you think of Alan and Ben?

• Alan: intelligent, industrious, impulsive, critical, stubborn, envious.
• Ben: envious, stubborn, critical, impulsive, industrious, intelligent.

Most people describe Alan far more positively than Ben. Because the first word used to describe Alan is ‘intelligent’, the more negative character traits are considered to be justified. However, the words used to describe Alan and Ben are identical. Only the sequence has changed. The first thing that is presented has a disproportionate effect on your message, and sets the tone for all the rest. For example, if the overall message you want to communicate is that your company is making good progress, don’t start with the only negative story. When you are going to talk about what you need to improve and what is going well, start with what is going well. Of course, if you have just taken over the customer experience responsibility, you may want to be Machiavellian and start with the bad news, implicitly blaming it on the prior leadership. You could then reverse the sequence for the same information a few weeks later, and people will think you are making progress. Naturally, if you do this blatantly or often, you will lose trust.

Jumping to conclusions

We all jump to conclusions without being aware of it. Kahneman offers the example, “Ann approached the bank” to illustrate this point. You almost certainly formed an image of a woman walking towards a bank, possibly to deposit money. However, the statement is ambiguous. If we had also been told that Ann was on a canoeing trip, our conclusion would be different and we would see her paddling to the side of the river.

A lot of customer experience information is ambiguous. People’s System 1 will jump to conclusions and System 2 will sleep, unless you force it into action. You have probably heard of the metaphor of blind people trying to identify what they are touching, when each is able to feel just one part of an elephant. Turn this on its head and you have confirmation bias. If your audience has been primed to think, for example, that customer service is awful, they will use whatever they can in your data to support that conclusion, and may not even realize they are doing so.

How to get System 2 to work

In other words, how do you get people to react totally rationally to the NPS numbers you present and the financial business case you may be making? The sad truth is that you can’t do much about it. Your best solution is to force your System 1 to look at what you are going to present. Think in terms of someone who is going to ignore most of the data. What are they going to see when they look at your slides? What are they going to hear when you speak? Their intuitive, emotional reaction will win out. System 2 will act only for brief periods. Plan on it.

There is some science that suggests that asking people to do a simple mathematical puzzle that requires System 2 to get the correct answer will help engage System 2, at least for a brief period. Priming people by giving them a puzzle to do in a break, or just having it on the screen before your presentation may help. There is also evidence that making something difficult to read forces System 2 to engage. In any case, you should kick off by using a real customer quote or example, appealing to your audience’s emotions before you start showing data.

[1] Daniel Kahneman: Thinking, Fast and Slow, published by Farrar, Straus and Giroux, 2011, ISBN 978-037427631.

[2] The full set of Asch’s experiments in this area is described in the article below. The example used is “Experiment VI” in the article.
http://www.all-about-psychology.com/solomon-asch.html.

Looking forward

The next article in this series will examine ‘the law of small numbers’, an important thing to consider when you have small sample sizes.

 

As is often the case, the above is a slightly-edited version of a chapter in one of our books; in this case Net Promoter – Implement the System All of our books are available in paperback and Kindle formats from Amazon stores worldwide, and from your better book retailers.

«< 2 3 4 5 6 >»

↑

Net Promoter™, Net Promoter System™, Net Promoter Score™, NPS™ and the NPS™-related emoticons are registered trademarks of Bain & Company, Inc., Fred Reichheld and Satmetrix Systems, Inc.
CustomerStrategy.net is owned by Maurice FitzGerald Consulting, Geneva, Switzerland • Site developed in conjunction with iCulture and iBusiness

Hide Buttons