Temkin Experience Rating

This is the 11th article in a series about customer experience strategy, and the sixth in a subset about CX measurement and improvement systems.

Various consulting companies propose proprietary measurement systems. The Temkin Group, for example, has both the Temkin Loyalty Index and the Temkin Experience Rating. They are useful examples as they both have one thing in common with the systems proposed by other consulting companies: they are composite metrics, aggregating customer responses to multiple questions. Since they are good examples, here are the details of what Temkin proposes.

Temkin Loyalty Index

From the Temkin website, “The Index is based on evaluating consumers’ likelihood to do five things: repurchase from the company, recommend the company to others, forgive the company if it makes a mistake, trust the company, and try the company’s new offerings.”

Temkin Experience Rating

Again, from the Temkin website, “The Temkin Experience Ratings are based on evaluating three elements of experience:

• Success: How well do experiences meet customers’ needs?
• Effort: How easy is it for customers to do what they want to do?
• Emotion: How do customers feel about the experiences?”

The Temkin Experience Rating report is available free each year on the Temkin site. An Excel file with all the data costs $395. Within the report, they state “To improve customer experience, companies need to master four competencies: Purposeful Leadership, Compelling Brand Values, Employee Engagement, and Customer Connectedness.” It should be obvious and it is indeed stated that customers would never come up with such a list. The improvement suggestions come from a completely different study. The Temkin Experience ratings are calculated using an average of the answers to three rating questions. Customers are not asked what should be improved. There is no claim that you can predict revenue from the experience ratings, though there is such a claim for their Loyalty Index. The diagram at the top of this article shows the questions asked, copied from the Temkin site with permission:

It is not at all evident that these are the perfect questions to ask in a study of customer experience. I don’t dispute that they are valid questions. I would just find it difficult to explain and defend them in front of an audience.

Compaq and HP: The Account Loyalty Index

The Account Loyalty Index was a metric from HP and Compaq’s deep-relationship survey and is now part of the HPE Relationship Assessment Process. It is a composite of three individual metrics and is an excellent predictor of customer spending. All three are on a 1 to 5 scale and they are averaged to come up with the ALI number. The component metrics are ‘Overall satisfaction’ (CSAT), ‘Likelihood to repurchase’ and ‘Likelihood to recommend’. ALI scores of 3.5 or more always lead to increased spending. We loved the metric, though things gradually got more complicated. When HP merged with EDS, EDS brought its own composite metric from their Voice of the Client process. We never merged the two measurement processes, as they served different purposes. The EDS business was mainly long-term outsourcing contracts. VOC measured customers’ views on the performance of individual agreements, while the Compaq / HP process was at the overall relationship level. The ALI and VOC metrics both took quite some time to explain to new audiences. The HP PC and Printer business had a different set of high-spending customers and its own different process and metrics. Once Meg Whitman came in as CEO, she found this all too confusing and imposed the Net Promoter System throughout the entire company.

Advantages and disadvantages

The use of composite ratings has advantages and disadvantages. The principal advantage of well-designed composite metrics like the Temkin Loyalty Index is that they should be better predictors of future market share than metrics based on answers to a single question. The major disadvantage of composite metrics is that they are difficult to explain and understand. It is hard to overstate how important that difficulty is. If you are not able to explain your metric in 10 to 20 seconds, I don’t believe it will ever be completely trusted. Composite metrics all take too much time to explain. Unless the people to whom you are presenting work in customer experience full-time, you will have to start every presentation by explaining it again, as they are really difficult to remember. Too many audiences will start to doze off while you are explaining how the metric works. Those who have heard it before or who think they already understand it will just start to look at their phones or do their email while you are speaking.

Conclusion and personal opinion

The following is a personal opinion. You may have a different experience, and I would enjoy hearing about it if so. It is also possible that my views arise from the communication difficulties that are typical in large organizations. Anyway, here goes:

The relative accuracy of composite metrics is irrelevant. They are close to impossible to communicate concisely and consistently. Do not use them.

Feel free to agree or disagree below.

This article is a slightly modified version of a chapter in Customer Experience Strategy – Design & Implementation, one of four books written in collaboration with my artist / cognitive psychologist brother Peter. The others are Net Promoter – Implement the System, Customer-Centric Cost Reduction, and a book of Peter’s illustrations, with learning points: “So Happy Here” – The Absurdist but Essential Guide to Better Business.

Next time: Journey Mapping

Various consulting companies propose proprietary measurement systems. The Temkin Group, for example, has both the Temkin Loyalty Index and the Temkin Experience Rating. They are useful examples as they both have one thing in common with the systems proposed by other consulting companies: they are composite metrics, aggregating customer responses to multiple questions. Since they are good examples, here are the details of what Temkin proposes.

Temkin Loyalty Index

From the Temkin website, “The Index is based on evaluating consumers’ likelihood to do five things: repurchase from the company, recommend the company to others, forgive the company if it makes a mistake, trust the company, and try the company’s new offerings.”

Temkin Experience Rating

Again, from the Temkin website, “The Temkin Experience Ratings are based on evaluating three elements of experience:

• Success: How well do experiences meet customers’ needs?
• Effort: How easy is it for customers to do what they want to do?
• Emotion: How do customers feel about the experiences?”

The Temkin Experience Rating report is available free each year on the Temkin site. An Excel file with all the data costs $395. Within the report, they state “To improve customer experience, companies need to master four competencies: Purposeful Leadership, Compelling Brand Values, Employee Engagement, and Customer Connectedness.” It should be obvious and it is indeed stated that customers would never come up with such a list. The improvement suggestions come from a completely different study. The Temkin Experience ratings are calculated using an average of the answers to three rating questions. Customers are not asked what should be improved. There is no claim that you can predict revenue from the experience ratings, though there is such a claim for their Loyalty Index. The diagram at the top of this article shows the questions asked, copied from the Temkin site with permission:

It is not at all evident that these are the perfect questions to ask in a study of customer experience. I don’t dispute that they are valid questions. I would just find it difficult to explain and defend them in front of an audience.

Compaq and HP: The Account Loyalty Index

The Account Loyalty Index was a metric from HP and Compaq’s deep-relationship survey and is now part of the HPE Relationship Assessment Process. It is a composite of three individual metrics and is an excellent predictor of customer spending. All three are on a 1 to 5 scale and they are averaged to come up with the ALI number. The component metrics are ‘Overall satisfaction’ (CSAT), ‘Likelihood to repurchase’ and ‘Likelihood to recommend’. ALI scores of 3.5 or more always lead to increased spending. We loved the metric, though things gradually got more complicated. When HP merged with EDS, EDS brought its own composite metric from their Voice of the Client process. We never merged the two measurement processes, as they served different purposes. The EDS business was mainly long-term outsourcing contracts. VOC measured customers’ views on the performance of individual agreements, while the Compaq / HP process was at the overall relationship level. The ALI and VOC metrics both took quite some time to explain to new audiences. The HP PC and Printer business had a different set of high-spending customers and its own different process and metrics. Once Meg Whitman came in as CEO, she found this all too confusing and imposed the Net Promoter System throughout the entire company.

Advantages and disadvantages

The use of composite ratings has advantages and disadvantages. The principal advantage of well-designed composite metrics like the Temkin Loyalty Index is that they should be better predictors of future market share than metrics based on answers to a single question. The major disadvantage of composite metrics is that they are difficult to explain and understand. It is hard to overstate how important that difficulty is. If you are not able to explain your metric in 10 to 20 seconds, I don’t believe it will ever be completely trusted. Composite metrics all take too much time to explain. Unless the people to whom you are presenting work in customer experience full-time, you will have to start every presentation by explaining it again, as they are really difficult to remember. Too many audiences will start to doze off while you are explaining how the metric works. Those who have heard it before or who think they already understand it will just start to look at their phones or do their email while you are speaking.

Conclusion and personal opinion

The following is a personal opinion. You may have a different experience, and I would enjoy hearing about it if so. It is also possible that my views arise from the communication difficulties that are typical in large organizations. Anyway, here goes:

The relative accuracy of composite metrics is irrelevant. They are close to impossible to communicate concisely and consistently. Do not use them.

Feel free to agree or disagree below.

This article is a slightly modified version of a chapter in Customer Experience Strategy – Design & Implementation, one of four books written in collaboration with my artist / cognitive psychologist brother Peter. The others are Net Promoter – Implement the System, Customer-Centric Cost Reduction, and a book of Peter’s illustrations, with learning points: “So Happy Here” – The Absurdist but Essential Guide to Better Business.

Next time: Journey Mapping