Surveys

When your employees are measured individually on the results of surveys, ‘gaming the results’ is common and easy. What follows may seem a bit cynical, but has its basis in reality. If you are measured on survey results and want to manipulate them, here is how:

How to game transactional surveys:

• First ensure the person you are speaking to is happy, then ask them to take the survey. This is probably easiest where a transaction is taking place by phone and the script requires the employee to ask whether the customer is willing to take a survey. Simply omit to ask the unhappy ones. This can be risky for your service center people if asking is mandatory and all conversations are recorded “for training purposes”.
• Sending system-generated emails with survey requests is common at the end of service center calls. If you have the authority to update customer records, and the customer is unhappy, simply change their email address before the survey goes out to something that is made up and will bounce.
• If you really need a bump in scores, replace the customer’s email address with one that belongs to you. Answer the survey on the customer’s behalf, giving yourself a top rating.
• Simply tell the customer that your pay and job security depend on them giving you the maximum rating. Car dealerships do this all the time, so it must work. Since Uber drivers and passengers both benefit from a top rating of five, ‘5 for 5’ bartering is common.

How to get whatever result you want from relationship surveys:

• Only ask your friends at the customer or partner to take the survey.
• Pay no attention to their organization chart to find out who actually matters, favoring the people you have met.
• Decide who should be asked about a major project that has been completed only after it is complete. That way you can avoid surveying anyone who has negative views. Make certain to avoid specifying the people to be surveyed in the contract before you start.

How to game your company’s product survey system:

• Only ask members of your user group or another enthusiastic club of people about your product.
• Exclude anyone who has a product escalation or service event in process from your survey list. Justify it by saying that you really should not bother them.
• Exclude anyone who bought a software product more than a year ago and has not updated to a more recent version.
• Don’t do a product survey when you know there is any problem of any kind with the product. Wait for the problem to be fixed first. Use vacation periods in another country or a long weekend as an excuse to anyone who wants to know why.
• Make sure the wording of the email or other request to take the survey does not use the customer’s name, comes from a generic mailbox, is not in the local language, and makes no commitment to improve things. That way response rates will be lower. Low response rates tend to produce better scores because the people who love your products want to help you and will answer anyway. People who are less happy feel you have wasted enough of their time already and don’t bother.
• Provide an incentive like entry in a prize draw for a shiny gadget to everyone who answers the survey. People won’t believe they can win if they provide negative reviews.

Of course, if you succeed in cheating like this, nothing good will happen for customers, but at least good things will happen for your metrics, and perhaps your bonus.

Gaming of scores seems to be particularly common in the automobile industry. It is the rule, rather than the exception in the USA. The General Motors head of customer experience explained it in a pubic seminar attended by one of my HP colleagues. Sales people typically need to average over 95% satisfaction in the GM dealer surveys to be able to participate in their bonus scheme. One low score can require ten perfect scores to even it out. David Mingle, Executive Director, North American Customer Experience for GM said that he knows this and did not plan to do anything about it. He said he personally pays no attention to the scores, only to the verbatim comments, saying customers tend to be honest in their written comments. The sales people are only measured on the scores and don’t care much what the customer writes about the experience.

OK, all of the above may seem a bit cynical. I have seen all of these happen in practice. They are ways of avoiding finding out what your customers are thinking. The best way to avoid survey gaming is to ensure that no single individual who can directly influence the results is measured on those same results. Collective team metrics are usually fine. Ensure the purpose of your research is clear and that employees are motivated to help customers.

Comments welcome below. Please sign up for our newsletter here.