Sometimes, executives answering surveys have no idea what they are talking about (Discuss!)
There is a somewhat strange phenomenon that I have observed at an anecdotal level over the years. It is limited to B2B surveys and causes all sorts of problems. Or at least that’s what I think, and I would like to understand your views too. Here is what I want to discuss:
When we send relationship surveys to B2B customers, we always try to identify key decision-makers and decision-influencers and we ask them to answer a set of questions. This can be done in person, remotely, or by sending a link to a web-based survey, for example. Let’s imagine a situation where a company selling ERP software believes their customer’s Vice-President of Procurement is a key decision-maker. The company sends a survey request out by email and the procurement VP decides to respond. The survey follows a fairly standard format of an NPS question, a couple of open questions about what should be improved, and then a set of 12 questions asking the procurement VP to provide scores about product reliability, the contact center experience, how easy the software was to learn to use, and so on.
This person may have no idea what they are talking about
You may already see where I am going with this. The procurement VP does not use the software and really does not know what is going on. Nevertheless, they have decided to spend five or ten minutes filling out the survey, so they are going to give you scores for everything. Uninformed scores. Wrong scores. Scores that probably correspond to their overall opinion of your company rather than the actual questions being asked.
Then you decide you need to take action on the bad scores, even though your performance may already be perfect.
In this case, the VP is probably out of synch with the rest of the team, especially those using the ERP software every day. These are the people who have useful suggestions, though they may have only a minor role (or perhaps none at all) in deciding whether or not to renew the contract with the software.
Here is my current opinion
In my possibly-wrong opinion, these scores are therefore a relatively accurate predictor of attitudes and future purchasing behavior at the aggregate level (as is ther overall relationship NPS score) but are useless at the level of the individual questions, in terms of telling you which operational improvements you should prioritize to address those attitudes.
Of course you may well find purchasing VPs who have good knowledge of most operational touchpoints, but I think those are rare people in large companies.
I’m not totally certain what to do about this. Do you have some ideas?
I believe the major customer-touching teams in your company do all need input about what they could do better. This issue is that the input needs to be more accurate than it probably is at the moment. For the overall company relationship surveys that are answered by high-level executives, I doubt that adding a request not to answer the driver questions if you don’t know what you are talking about would be a good idea. Maybe we should supply the procurement VP with relevant operational data? Maybe we should accompany all relationship survey requests with the results and trends from any and all transactional surveys we may do?
For transactional surveys, the main issue we all face is that response rates are very low, so we can’t be sure they are accurate. That’s where analyzing operational data trends for all the customers helps. If the operational data numbers and trends don’t match the transactional survey results, I have to say I believe the survey results are wrong, probably because you are not getting the correct people to take the surveys.
So, what do you think? What should we CX and Customer Success experts do about this horrible situation? Yes, I have some ideas ideas. We need more solutions. Please help!
LinkedIn version of this post here
Notes
OCX Cognition predicts customer futures. Our breakthrough Customer AI solution lets enterprises transform what’s possible in customer experience. Reduce your customer risk, break down walls between teams, and drive speedy action – when you can see what’s coming, you can change the outcome. Building on more than 100 years of CX-focused expertise in our small team and thousands in teams we have led, we’ve harnessed today’s advances in AI, elastic computing, and data science to deliver on the promise of customer-driven financial results. Learn more at www.ocxcognition.com.
Maurice FitzGerald is Editor-in-Chief, Content at OCX Cognition. He retired from HP where he was VP of Customer Experience for their $4 billion software business and was previously VP of Strategy and Customer Experience as well as Chief of Staff for HP in EMEA. He and his brother Peter, an Oxford D.Phil in Cognitive Psychology, have written three books on customer experience strategy and NPS, all available from Amazon.
