Thoughts about survey infrastructure for Net Promoter and other feedback/improvement systems

/ , / , , , , , /

Unless yours is a very small company, you will not be able to implement and operate an effective Net Promoter System or any other feedback and improvement system without appropriate technology. To the extent possible, you should use Cloud-based (SaaS) solutions that do not require you to invest in hardware. Among other things, an advantage of this approach is that you will not be constrained by a corporate IT budget, and can advance on your own. Cloud software is always up-to-date. For the sake of this discussion, Customer Relationship Management software, used by sales and call center people, is out of scope.

Simplest possible research infrastructure

You can use Excel with Outlook mail merge to send surveys, and that can work in small volumes. That should work with Open Office too. The primary challenge is ensuring it looks OK on a phone. Here are the advantages of this simple approach:

  • You probably don’t have to spend any money to get into motion.
  • Assuming small volumes, you can simply ask people to reply to the email to send their survey responses.
  • The emails can go directly from anyone you choose, such as a product manager, or from a process mailbox in the name of the product manager.
  • If you have designed your survey correctly, customers should be able to see the entire survey on a single screen when they open the email. This is almost impossible when using a link to a web-based survey. When customers see the full survey, it improves their belief in your statement that it will only take them a couple of minutes to respond. That in turn will improve response rates compared to a web link.

There are disadvantages, even in small volumes:

  • Managing opt-out lists is difficult. Any survey you send should include a way for the customers to decline the survey and to tell you they no longer want to receive any surveys. If you work from a single Excel contact list, you can probably handle it correctly. If multiple people send surveys from multiple contact lists, it can be difficult to coordinate opt-outs across lists. Failing to manage opt-outs correctly can land you in legal trouble.
  • To do any analysis of the results, you will have to copy-and-paste or otherwise transcribe what has been sent back.

Basic survey infrastructure

Getting beyond the extreme mentioned above, I recommend using SurveyMonkey and MailChimp for basic needs. There are a number of similar alternatives that will also work just as well. (I have no commercial relationship with any software company and consider myself unbiased.)

SurveyMonkey advantages:

  • There are different levels of monthly SurveyMonkey subscriptions, going from free to advanced solutions with no customer-visible evidence that you are using SurveyMonkey. (Really, they could have chosen a name that would make them sound less like amateurs.)
  • Higher subscription levels include simple text analysis.
  • It runs in many languages.
  • There are ‘certified’ NPS templates.
  • Lots of people use it, so lots of advice is available.
  • You can download results as Excel files for your own analysis.
  • You can send emails directly from their service.
  • If you don’t like it, don’t renew your monthly subscription.

SurveyMonkey disadvantages:

  • If you use a separate email solution, you can only provide a link to the SurveyMonkey survey in your email. There is no way of embedding the rating question in your email, so response rates will be lower (less than half) than with solutions from Medallia, Promoter.io and others that embed the rating question in the outbound message.
  • When I say the text analysis is basic, it really is basic. So is Medallia’s, at the time of writing.
  • Some tracking options are only available if you use SurveyMonkey’s outbound email service.

MailChimp advantages:

  • MailChimp lets you see who has opened your messages and who has clicked on any link they contain. This is particularly useful for A/B testing. By this I mean, for example, sending email with different subject lines to each half of your mailing list to see which subject lines get better open rates, and doing the same for different requests to take your survey. You should only change a single thing at any time to have a valid A/B test.
  • If your list has up to 2,000 customers and you want to send no more than a total of 12,000 emails per month, you can do it for free. The paid plan starts at $20 per month for 500 customers, for example.

MailChimp disadvantages

  • MailChimp tracks recipients’ actions, but does not disclose that it is doing so, at least by default. I am already aware that this has been raised as an issue with the national data privacy organization in one European Union country.

More advanced solutions

Two capabilities are the most critical when considering advanced solutions for managing NPS surveys. For clarity, I am talking about software that only manages surveys. There are of course many solutions that manage surveys together with many other things.

  1. You should embed the rating question in outbound emails to customers. This will give you double to triple the response rate, compared to sending the same survey with just a link in the email. There are various reasons for this. Given the number of phishing and ransomware attacks we hear about, people are reluctant to click on links they do not absolutely trust. It helps if the email comes from your company email server and that this is clearly visible.
  2. You need to be able to do a first-pass analysis of the verbatim answers without human intervention. I blogged about this some time ago here.

While many vendors push it hard, I am not a fan of encouraging customers to respond on a phone. The main value you get from NPS research is the answers to the open questions. I feel you should tolerate phone-based text input, understanding that it will be brief. Your outbound email should say something like “You may be most comfortable providing feedback on a PC, Mac or tablet.”