Want to build a better product? Capture the voice of hard-to-survey populations

So, how do you reach and engage them?

Voice of the Customer (VoC) programs are structured processes where organizations collect and analyze customer feedback across channels and interactions to understand their wants, needs, and experiences to improve product and service design. VoC programs typically use a variety of research and data collection methodologies, such as surveys, customer interviews, social media listening, and customer service analytics. VoC programs may also incorporate ethnographic research, user experience research, and more.

Product Managers and Service Design teams in tech, industrials, consumer services, healthcare and other complex industries need research to inform costly decisions, but some populations and topics are notoriously hard to survey. This means that the voice of customer types who are hard-to-survey may be underrepresented or missing entirely from overall VoC insights.

Today, we must accept that no single VoC research tool engages all segments equally. Though a single approach appears to have more rigor and thus more statistical significance, a combination of approaches that can capture VoC from all segments may, in fact, yield more value to the organization. TL;DR you need to blend the science and art in your market research.

As a consultant, most of my prior VoC work is with SaaS, Consumer Products, and Industrials customers who serve distinct customer segments that materially differ from the general population. Healthcare, however, is interesting because large healthcare providers, insurers, and health services organizations often serve the general population within their geographic service areas.

I reached out to Sandra Chacon to get her perspective on how to design Voice of the Customer research programs to more effectively engage hard-to-engage populations in healthcare. She has a distinctive point of view because she started her career in healthcare services research at UCLA focusing on underserved communities then worked with a national health services organization to drive research and analytics related to the patient experience, health equity, and healthcare outcomes.

Though we work in different industries, we identified some common challenges in ensuring representation of all segments in our VoC research.

Common populations or segments that are hard-to-survey include:

• Parents with one or more kids aged 0 to 5
• Tradespeople
• Low income individuals and households
• Executives and business owners

If you’re wondering why a certain population or segment is hard-to-survey, think through these elements:

Location. Where do you find them? Do enough potential participants from this group congregate in any given location to make it time-efficient & cost-effective to engage them there? Do researchers and participants alike feel safe here? Will your emailed survey invitations make it to the inbox or get flagged as SPAM?

Capacity to participate and to understand your questions. In my experience, tech companies are particularly bad about expecting to quantify everything, even questions that are so sophisticated and/or granular that only industry insiders can understand, let alone actively think about. Let’s talk about the barriers to participating and understanding your questions.

• Can they see your questions? 4.2M US adults aged 40 and older have uncorrectable vision impairment.
• Can they hear your questions, if spoken to them? 13–15% of US adults have partial hearing loss, rising to almost 27% of those aged 65 and older.
• Can they understand your questions? 1 in 5 US adults have low English literacy skills, meaning they struggle to understand, use, or engage with written text, such as survey questionnaires. Two-thirds of the low English literacy group were born in the US. 54% of adults are below 6th grade level reading proficiency. It’s more challenging than you think to write clear and useful survey questions at or below a 6th grade level. Note: The preceding sentence requires 8th grade proficiency to understand. Getting to 6th grade level requires stripping away adverbs, clauses, and details that may be essential to your VoC. Does your research involve numbers, weights and measures, or prices and discounts? Keep in mind that 29% of US people ages 16 to 65 have numeracy skills at or below Level 1, meaning they struggle to complete multi-step calculations, work with measurements and spatial representation, interpret data in texts, tables and graphs, or other higher-level mathematical thinking.
• Can they give their full attention to your research? What else may be competing for or constraining their attention? Will you lose their attention before they’ve gotten through your basic screener questions, let alone your real ones?
• No wonder it’s hard to get useful, actionable insights from surveys!

Trust. Do they trust or distrust people generally? Do they trust or distrust technology generally? Do they readily trust the individual researcher or team you sent? Do they trust your organization? Do they have high confidence they will receive the promised compensation for participation? What can they reasonably expect will happen with their data? Some people think retail intercepts or survey requests are an intrusion or nuisance while others are happy to share insights about a product or service as long as you don’t make requests they perceive as invasive, such as marital status, household income, or to take video of them while they browse your website or brick & mortar shop.

You may be wondering how we define hard-to-survey topics. There are three main elements that make things hard-to-survey:

• High complexity and low purchase/use frequency categories. SaaS, enterprise software, healthcare services, therapy, consumer electronics, and insurance of all types. The more intangible something is, the harder it is for customers to articulate their thoughts in your VoC program.
• Level of nuance asked of customers to inform your business decisions. A sizable proportion of people struggle to be specific and/or to go below the surface in their responses. Some will resist or outright reject the 5 whys approach, even in an in-depth interview led by a skilled researcher.
• If you need responses from the same participants repeatedly over a period of time to measure outcomes. (These are called longitudinal studies.) It’s hard enough to get someone to participate in a survey, focus group or in-depth interview once, let alone to do so at regular intervals over a period of months or years.

Here’s a memorable example:

A discount retailer believed a $50 unlocked smartphone would do very well in its stores. Most of the chain’s stores are in low-income and rural areas, so its customers fit our description of hard-to-survey. Unlocked in this case meant (1) unsubsidized by a wireless carrier and (2) usable across all the major wireless networks. To make a smartphone available at such a low price, the manufacturer would need to know which features customers expected and which components it could remove or downgrade to reduce the cost of the device — such as using fewer radios or a less-powerful chipset inside the phone. But outside the wireless and tech industries, who could realistically be expected to know which frequencies or how many gigahertz processing speed they wanted or the implications of tradeoffs? These are clearly hard-to-survey topics. The store’s customer base knew they wanted cheap smartphones that let them do what others do, like listen to music, take photos, and watch videos. Ultimately, I took a novel approach to gather Voice of the Customer. Sentiment analysis of online reviews for the cheapest unlocked smartphones available in the US revealed that devices at the $100 price point crashed whenever the user received a message while listening to music. From this insight, it was reasonable to infer that $50 smartphones would perform even worse, selling well initially but overwhelming the retailer with customer returns. The account director and I advised our retail partner against bringing this type of product into its assortment.

Strategies for gathering VoC from hard-to-survey populations on hard-to-survey topics

1. Use multiple VoC methodologies

• Remember, there’s no single method that works across all populations or segments.
• Lead with qualitative research and a higher-skilled research team to rapidly test and refine hypotheses that you will measure at scale in quantitative studies. You might use in-depth interviews, ethnographies (in-home, at-work or in-store), shop-alongs, or observational studies.
• Phone surveys gave way to digital surveys, but you may need both methods to reach people. People are over-surveyed, incentive amounts have declined over time, your email invitation may not have even reached their inbox, and their phone silenced your follow-up call as suspected spam. And as Sandra pointed out, when you’re engaging with low-income populations in a longitudinal study, it’s wise to ask for an alternative way to get ahold of the person just in case their phone gets disconnected.

2. Expand how you identify and recruit participants

• Have your researchers go to gemba and observe. Identify locations that serve as first-line filters for the population profiles you’re looking for. When parents of young children are not at home or at work, you’re most likely to find them at grocery stores, pediatrician offices, public libraries and parks. Your best chance to engage tradespeople is not on the job site, which may be loud and dangerous, but in the early morning when they’re picking up materials from a lumberyard or building supply warehouse. You’ve got a sliver of time to engage participants, so consider reducing the number of qualifying and overall questions, empowering researchers to ‘eyeball’ certain attributes such as age group, and balancing use of asking and observing to gather insights.
• Proactively solve for practical constraints that impede participation. That could mean providing or reimbursing parking, offering toys or entertainment to occupy young kids while parents talk to researchers, providing prices or cost savings in both monthly and annual terms, and so on.
• Talk to first-line aggregators of insights such as store managers, pharmacy clerks, and staff to supplement what you’re observing or hearing.

Incentivize your target population appropriately

• Form of payment. Cash is universally appealing. Gift cards to the store where you’re conducting retail intercepts may be welcome, but less so if the prospective participant only shops there on behalf of their employer. Digital gift cards that cannot be combined with other forms of payment are deceptive and should be avoided, especially if they’re in low denominations.
• Amount of payment. If you want to recruit higher-income participants to your VoC study, expect to provide larger rewards. Donations to a charitable cause, in our experience, can motivate executives and business owners more so than the equivalent amount in cash — as long as the dollar amount is still meaningful. If you want to incentivize ongoing participation in a longitudinal study, don’t split the total dollar amount into equal payments or withhold the whole payment until the end. Sandra had great success improving participation at the 2nd and 3rd milestones in a longitudinal study by offering increasing reward amounts, such as $15, then $25, then $60 instead of $100 total or 3 equal payments because she understood the psychology of money for the relevant population.
• Non-monetary incentives. In our combined experience, sometimes people just want to vent or to feel their voice is heard and don’t need an incentive to participate. Sometimes the best incentive, especially for professionals and business owners, is to offer outside-in insights or similar exchange of value.

Above all, don’t ask questions that people cannot reasonably answer. The most absurd example of this was the patient satisfaction survey asking multiple detailed questions about my patient experience with the anesthesiologist. If you’re a patient being put under for a procedure, you logically wouldn’t remember much, if anything, between when the anesthesiologist entered the room and when you woke up post-procedure. Sorry about your Press-Ganey score, Mr. Anesthesiologist, but I literally couldn’t answer the questions. If only there was a free-response option I’d tell you “felt nothing, remember nothing, woke up, 10 out of 10, would recommend!”

Readers, what’s your most memorable VoC moment?