“If we have data, let’s look at data. If all we have are opinions, let’s go with mine.”
Jim Barksdale, former CEO of Netscape, said that decades ago, and it remains the default position of many leaders today. Customer Interviews and Focus Groups are a structured way to take qualitative conversations and turn them into quantifiable data… the data you need to influence your CEO.
Speaking directly with customers can be difficult, messy, time consuming, distracting, and intimidating. It’s one of those tasks which, objectively, every Product Manager will say is very important, but it is also the one which is not done consistently. It’s easier to send a survey, speak with internal stakeholders at your company, or even to monitor and analyze comments on social media. Yet the insights you can glean by speaking directly with your customers – the people who are using your product or who have chosen to buy it – can enliven dry powerpoint statistics.
Get out there and interview your customers, run focus groups, and learn how they are using your product, why they bought it, and whether your future plans will meet their future needs. Some say NIHITO (“Nothing Important Happens in the Office”), and if you believe it, get out of the office and meet some customers (physically or virtually).
What are Interviews & Focus Groups and Why Use them to Gather Customer Feedback?
“Customer interviews are a common mechanism for gathering the voice of the customer. Customer interviews are usually conducted one-on-one with an individual customer or with a small number of people from the same business or family unit. They provide an opportunity to get in-depth information from a single customer,” writes Kenneth Crow of DRM Associates.
Focus Groups involve a “small number of people (usually between 4 and 15, but typically 8) brought together with a moderator to focus on a specific product or topic. Focus groups aim at a discussion instead of on individual responses to formal questions, and produce qualitative data (preferences and beliefs) that may or may not be representative of the general population.”
A distinction: focus groups are not usability tests and usability tests are not focus groups. “If you want to learn how quickly and effectively a participant can complete tasks with your product, then you should run Usability Testing. Watch your [participants] use your product to see where they are successful and where they struggle in order to improve the user experience.” More on usability tests in a future post.
Customer interviews can be more time-consuming than focus groups, but because you are speaking directly with a single customer, you can focus on their particular experience, and also hear directly from them. Alternatively, Focus Groups have the advantage of allowing participants to hear and respond to what each other is saying. In the hands of a skilled moderator, a focus group can highlight key issues without devolving into groupthink.
Focus Groups are not Usability Tests. “Would you purchase this product?” is only loosely coupled with “How would you complete this task?”
Focus Groups are not Usability Tests. Tweet This
Focus Groups are not Usability Tests.Tweet This
What kind of feedback will it get you?
Customer interviews and focus groups are often used to validate and refine assumptions related to defining a product and bringing it to market. They can help you reduce the risk of going to market with a dud of a product. For example, according to Andrew Kaplan, LinkedIn uses customer interviews and focus groups to validate these topics:
- “We want to know who is our likely target customer.
- We want to prioritize the features we develop now rather than later.
- We want to know if there is any market for our product and if we should even build what we’ve planned.
- We want to test our narrative and positioning.
- We want to learn which other solutions are the closest comps.
- We want to understand how to price.”
I recommend that you read his full post. It contains lots of in-the-trenches insights on conducting customer interviews, especially in the “Ask the right questions” section.
Collecting the Feedback: Difficult. It takes time to figure out what you want to ask, and how you want to ask it. Conducting the interviews themselves is taxing both mentally and temporally. Yet the insights you can gather are second to none. Conducting a customer interview can be taxing both mentally and temporally. It can also be the most important conversation you have all day.
Analyzing the Feedback: Difficult. Do enough interviews and patterns might start to emerge, but it takes time and a keen assessment to pull out the patterns and trends. Customers are notoriously bad at predicting the future and expressing their needs in terms of new product features. But they are excellent at telling you where your product falls short and the pain points they keep experiencing. You need to be able to ask appropriate questions, and listen for what they are, and are not, saying.
Reach: Limited. Surveys are used when you need to canvas a broad spectrum of customers. Focus Groups and individual interviews are much more intimate.
Scalability: Difficult. You can share interview scripts and objectives so focus groups and interviews can be conducted by any number of people. But inherently, these types of interviews are more intimate, conducted in real time by real people. They take time.
Cost: Expensive. Even if conducted virtually, finding customers to interview, scheduling and conducting the interviews, and analyzing the results takes both manpower and money.
Pros… with Benefits
- Follow-up questions. As opposed to so many other forms of feedback, when you get an answer, you can immediately ask follow-up questions and probe deeper for more substantive insights. Pair surveys with customer interviews and you’ll be able to establish the baseline data from the survey, then follow-up with specific customers to better understand the answers and motivations driving those answers.
- Interviews are unstructured. If you want structure, and answers to specific questions, save yourself some time and just send out a survey. But you’ll learning an incredible amount from exploring the buttery nooks and crannies which come up in direct conversation. This means you must listen carefully and be comfortable deviating from your script with ad hoc questions.
- Focus groups allow participants to have conversations with each other. Customers can learn from others’ experiences, and corroborate or refute each other’s experiences. It can be a powerful, self-policing dynamic.
Cons… with Weaknesses
- Not scalable. See “The Facts” above.
- Time-consuming. Meeting with customers takes time – there is no way around that. But it is less time-consuming than making the wrong assumptions, bringing the wrong product to market, then correcting your mistakes.
- Lots of Logistics. Setting up and conducting the interviews requires a lot of planning to get a meaningful cross-section of data.
- Interviews are unstructured. That same bullet point in the Pros section above could also be viewed as a Con by some. Because interviews are less structured, some people may find it challenging to conduct them.
- Your hypothesis may get busted. What you learn in customer interviews and focus groups can emphatically invalidate your hypothesis. While impersonal data can be dismissed, it’s harder to write off the customer sitting in front of you who is explaining why your approach won’t work in their world.
- Interviewees may not be representative of your target audience. Are the customers you have chosen to interview, either individually or in a focus group, truly representative of your target audience? The composition of your focus group may skew the results and give you a false positive (or a false negative) upon which you are basing your findings.
Using Interviews & Focus Groups Throughout the Product Development Lifecycle
At what point(s) in the PDLC will this type of feedback be most useful?
- Phase 1: Conceive – imagine, specify, plan, innovate
- Phase 2: Design – describe, define, develop, test, analyze, validate
- Phase 3: Realize – manufacture, make, build, procure, produce, sell, deliver
- Phase 4: Service – Use, operate, maintain, support, sustain, phase-out, retire, recycle and dispose
Tapping into Interviews & Focus Groups is helpful throughout the Product Development Lifecycle.
- Phase 1: Before you go too far with an idea, put it in front of customers to get a sanity check. Just make sure you have something concrete for them to respond to – a prototype, wireframe, messaging… something tangible. Otherwise, you’re brainstorming, not validating.
- Phase 2: Involve customers as the product or feature evolves. Give them early access to alpha and beta versions, and interview them to understand whether it is meeting their needs and if they would pay for it.
- Phase 3: As you get closer to going to market, use focus groups and structured customer interviews to tighten up messaging and target appropriate buyers and influencers.
Best Practices & Pro-tips
Learn to Ask Great Questions
When it comes to conducting a customer interview or running a focus group, the questions you ask form the backbone of the experience. So, before spending too much time planning out the interview or focus group:
- First make sure you know how to ask sparkling questions and are prepared to ask follow-up questions that blow past superficial responses.
- Understand the differences between open- and closed-ended questions and when it is most appropriate to use each.
- Learn to recognize non-answers and how to keep the conversation going so you get to a real, substantive answer.
- Practice being quiet and just observing, since silence can make you a better interviewer as can paying attention to nonverbal cues.
Lucky for you, I’ve already written a couple of posts on these topics which <humblebrag> I’ve been told are quite phenomenal, although I find it hard to believe </humblebrag>. Check them out so you, too, can tell me how amazing they are.
Good interviews and focus groups require great planning. Understand your objectives and subject matter, structure the questions you want to ask so you can compare results across sessions, and turn anecdotes into trends.
- What are your learning objectives? “[W]rite out the high level goals for your research based on the questions you have about your users, and assumptions you have made about your solution.” [source]
- Who is your target audience? This could be defined by demographics, but it is also effective to define the audience based on needs and behavioral characteristics. And, while the tendency may be to recruit users (end-users), don’t ignore the buyers, because they might have different objectives, motivations or constraints than the users. And if the buyers are not bought-in, it may make it that much more difficult for the key users to influence the final decision. Note: A “buyer” is the person who controls the buying decision; buyers are not always users.
- Create and stick to a budget. Unless you are in the enviable position of having a blank check and unlimited budget, you should take the time to figure out how much you can spend in the program, and also how much you might need in order to run effective interviews or focus groups. They can be done on the cheap, yet once you start organizing focus groups, expenses can mount pretty quickly.
- Recruiting & scheduling. With the audience defined, make a session schedule and start recruiting participants. Create a discussion guide / script for the session to ensure you are consistently setting up each session, and are giving everyone the same starting point. It should make it easier for you to compare responses across sessions. With script in hand, you will cover key points in every interview and remain consistent with how the questions and discussions are framed. Remember: checklists are a good thing.
Running a Customer Interview or Focus Group
Start by establishing the ground rules for the interview. Your primary objective is to make sure that participants understand their role in this process, while also making them feel comfortable enough to honestly express themselves. Let them know what the information will be used for, whether it will remain anonymous or be attributed to them, and with whom it will be shared.
A logistical side note: assign a designated note-taker, preferably a different person than the one conducting the interview. If it is not possible or practical to have a note taker, ask participants if you can record audio or video of the interview itself. Just remember that many people become self-conscious when there are recording devices present.
- “Need — the motivation that is driving the participant’s actions and feelings.”
- “Pain point — barriers or roadblocks that prevent the participant from fulfilling their need.”
- “Touchpoint — people, places and things the participant interacts with through their experience.”
- “Quote — something the participant has said that helps clearly exemplify the learnings or next steps you have identified.”
How Not to Interview
Justin Wilcox gives some sound advice on interviews:
- “1st rule of validating your idea: Do not talk about your idea…. Once your idea pops into your brain during an interview, your body will literally turn against you. It will start looking for validation that the idea is good. Your interpretations of statements, intonation, body language will all be skewed. What’s worse, the person you’re talking to subconsciously knows what you’re looking for, and based on our desire to build relationships, will want to help you…. avoid thinking about your idea during the interview – and certainly avoid talking about it.” He suggests to instead remain focused “on the customer and their problems.”
- “2nd rule of validating your idea: Do not ask about the future… No hypotheticals, no projections, no guesses. The way I remember this rule? I never ask a question with the word “would” in it (‘If we built a product that solved X problem, would you use it?’, ‘How much would you pay for something that did X?’, ‘Would you like your existing solution better if it did X?’”
How TO Interview
Justin Wilcox uses these 5 questions as a springboard for the customer interview:
- “What’s the hardest part about [problem context]?
- Can you tell me about the last time that happened?
- Why was that hard?
- What, if anything, have you done to solve that problem?
- What don’t you love about the solutions you’ve tried?”
He goes on to explain that for question 1, “you want to ask about a significant problem context – situations that occurs frequently enough, or are painful enough, to warrant solving.”
Other questions Justin will often include in his script are:
- How often do you experience this problem?
- How much are you spending to solve this problem now?
- Where do you find information about [problem context] online?
Read the rest of his post for more insights.
Thanks for the data… Now What?
Summarize the findings of each session and all meta-findings as soon as possible. Field Guide provides some concise guidelines in a post on Medium:
- “Barriers — if your participant runs into pain points in a digital or real world experience, you should make note of that as a problem that could potentially be addressed by your team.
- Patterns in feedback — if the same points of feedback keep coming up, from an individual or across many participants, that should be noted.
- Language used — how did the participant talk about their experience, and how did they phrase it? This is useful information to collect because it can help inform how you communicate your product feedback to your team.
- Important feedback — while this is entirely subjective, trust your instincts. Whatever feedback stuck out for you in the moment likely did so for a reason. You can analyze it further once you are done all of your sessions to see if it still holds weight.
- Issues and bugs — depending on how big they are, they might be fixable before your next session. If not, it should at least inform what needs to be fixed moving forward.”
The bottom line of your summary: did the customers validate or invalidate your hypothesis? What are the key learnings? Are there any adjustments you would make in the next sessions based on what you learned?
Above all, don’t jump to conclusions. Allow the series of interviews or focus groups play out before deciding to pivot or abandon. It’s hard, but essential, that you get critical mass. Just don’t draw conclusions that weren’t there. “Resist the temptation to infer things that your customers didn’t tell your explicitly,” says Andrew Kaplan.
Customer Interviews and focus groups are the best way to get detailed information about how and why customers are using your products. Speak with your customers and seek to understand their motivations. Through strategic use of interviews, you can validate your hypothesis about future product direction and get confirmation about your upcoming product improvements. Make no mistake – conducting and analyzing customers interviews is very difficult and time-consuming. Done well, they will give you the confidence that the product you are releasing will be a winner.