15 Minute Read
Writing effective questions for a discussion guide is critical in order to meet the research goals. When you start writing a discussion guide, it’s normal to worry about how many or how few questions you should prepare, whether or not questions are leading or confusing, and if they seem forced or unnatural.
The psychology is simple: view your research participants as human. Prepare for the discussion with a great discussion guide, but also go into research conversations with the intent to actively listen and engage in meaningful dialogue.
In this blog post, we’ll dive into how to craft questions that pique contemplation and prompt narratives. Along the way keep in mind that discussions are just that, a conversation, and shouldn’t be a barrage of questions. The information you glean from asides, comments, and editorial flights of fancy could end up being just as valuable as the direct answers to explicit inquiries.
If you don’t have clear and valid research goals to work backward from, it’s hard to write effective research questions. It’s important first to ensure the things you want to learn can in fact be answered with qualitative research. From there you can narrow the problem space.
Try gauging your objective along one of these 4 simple problem-solving dimensions:
Every interview question you include in your discussion guide should be directly related to your research goals. Generally, effective discussion guide templates include the following sections:
Your attention to detail should shine through in each participant’s interview experience. It’s important not to combine multiple user experience research questions that ought to remain separate. For example, if you’re a marketplace and want to understand a journey from different vantage points, it makes sense to create a separate discussion guide for each persona.
Many relevant personal questions, such as those regarding income, voting habits, and purchasing practices, can be sensitive. Try adding information that helps participants feel more inclined to answer genuinely as opposed to presenting the "best version" of themselves. I like to start my discussions off with icebreaker questions to shake off the nerves and encourage participants to trust me as the researcher and open up.
Acquiescence bias, also known as "yes bias" or “agreeance bias” appears in the audience as a form of recurring false positives. In short, people agree with statements prompted by the researcher even if it isn’t a true reflection of how they feel.
Many factors can contribute to this form of bias and it can be tricky to solve since it's occurring in the audience. Use your initial questions to establish a behavioral baseline, then iteratively rewrite them to see if you're unknowingly leading or influencing participants.
Closed questions can be answered with yes or no, and are a quick way to stifle discussion. Open-ended questions on the other hand encourage the participant to engage in dialogue and use storytelling. It requires them to reply with hard facts. Open-ended questions are what will create rich, qualitative data. Open-ended research questions start with “describe why/how..” or “tell me more...” or “explain how/why…” or “walk me through…”
Beware of asking participants to predict the future or speculate on their future behavior. This is called a future-based question and it has no place in a qualitative research setting. It isn't reliable because there is typically a delta between what people want and say they will do with what they actually will do. An example of a good past-based question might sound like “Think about the last time you ordered takeout. Walk me through the experience of placing your order.”
Try policing your tone to sidestep language that might nudge opinions one way or the other. This might be a good place to self-check for biased language, but nothing beats taking proactive steps. An example of a question written with bias is: “Describe how [product/service] made your experience ordering takeout easier.” This question assumes the participant had a positive experience. Instead, write it with a neutral tone like “Describe your experience ordering takeout with [product/service.]”
It’s tempting to want to cram as much as you can into a discussion guide, leading to a common mistake of including double-barreled questions. Also called compound questions in legal proceedings, double-barreled questions usually contain a conjunction like “and”. They can result in skewed or missing data because it asks the participant to respond to two topics or issues at the same time but only allow for answering one.
Don't trust yourself to identify biases – They wouldn't be predispositions if people could somehow just turn them off with effortless grace. Talk to people outside your peer, colleague, social, or other groups. Solicit their honest opinions on how your questions come across.
Creating effective user research questions for your discussion guide takes time, diligence, and a willingness to refine. Fortunately, this is the perfect kind of work to tackle in pieces, gradually building up a knowledge library filled with trustworthy and rich qualitative data that will help your teams make more informed and better decisions.