What is Evaluative Research vs Generative Research?
Evaluative research is what most people think of right away when they think about UX Research. Usability sessions are a method for carrying out evaluative research, which happens at the end of a design or development cycle. The goal is to evaluate whether assumptions that have been made are true, and that the output is not only usable, but grounded in real wants and needs from real people.
Conversely, Generative Research happens at the beginning of a design cycle during the discovery phase. It involves talking to people to find hidden opportunities to innovate or problems to solve with design and technology.
Research requires a genuine curiosity and open mind; but more importantly the vulnerability to leave your ego and expertise and at the door, two things that stand in the way between your user and their needs.
In this article, I’m sharing when to do evaluative research, who should be involved, and how to prepare for the session.
Teams often struggle with how to schedule regular evaluative research alongside product development, as if it were another ceremonial activity inside a sprint. While you should regularly talk to users as generative research, I prefer to look for early signals that the team needs clarity and confidence around a solution rather than sticking to a rigid cadence for evaluative research.
In general, you will always learn something by getting your solution into the hands of real people, as soon as possible. The sooner you do evaluative research, the less married to the solution the team will be, and the less risk there is to the business.
Some signals it might be time to do some research:
Any one of these signals would be enough to kick off a usability session. The more you work with your team the earlier you can get ahead of these signals.
At a minimum, a moderator and note-taker should be involved in the usability session. I’ve seen this team as a mix of researchers, product managers, designers, and developers, but usually moderated by the designer or researcher.
In general it’s best to keep these sessions intimate, like a conversation rather than a panel interview. The more comfortable the participant feels, the better and more authentic the results will be. What this *doesn’t* mean it to keep the unsynthesized research to yourself.
The more you can reduce the distance between the team who uses the research with the actual research itself, the better. The more approachable research is to the team, the more involvement there will be. You’ll start to see greater advocacy for research when they experience the research firsthand. The team will start to bring up a specific person and their needs from a research session, rather than referring to everyone point blank as “users.”
Some ideas to involve the team:
As a rule of thumb, it’s best not to have “silent” attendees just observing. If they’re in, they should be actively involved in some way. Otherwise, share the video with them later.
Once you’ve identified an early signal that it’s time to do some testing, you’ll need to extract from the team what it is you’re setting out to learn.
I suggest always having a running list of questions to answer that pop up organically through doing the day-to-day work. At Notably, we use Notion to track potential research programs as a simple “questions to be answered”, tagged by feature, and link back to any conversations, tickets, or emails that surfaced this question.
This habit builds a culture of humility, admitting that we don’t know everything about our users and are constantly making assumptions and taking bets. Research helps you make better assumptions and take smarter bets.
It also jumpstarts this next activity for planning your usability session.
Map out the workflow you plan to test, complete with screenshots of each step in the flow. This can be done on a physical whiteboard or digital whiteboard tool. Create a row for each team member or role (depending on the size of your org) that spans across the workflow. Facilitate a working session to pull questions about this workflow from the team:
This becomes the framework to inform what will become your research plan: what you’re setting out to learn. This is a good team building exercise that gives everyone a chance to be heard and contribute to the research plan, or as one of our clients referred to as “product therapy.”