Generative versus evaluative research: when to use each

WHAT THIS TESTS: This question checks if you understand research as a strategic timing decision rather than a checkbox. Senior engineers are expected to know that generative research explores user needs and behaviors before solutions exist, while evaluative research measures how well a product meets those needs after something is built. The interviewer wants to see that you can align research phases with design and development milestones, and that you know why skipping generative work leads to building the wrong thing efficiently.

A GOOD ANSWER COVERS: First, define generative research as exploratory or discovery work that happens before any technical design to uncover problems, user mental models, and opportunity spaces. Second, define evaluative research as assessment or validation work that happens once an initial version exists to test usability, task success, and solution fit. Third, explicitly map them to the project timeline: generative before initial technical design to inform requirements, evaluative after a build to refine and iterate. Fourth, note that the two are complementary, not interchangeable, and that combining them prevents wasted engineering effort.

COMMON WRONG ANSWERS: A major red flag is saying all user research is the same or that testing only happens at the end. Another is claiming generative research is unnecessary because product managers or engineers already know the user problem. Some candidates describe evaluative methods like usability testing but cannot explain what generative methods look like, such as ethnographic interviews or diary studies. Treating research as a single phase rather than an ongoing discipline also signals inexperience.

LIKELY FOLLOW-UPS: The interviewer may ask which specific methods fit each bucket, such as contextual inquiry for generative versus usability testing for evaluative. They might ask how to advocate for generative research when leadership wants to ship fast, or how you would run evaluative research on a backend-heavy feature with no UI. Another follow-up is how you measure the ROI of generative research when it takes four to eight weeks.

ONE CONCRETE EXAMPLE: Imagine your team wants to build a money-saving app for millennials. Before writing any technical design documents, you conduct generative interviews and diary studies to learn how millennials think about savings, what triggers their financial anxiety, and what tools they currently hack together. This shapes the feature set and architecture. After releasing an MVP budget tracker, you run evaluative usability tests and analyze task completion rates to see if users can set savings goals without confusion. The generative phase prevented building a feature nobody needed; the evaluative phase fixed interaction bugs before scale.

Source: lyssna.com