How do you avoid confirmation bias when interpreting usability test actions?

WHAT THIS TESTS: Whether you understand that confirmation bias is a cognitive error where prior beliefs distort how you perceive new information, and whether you can name specific behavioral protocols to counter it during live usability observation. Interviewers want to see that you do not rely on generic promises of objectivity but instead engineer the observation process to surface disconfirming evidence.

A GOOD ANSWER COVERS: First, a concise definition of confirmation bias as the tendency to pursue or interpret information in a way that conforms with preexisting beliefs while discarding contradictory evidence. Second, the explicit link to UX context: the more time you have invested in a design or assumption, the stronger your bias will be when watching users. Third, concrete mitigation tactics used during a usability test: observe silently without coaching the user, write timestamped notes that describe behavior neutrally before interpreting it, have a neutral colleague observe the same session, predefine falsifiable hypotheses rather than leading questions, and deliberately look for evidence that would prove your assumption wrong. Fourth, the idea of separating data capture from analysis so that raw observations are recorded before group debriefs introduce social confirmation bias.

COMMON WRONG ANSWERS: Claiming that you simply stay objective or trust your engineering instincts. Proposing to ask the user leading questions like whether a specific button was hard to find. Suggesting that you would explain the design to the participant when they struggle, which turns the session into a defense of the design rather than an inquiry into behavior. Stating that you would only watch recordings of successful tasks or that you would discount outliers.

LIKELY FOLLOW-UPS: How would you reframe a leading survey question into a neutral one? What would you do if your PM insists the feature is fine despite clear usability failures? How do you prevent confirmation bias when analyzing qualitative data after the session? Can you describe a time when you changed your mind about a design because of disconfirming evidence?

ONE CONCRETE EXAMPLE: Imagine an ecommerce site with high cart abandonment. The team believes the red checkout button is hard to find. A biased observer might ask users whether the red checkout button was difficult to locate, priming them to confirm the hypothesis. An unbiased approach defines the hypothesis as falsifiable before the test, then watches silently as users attempt to purchase, noting exactly where they pause or click without mentioning color or button labels. Afterward, the team compares timestamps across multiple participants and only then decides whether the button location was actually the bottleneck or if another issue such as shipping cost timing caused abandonment.

Source: nngroup.com