Design a screener survey to identify non-Feature X mobile users

WHAT THIS TESTS: This question tests whether you can translate a research goal into a screener that defines inclusion and exclusion criteria precisely, ensuring participants are relevant to the product while reducing bias. Interviewers want to see that you understand screeners improve data quality and save resources, and that you can avoid biased samples common in research panels.

A GOOD ANSWER COVERS: A strong answer structures the screener in four layers. First, it defines inclusion criteria by asking broad behavioral questions to confirm 3x weekly mobile app usage, such as how many days in the past week they opened the app and what tasks they performed. Second, it defines exclusion criteria using indirect disqualifiers for Feature X rather than naming it, for example by describing the task flow or UI element unique to that feature and asking whether the user has completed that action. Third, it adds an attention check to filter out speeders and the IT professionals or web-savvy panelists who often dominate research panels and can skew results. Fourth, it accounts for attrition by over-recruiting 7 to 8 participants to secure 5 completes, since screeners save time and money only when you do not discard data mid-session.

COMMON WRONG ANSWERS: The biggest red flag is asking Have you ever used Feature X directly by name. Users often do not know internal feature names, so they may answer no incorrectly, or they may infer the desired answer and lie to qualify. Another red flag is revealing the study topic too early, which lets participants research the feature before the session. A third is failing to establish exclusion criteria at all, or relying solely on a single self-reported frequency question without a behavioral anchor.

LIKELY FOLLOW-UPS: An interviewer might ask how you would verify the never used claim if you only have analytics data rather than self-report. They might also ask how you would handle a user who qualifies but then admits during the session that they actually have seen Feature X, or how you would adjust the screener if Feature X is so subtle that users could have been exposed without knowing it.

ONE CONCRETE EXAMPLE: If Feature X is a new in-app camera filter, the screener should first ask how many days in the last seven they opened the mobile app. Then it should show a screenshot of the camera icon path and ask Have you ever used this tool to edit a photo before posting? Anyone who says yes is disqualified. An attention check asks them to select strongly agree for this statement. Finally, the recruiter invites 8 people to ensure 5 show up.

Source: nngroup.com