Design a screener survey to identify non-Feature X mobile users
Behavioral screener design with inclusion and exclusion criteria.
Ask broad frequency first, use indirect disqualifiers for Feature X without naming it, add attention checks, and over-recruit 7-8 to secure 5.
What's really being asked
This question tests whether you can translate a research goal into a screener that defines inclusion and exclusion criteria precisely, ensuring participants are relevant to the product while reducing bias. Interviewers want to see that you understand screeners improve data quality and save resources, and that you can avoid biased samples common in research panels.
The full answer
A strong answer structures the screener in four layers. First, it defines inclusion criteria by asking broad behavioral questions to confirm 3x weekly mobile app usage, such as how many days in the past week they opened the app and what tasks they performed. Second, it defines exclusion criteria using indirect disqualifiers for Feature X rather than naming it, for example by describing the task flow or UI element unique to that feature and asking whether the user has completed that action. Third, it adds an attention check to filter out speeders and the IT professionals or web-savvy panelists who often dominate research panels and can skew results. Fourth, it accounts for attrition by over-recruiting 7 to 8 participants to secure 5 completes, since screeners save time and money only when you do not discard data mid-session.
The mistakes people make
The biggest red flag is asking Have you ever used Feature X directly by name. Users often do not know internal feature names, so they may answer no incorrectly, or they may infer the desired answer and lie to qualify. Another red flag is revealing the study topic too early, which lets participants research the feature before the session. A third is failing to establish exclusion criteria at all, or relying solely on a single self-reported frequency question without a behavioral anchor.
What usually comes next
An interviewer might ask how you would verify the never used claim if you only have analytics data rather than self-report. They might also ask how you would handle a user who qualifies but then admits during the session that they actually have seen Feature X, or how you would adjust the screener if Feature X is so subtle that users could have been exposed without knowing it.
A concrete example
If Feature X is a new in-app camera filter, the screener should first ask how many days in the last seven they opened the mobile app. Then it should show a screenshot of the camera icon path and ask Have you ever used this tool to edit a photo before posting? Anyone who says yes is disqualified. An attention check asks them to select strongly agree for this statement. Finally, the recruiter invites 8 people to ensure 5 show up.
Interview question
When designing a screener to find mobile users who have never used Feature X, which strategy best reduces bias and improves data quality?
- a.Use a single self-reported frequency question to confirm app usage and skip exclusion criteria to keep the screener short.
- b.Begin with broad behavioral questions, then use an indirect description of Feature X's unique task flow to disqualify users without naming the feature.Correct
- c.Ask users directly if they have used Feature X by name, then disqualify those who say yes.
- d.Reveal the study topic early so participants can honestly assess their Feature X experience before screening.
Why? this is the answer
C is correct because broad behavioral questions verify genuine app usage, while indirect disqualifiers prevent users from misunderstanding internal feature names or faking eligibility to qualify. A is the most tempting distractor because direct questions seem efficient, but they actually increase false responses since users may not know internal names or may lie to participate.
Just read this? Test yourself on what you have been reading.
Read the original → nngroup.com
- #ux research
- #screener design
- #user recruiting
- #research bias
- #inclusion criteria
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on ux research — each one lists the topics its interview covers.
See open roles