Intermediate everything in Design & UX, page 8
How do you reconcile conflicting qualitative stories with quantitative data?
Integrating mixed-methods without picking sides. A strong answer triangulates definitions, scope, and timing; uses qualitative context to explain outliers; and treats conflict as a signal to dig deeper.
What is the multiple comparisons problem in UX research?
This tests whether you know running many tests inflates false positives. Strong answers define family-wise error, give a UX example like comparing twenty metrics in one A/B test, and name a correction like Bonferroni.
How many users do we need to survey?
Tests pragmatism over guesswork. A strong answer asks if this is a census or sample, population size, whether stratified or multi-group designs apply, and cost, time, and power constraints. Red flag: blurting a number without probing scope, design, or budget.
How do you handle a powerful emergent theme outside original research questions?
Validate recurrence and density, weigh timeline and goals, then document separately, expand formally, or park.
How do you determine when you've reached thematic saturation?
This tests operationalizing saturation beyond gut feel. A strong answer cites Guest et al.'s Base Size, Run Length, and New Information Threshold, and flags consecutive interviews adding no new high-level themes.
What is your process for reconciling significant coding differences?
This tests intercoder reliability and systematic reconciliation. Quantify divergence via agreement metrics; convene to refine codebook definitions and edge-case rules; re-code a sample to verify alignment. Red flag: treating coding as pure subjective opinion.
How do you synthesize low-level codes into high-level themes?
Tests systematic thematic analysis. Strong answers cover tagging observations with codes, grouping related findings into themes that emerge across participants, and choosing methods like affinity diagramming based on context.
How would you design an experiment measuring API latency impact on retention?
Randomize users into control and delay groups; track D7 retention and P99 latency; analyze with quantile metrics.
How would you model user events to analyze cancellation paths?
Timestamped events with user and session IDs, separate entity snapshots, and time-range partitioning.
What usability metrics would you instrument, and how do regressions drive priorities?
Tests operationalizing UX into engineering signals. Strong answers list success rate, time on task, and error rate; triage regressions by criticality; and pair quant drops with qual diagnosis. Red flag: calling every regression P0 or using vanity metrics.
How could you adapt usability testing principles to evaluate two API designs?
Tests holistic API UX beyond static syntax. Strong answers propose observing developers in realistic IDE and doc contexts, noting errors and confusion across both designs.
How would you use a journey map to find technical root causes?
Map journey stages to logs and traces, find bottlenecks, propose API or data model changes.
How do you mitigate confirmation bias when a user validates your solution?
In the moment, probe for exceptions; in synthesis, triangulate and invite reviewers.
Translate qualitative insights into user stories and requirements
Clusters themes by frequency/severity, reframes pain points as user stories with clear AC, maps to technical spikes, prioritizes by impact.
Propose a strategy for finding and vetting low-incidence participants
This tests sourcing creativity and screening rigor for rare populations. A strong answer proposes niche outreach, snowball referrals, behavioral screeners, and anti-fraud checks. A red flag is simply boosting spend on generic panels or relying on coworkers.
Describe a workflow to automate a 50-person diary study
Tests operational UX research automation and API integration. A strong answer maps a pipeline of calendar APIs, email triggers, and reminder bots with no-show fallbacks. Red flag: manual spreadsheets or survey tools lacking scheduling logic.
What two techniques mitigate professional-tester bias in screener design?
Swap guessable questions for behavioral ones about real tasks, and add exclusion criteria to block web-savvy professional testers.
What criteria and questions ensure right users for a specialized dev tool?
Designing a recruiting pipeline that qualifies niche experts and secures attendance. A strong answer covers role criteria, layered tech screens, and no-show backups. Red flag: relying on job titles without verifying real tool usage.
Propose two research methods to investigate checkout abandonment
Tests pairing behavioral and attitudinal research. Great answers match funnel analytics to find drop-offs with usability tests to uncover why users cannot complete fields or leave in anger.
What technical instrumentation supplements a two-week diary study?
Log anonymized feature frequency, session duration, and device switches; link via participant ID.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles