Intermediate interview questions in Product Management, page 28
Detecting and fixing metric hacking
Look for diverging counter-metrics and anomalous patterns, then pair KRs with guardrail metrics or redefine to a truer proxy.
Determine A/B test sample size
Define baseline rate, minimum detectable effect, significance (alpha), and power (1-beta); smaller effects and stricter thresholds need more users.
Explain deferred deep linking flow
Capture link payload server-side at click, route to the store, then match the new install to the click on first launch to route the user.
Determine Sample Size for a 2% Lift A/B Test
This tests your grasp of statistical power and the business trade-offs in experimentation. A great answer defines baseline conversion rate, minimum detectable effect (MDE), and statistical power. A red flag is ignoring the business context of MDE.
Calculate Sample Size for a 2% A/B Test Lift
This tests if you connect statistical inputs to business goals. A good answer defines baseline rate, minimum detectable effect (MDE), and power, then explains MDE as a cost/benefit trade-off.
How do CI and testing support the Scrum value of Commitment?
Tests if you can connect technical practices to business value. A great answer links CI/CD to the 'Definition of Done' and explains how automated tests de-risk the sprint commitment.
Getting tech work onto a feature roadmap
Translate debt into velocity, risk, and cost impact; attach it to upcoming features; propose a sustainable allocation.
Use Difference-in-Differences without an A/B test
Give a scenario like a region-wide launch, apply Difference-in-Differences comparing treated vs control over time, and state the parallel-trends assumption.
Detect fraudulent app installs
Click-to-install timing distributions, device and IP fingerprints, post-install engagement, and attribution anomalies.
When is an A/B test not feasible, and what is DiD?
This tests your grasp of causal inference when randomization isn't possible. Explain a scenario like a state-level launch, introduce Difference-in-Differences (DiD), and state its core parallel trends assumption.
When is A/B testing not feasible, and what is an alternative?
Tests your grasp of causal inference when randomization isn't possible. A great answer names a scenario (like a regional launch), proposes Difference-in-Differences (DiD), and explains its core 'parallel trends' assumption.
How do you convince a PO to prioritize technical debt?
Tests your ability to influence without authority by translating technical issues into business impact. A great answer quantifies the cost of inaction (e.g., slowed velocity) and proposes concrete Scrum strategies like allocating 20% capacity.
Phased rollout with feature flags
Targeting rules by segment, percentage ramps, monitoring at each gate, and a fast kill switch.
Define and calculate Weekly Active Users
Define a meaningful active action, count distinct users over a rolling 7-day window, and exclude bots and background syncs.
Implement a welcome-message A/B test
Deterministic hash of a stable ID for sticky assignment, conditional rendering of the personalized variant, and exposure plus click tracking keyed to the same ID.
Define and calculate Weekly Active Users (WAU) for Slack
Tests translating a business metric to a technical spec. Define 'active' by key actions (sending messages, not just opening), then COUNT(DISTINCT user_id) on an events table, filtering out bots and background syncs. A red flag is a generic definition.
How would you define and calculate Weekly Active Users (WAU)?
This tests your product sense and technical precision in defining a core business metric. A great answer defines 'active' with specific user actions, outlines the SQL/event-based calculation, and discusses pitfalls like bots and background events.
How do you fix a disconnect between team velocity and value?
This tests your ability to distinguish output from outcome. A good answer investigates the "why" with the PO, proposes experiments to test value hypotheses, and suggests refining processes like sprint planning, rather than just blaming the PO or trying to…
Designing a tiered entitlements backend
Plans map to features and limits, subscriptions link account to plan, an entitlement check enforces access; handle upgrade/downgrade with effective dates and proration.
Cross-platform stateful onboarding sync
Store onboarding state server-side keyed to the user, expose idempotent step-completion APIs, and push updates to other clients.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles