More in Product Management — page 3
Technical principles for building a learning-focused MVP
WHAT IT TESTS: building to learn, not to scale. OUTLINE: validate the riskiest assumption first, use off-the-shelf and manual where possible, instrument for learning, defer scalability and polish.
How vision, strategy, and roadmap relate
WHAT IT TESTS: grasp of the strategy hierarchy. OUTLINE: vision is the long-term aspiration, strategy is the chosen path to it, the roadmap is time-bound execution that delivers the strategy.
Define and calculate Weekly Active Users
WHAT IT TESTS: turning 'active' into a defensible metric. OUTLINE: define a meaningful active action, count distinct users over a rolling 7-day window, and exclude bots and background syncs.
Use Difference-in-Differences without an A/B test
WHAT IT TESTS: causal inference when randomization is impossible. OUTLINE: give a scenario like a region-wide launch, apply Difference-in-Differences comparing treated vs control over time, and state the parallel-trends assumption.
Determine A/B test sample size
WHAT IT TESTS: the inputs to a power calculation. OUTLINE: define baseline rate, minimum detectable effect, significance (alpha), and power (1-beta); smaller effects and stricter thresholds need more users. RED FLAG: ignoring power or treating MDE as fixed.
Design analytics event schema validation
WHAT IT TESTS: protecting data quality at scale. OUTLINE: define a schema registry, validate at both client (fast feedback) and server (authoritative gate), and quarantine failures to a dead-letter store.
Calculate MRR with SQL including annual plans
WHAT IT TESTS: correct MRR definition and date filtering. OUTLINE: sum monthly_price for subscriptions active this month, filter on start and end dates, and normalize annual plans by dividing annual price by 12.
Interpret a p-value in an A/B test
WHAT IT TESTS: correct meaning of a p-value. OUTLINE: define it as P(data this extreme | null true), interpret 0.03 against a 0.05 threshold, and state what it is NOT. RED FLAG: claiming a 3% chance the result is wrong or that the effect is.
Define a consistent day across timezones
WHAT IT TESTS: consistent date bucketing across global timezones. OUTLINE: store events in UTC, capture the local/source timezone, then convert to a single reporting timezone at query time.
Implement CDC from OLTP to warehouse
WHAT IT TESTS: low-impact replication of inserts, updates, and deletes. OUTLINE: contrast log-based CDC with query-based timestamp polling, cover deletes and load on source, then pick log-based for minimal impact.
Design a multi-touch attribution model
WHAT IT TESTS: spreading credit across a long journey. OUTLINE: pick a model (time-decay, position-based, or data-driven Shapley), stitch touchpoints by user identity into ordered paths, then assign fractional credit.
Reframe time series for a tree model
WHAT IT TESTS: turning forecasting into supervised learning. OUTLINE: lag and rolling-window features, calendar and cyclical encodings, then split chronologically to avoid leakage. RED FLAG: random shuffling that lets future data leak into training.
Find leading indicators of long-term churn
WHAT IT TESTS: turning a lagging metric into early signals. OUTLINE: cohort renewers vs churners, compare first-30-day engagement depth and breadth, validate correlations and check causality. RED FLAG: jumping to a model before defining the cohort and target.
Calculating Daily Active Users in SQL
WHAT IT TESTS: Metric definition plus dedup SQL. OUTLINE: Need per-event user_id and timestamp and a clear active definition; count distinct user_id within the day in a fixed timezone. RED FLAG: Counting rows or fuzzy date-boundary and timezone handling.
Client vs server tracking: pros, cons, examples
WHAT IT TESTS: Tracking placement trade-offs with examples. OUTLINE: Client-side wins on UI context but loses data to blockers and tampering; server-side wins on reliability and trust but misses pure UI events. RED FLAG: Picking one for everything.
Building a conversion funnel in SQL
WHAT IT TESTS: Funnel SQL and drop-off reasoning. OUTLINE: Count distinct users reaching each ordered step, compute step-over-step conversion; the biggest drop-off is the lowest consecutive ratio. RED FLAG: Comparing each step to the total, or counting events.
Bundled analytics vs warehouse-native trade-offs
WHAT IT TESTS: Judgment on analytics stacks. OUTLINE: Warehouse-native gives one source of truth and SQL flexibility but shifts modeling, performance, and UX onto your team; bundled tools are turnkey but siloed. RED FLAG: Framing it as cost alone.
Investigating analytics vs database count gaps
WHAT IT TESTS: Data-quality debugging. OUTLINE: Causes include ad-blocker loss, differing identity logic, timezone mismatches, filtering, and pipeline delay; investigate by aligning definitions and tracing one user. RED FLAG: Trusting one source blindly.
Architecting multi-touch attribution
WHAT IT TESTS: Identity stitching and credit assignment. OUTLINE: Ingest touchpoints, resolve to one identity, order into paths, apply a model; last-touch is trivial, time-decay needs the full path. RED FLAG: Ignoring cross-device identity.
SQL for a three-step onboarding funnel
WHAT IT TESTS: Funnel SQL with ordering correctness. OUTLINE: Anchor the 30-day signup cohort, count distinct users reaching each later step in timestamp order; conversion is each step over the prior. RED FLAG: Counting any occurrence regardless of order.