📊 Product Management

Design quantifiable proxy metrics for a microservices velocity claim

Use DORA velocity and stability metrics plus leading indicators like build and PR cycle time.

Outline architecture for a weekly email digest of unread notifications

This tests batch processing and scheduled delivery at scale. Pre-aggregate unread counts, shard digest jobs across a distributed scheduler, and cache unsubscribes for fast filtering. Never scan the notifications table at send time for millions of users.

Explain Simpson's Paradox and construct a user engagement scenario

Tests whether you spot trends reversing when population mixes differ. Good answers define the paradox, give a numerical example with per-segment wins but aggregate loss, and warn against segment-only decisions.

Propose a multi-sprint strategy to fix an inverted test pyramid

Audit E2E for redundancy; scaffold unit/integration tests; migrate per sprint, keep critical E2E; gate on fast feedback.

Explain Simpson's Paradox with a user engagement example

Tests if you see beyond aggregate data. Define the paradox, give a numerical example where a feature fails overall but wins in segments (e.g., new vs. returning users), and name the confounding variable. A vague definition without numbers is a red flag.

How do you fix an inverted test pyramid?

This tests your ability to create a pragmatic, multi-sprint plan to improve test suite health. A good answer involves analyzing tests, getting buy-in, then incrementally adding unit/integration tests while refactoring old E2E tests.

Explain Simpson's Paradox with a user engagement example

This tests your understanding of statistical pitfalls in A/B testing. A good answer defines the paradox, gives an example where a feature fails in aggregate but wins in every segment, and attributes it to a confounding variable.

How would you refactor an inverted test pyramid?

This tests your ability to create a pragmatic, multi-sprint strategy for tech debt. A good answer includes analyzing the suite, quarantining flaky tests, enforcing new coverage standards, and incrementally refactoring. A red flag is proposing a total rewrite.

How do outcome-oriented goals change implementation and testing?

Tests whether you engineer for measurable behavioral change, not just shipping. Strong answers cover baselining, telemetry, small experiments, and user-data validation. Red flag: treating the goal as a PM issue and focusing only on on-time delivery.

Compare fan-out-on-write vs fan-out-on-read for an activity feed

Tests whether you tie feed architecture to read/write ratios and follower distribution. Strong answers contrast push O(1) reads with celebrity storms against pull O(1) writes with read amplification, then propose a hybrid threshold.

Why can't you t-test p99 latency, and what's a valid alternative?

Explain that t-tests target means while p99 variance depends on tail density; propose bootstrap CIs or permutation tests.

When should cross-cutting concerns use a shared versus duplicated solution?

Weigh change frequency and blast radius; decide via fitness function or ADR, not default shared code.

Why can't we t-test p99 latency in an A/B test?

This tests if you know why t-tests fail for percentiles. A t-test requires a normally distributed statistic (like the mean), but a sample p99's distribution isn't normal.

When to formalize a cross-cutting concern?

Tests your judgment on architectural trade-offs. A good answer uses the "Rule of Three," weighs volatility and complexity to decide, and avoids dogmatic DRY. A red flag is immediately abstracting without considering the high cost of a shared solution.

Why not t-test p99 latency? Describe a valid alternative.

This tests your grasp of statistical test assumptions. A good answer explains why p99 violates t-test normality, then outlines a resampling method like bootstrapping to build a confidence interval on the *difference* of p99s.

Shared Solution vs. Duplication for Cross-Cutting Concerns

Tests your judgment on trade-offs between duplication, coupling, and velocity. A good answer weighs the cost of premature abstraction against future refactoring, using triggers like the 'Rule of Three' and evaluating the concern's volatility.

What questions assess strategic alignment before estimating effort?

This tests whether you validate business context before scoping. Ask: what market problem is solved, which company competency or asset is leveraged, and what metric proves success. Red flag: estimating before confirming strategic fit.

Design a real-time personalized notification trigger system

Stream events to a delayed queue, expose a rule UI to non-technical users, and deliver idempotently.

Design an A/B test for a 'Buy Now' button color change

Tests structured experiment design from hypothesis to metric. Strong answers: define a falsifiable hypothesis; pick purchase conversion as primary; size the sample and duration; randomize by user; pre-commit to stopping rules.