How do you systematically manage and pay down experiment debt?

WHAT THIS TESTS: This question tests whether you understand that growth engineering velocity is a function of sustainable systems, not just shipping speed. Interviewers want to see if you can design processes that keep experiment code from permanently polluting the production codebase while still preserving the fast iteration cycles that growth teams require. The core tension is between allowing engineers to move quickly and preventing abandoned branches, stale feature flags, and untested paths from calcifying into technical debt.

A GOOD ANSWER COVERS: First, structural separation in the codebase. Mention isolating experiment code into dedicated directories or using component wrappers so that throwaway logic is visually distinct from production code. This makes cleanup obvious and allows you to enforce different code coverage or style standards for experiments. Second, enforce time-to-live policies on feature flags and experiments, paired with automated stale-experiment detection and a recurring cleanup ritual such as a quarterly debt sprint. Third, require a productionization gate: before experiment code graduates, it must meet standard testing and observability bars. Fourth, safety tooling like error boundaries to prevent a crashing experiment from taking down an entire page, plus automated test suites that verify key tests still pass under specific experiment subsets. Fifth, shared accountability: the PM or growth team owns experiment hygiene, not just engineering, and the experiment results dashboard must be thorough enough to kill losing tests quickly.

COMMON WRONG ANSWERS: A major red flag is proposing a blanket ban on experiments or adding heavyweight code review that destroys iteration speed. Another is treating experiment debt as purely an engineering problem without involving PMs in cleanup prioritization. Suggesting manual tracking spreadsheets instead of automated flag TTLs shows a lack of tooling imagination. Finally, ignoring safety mechanisms like error boundaries implies you have not operated in a high-velocity environment where broken experiments directly impact revenue.

LIKELY FOLLOW-UPS: The interviewer may ask how you would handle native mobile experiment debt where app store releases slow down cleanup, pushing you toward solutions like React Native Code Push, Flutter, server-driven UI, or webviews. They might also probe how you balance different test coverage standards for throwaway code versus productionized code, or how you prevent an experiment from negatively impacting core metrics through automated alerts and on-call rotations.

ONE CONCRETE EXAMPLE: At a previous company, we created an experiments directory adjacent to production features and required every experiment flag to have an expiration date in our configuration. If a flag exceeded its TTL without a decision, it auto-alerted the owning PM and engineer. Winning variants were promoted out of the experiments directory into the main codebase with full test coverage, while losing variants were deleted in a weekly cleanup job. We also wrapped all experiment components in error boundaries, ensuring a broken A/B test never 500-ed the entire landing page. This kept our deploy velocity at multiple times per day while keeping the main codebase clean.

Source: alexeymk.com