User session length dropped 15%: what technical issues and data to check?

WHAT THIS TESTS: Whether you can debug a metric drop methodically rather than guessing. Interviewers want to see you prioritize data quality and system changes before blaming users, and that you know which logs and dimensions actually isolate root cause.

A GOOD ANSWER COVERS: Four categories in order. First, exposure and traffic anomalies: check cumulative assignment counts or user funnel volumes to confirm the drop is real and not a sampling or traffic allocation issue. Second, releases and deployments: correlate the timing with code releases, app version updates, or feature flag changes; use deployment annotations and configuration history to narrow the window. Third, platform and version segmentation: split by app version, OS version, platform, and device type because a bug often hits only one segment; Amplitude tracks these properties automatically and they are reliable debugging dimensions. Fourth, instrumentation and data quality: look for duplicate events from re-rendering components, missing session end triggers, rapid-fire events from a single user, or bot traffic skewing the baseline; use frequency distributions and bot filters to verify.

COMMON WRONG ANSWERS: Jumping straight to user behavior or market seasonality without first ruling out technical causes. Blaming the metric definition without checking if the underlying event stream changed. Suggesting A/B test results or product changes before validating data integrity. Failing to segment by platform or version, which hides localized bugs. Proposing vague solutions like check the logs without naming specific charts or dimensions.

LIKELY FOLLOW-UPS: How would you differentiate a real user behavior shift from a tracking bug? What if the drop only affects iOS users on a specific app version? How do you handle bot traffic if your current filters are missing a new bot pattern? When would you decide the data is trustworthy enough to act on? How would you communicate a confirmed instrumentation bug to leadership?

ONE CONCRETE EXAMPLE: Suppose your session length dropped 15 percent on Tuesday. You pull the cumulative exposures chart and see orange anomaly dots marking a traffic drop, so you check the experiment configuration history and find a teammate changed a feature flag rollout percentage on Monday night. You then segment by app version and see the dip is isolated to version 3.2.1 on Android. Checking the event frequency chart reveals the session end event is not firing for that version after a backgrounding change, making sessions appear artificially short. You roll back the flag and file a hotfix for the missing event trigger.