How would you debug a sudden drop in a key metric?
Tests your systematic debugging of complex data systems. First, validate the drop against a source of truth and segment the data. Then, hypothesize causes like misattribution or data loss.
What's really being asked
This question tests your ability to apply a structured, systematic debugging process to a complex system involving frontend clients, backend services, and third-party analytics platforms. The interviewer is looking for a senior engineer who doesn't just guess but methodically narrows down possibilities. They want to see you move from broad validation to specific, testable hypotheses, distinguishing between data loss and misattribution.
The full answer
First, establish a source of truth. Compare the analytics tool's numbers (e.g., GA4) against your backend database or CRM (e.g., Shopify orders). Is the data missing entirely, or just misattributed?
Second, segment the data. Look for patterns in the drop. Is it consistent across all dimensions (device, browser, country, channel), or isolated to one? A 30% drop across everything suggests a systemic issue. A drop only on mobile Safari suggests a platform-specific problem.
Third, form hypotheses based on the pattern. For a systemic drop, suspect changes in the analytics platform itself, a third-party script update (e.g., ad blockers), or processing delays. For misattribution (e.g., a spike in 'Direct' traffic), suspect lost UTM parameters or broken cross-domain tracking.
Fourth, outline a verification plan. Start with non-invasive checks like browser developer tools, real-time analytics views, and checking third-party status pages. Escalate to inspecting raw event logs and data pipeline configurations if needed.
The mistakes people make
Jumping to a specific, complex technical cause without segmenting the data first. This shows a lack of methodical thinking. For example, immediately blaming a CDN configuration in a specific region before checking if the drop is global.
Immediately blaming the analytics tool as 'buggy' without evidence. This is unconstructive and avoids ownership of the problem.
Failing to establish a source of truth. Any analysis of analytics data in a vacuum is just speculation. You must compare it to what your system actually recorded.
Confusing data processing delays with permanent data loss. A senior engineer should know to wait 2-3 days for data to be fully processed before declaring a permanent drop.
What usually comes next
What if the drop is a consistent 30% across all segments? (This points to a systemic issue: an analytics platform change, a widely deployed ad-blocker list update, or a change in a shared third-party dependency.)
How would you differentiate between a processing delay and true data loss? (Check the numbers for the affected day again after 48-72 hours. If they have recovered, it was a delay. If they remain low, it's persistent loss.)
What monitoring could prevent this from being a surprise next time? (Automated alerting that runs daily, comparing backend conversion counts against the analytics platform's counts and flagging any discrepancy over a threshold like 5%.)
A concrete example
Our team saw a 30% drop in 'purchase' events in GA4 compared to our Shopify backend. We segmented by device, country, and channel, and the ~30% drop was consistent everywhere. This ruled out a specific browser or regional issue. Our hypothesis shifted to a systemic problem. We waited 72 hours, and the numbers remained low, confirming it wasn't just processing lag. This pointed to either a change in GA4's data processing or a new, widely adopted filter that was stripping our event metadata before it reached Google's servers.
Interview question
A key metric suddenly drops. Which initial action best demonstrates a systematic debugging approach?
- a.Immediately check the status pages of all third-party services and CDNs.
- b.Assume a data processing delay and wait 48-72 hours before taking any action.
- c.Report the issue to the analytics platform vendor, assuming a bug in their system.
- d.Compare the metric against a reliable backend source and segment the data by relevant dimensions.Correct
Why? this is the answer
Option D outlines the critical first steps of establishing a source of truth and segmenting data, which are essential for systematically narrowing down potential causes. Option A is a premature, specific technical check that should only occur after initial validation and pattern identification.
Just read this? Test yourself on what you have been reading.
Read the original → stape.io
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on analytics — each one lists the topics its interview covers.
See open roles