Sudden metric drop, no recent deployments. What's the cause?
This tests your ability to debug data discrepancies beyond code, focusing on the analytics pipeline. First, distinguish data loss from misattribution. Then, check processing delays and hidden data sources. A red flag is not segmenting data first.
What's really being asked
This question assesses your structured debugging process for data integrity issues, specifically when the cause isn't an obvious code change. It tests your understanding that analytics "truth" is often relative and depends on the measurement tool, its configuration, and its processing latency. Interviewers want to see if you can move beyond "the code is the problem" and reason about the entire data pipeline, from event generation to reporting.
The full answer
A good answer has three main parts. First, establish a source of truth and classify the problem. Compare the analytics tool's numbers (e.g., GA4) against a backend system (e.g., Shopify, CRM). If totals differ significantly, it's data loss. If totals are similar but channel numbers are off, it's misattribution. Second, for data loss, segment the data. Check if the drop is consistent across all dimensions like device, country, and channel. A uniform 30% drop across all segments suggests a systemic issue. Third, propose potential causes. These could include data processing delays (always use data at least 2-3 days old), differences in attribution models between platforms, or the introduction of un-instrumented data sources, like sales from a new physical store that are in the CRM but not tracked on the front end.
The mistakes people make
A weak answer immediately assumes a highly technical bug in the tracking snippet without a diagnostic process. Another red flag is confusing data loss with misattribution; for example, suggesting attribution model changes as a fix for a 30% drop in total conversions. Candidates also fail when they don't ask clarifying questions to establish a "source of truth" to compare against, instead treating the analytics platform as infallible. Blaming data sampling in the analytics tool without first checking if the report is actually sampled is another common misstep.
What usually comes next
"Let's say you find the drop is only on mobile devices. What's your next step?" (This probes deeper into platform-specific debugging). "How would you set up a monitor to catch this kind of issue proactively?" (This tests system design and operational maturity). "You mentioned differing attribution models. How would you explain this discrepancy to the marketing team without getting overly technical?" (This tests communication and stakeholder management).
A concrete example
"We saw a 30% drop in GA4 purchase events compared to our Shopify backend. Totals were off, so it was data loss. Segmenting the data showed the 30% loss was consistent across all channels, countries, and devices. This uniformity ruled out a browser or regional issue. The investigation revealed the company had recently started taking phone orders, which were entered directly into Shopify but never triggered a GA4 event. The 'missing' 30% of conversions were from this new, un-instrumented offline channel. The fix was to implement offline conversion tracking to send these backend events to GA4."
Interview question
Analytics report a 30% drop in conversions, but backend sales are stable. The drop is uniform across all segments. What is the most plausible explanation for this discrepancy?
- a.The analytics report is based on sampled data, causing an undercount of total conversions.
- b.A recent, silent browser update is preventing the tracking script from executing.
- c.The analytics platform changed its attribution model, affecting how conversions are credited.
- d.A new, un-instrumented sales channel was introduced, such as phone orders.Correct
Why? this is the answer
This is a classic data loss scenario, where events happen but are not tracked. A new, un-instrumented channel explains why backend totals are stable while analytics totals drop. An attribution model change (C) would only reallocate conversions between channels, not change the total count.
Just read this? Test yourself on what you have been reading.
Read the original → stape.io
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on analytics — each one lists the topics its interview covers.
See open roles