How do you root-cause a 20% revenue drop with no pipeline failures?

WHAT THIS TESTS: This question tests whether you treat a dashboard as a hypothesis rather than ground truth. Senior engineers are expected to lead incident investigations that distinguish between genuine business shifts and invisible data integrity failures. The interviewer wants to see structured skepticism, knowledge of the full data stack from ingestion to presentation, and the ability to communicate confidence levels to executives before declaring a root cause.

A GOOD ANSWER COVERS: A strong response follows a triage funnel. First, verify the metric definition by checking the semantic layer and recent code or config changes to the dashboard query. Second, reconcile the aggregate number against raw event logs and source-of-truth tables to see if the drop exists upstream or was introduced in transformation. Third, slice the 20% decline by high-cardinality dimensions such as region, platform, product line, or user cohort to identify whether a specific segment went to zero, which strongly suggests a silent pipeline gap rather than a market downturn. Fourth, inspect orchestration metadata for late-arriving partitions, stale snapshots, or schema drift in join keys that would not raise a task failure alert. Fifth, correlate with external business signals like traffic volume, conversion funnels, or payment processor logs to build a confidence score for a real demand shift versus a data bug.

COMMON WRONG ANSWERS: The biggest red flag is immediately calling the drop a business downturn without first proving data integrity. Another weak pattern is suggesting to rerun the pipeline blindly; since there are no failures, this wastes time and masks partial data loss. Candidates who stay only in the BI layer and never ask to see raw ingestion logs reveal shallow systems thinking. Proposing to roll back the dashboard version without checking upstream dependencies is also a trap.

LIKELY FOLLOW-UPS: The interviewer may ask how you would communicate uncertainty to the CFO while the investigation is open. They might also push on how to automate detection of silent failures in the future, or ask you to design a data reconciliation framework that compares warehouse aggregates to source events in real time.

ONE CONCRETE EXAMPLE: Suppose daily recurring revenue drops 20%. You query the raw billing events table and find the count is normal, but the revenue per event dropped. You slice by currency and see that a new exchange rate feed started producing nulls for EUR transactions, causing them to be excluded from the aggregate. The pipeline completed successfully because nulls are valid in the schema, but the join filter silently removed a geographic segment. This is a data integrity bug, not a business downturn.

Source: newrelic.com