How would you design a data quality monitoring system?

What's really being asked

This question assesses your ability to move from abstract principles like "high-quality data" to a concrete, operational system. The interviewer wants to see if you can think systematically about data reliability, not just code correctness. They are testing your understanding that data has its own lifecycle and failure modes separate from the application code that processes it. It's a test of operational maturity and risk management for data assets.

The full answer

A strong answer has four parts. First, frame the problem by defining "quality" for this specific use case: executive dashboards need timely, complete, and consistent data. Second, detail specific checks across key dimensions. For freshness, check that data isn't older than a defined SLA (e.g., 1 hour). For volume, check row counts are within a historical range (e.g., +/- 20% of the 7-day moving average). For distribution, check for unexpected changes in key categorical values or null rates (e.g., country_code nulls > 1%). For schema, validate column names, types, and order. Third, describe the monitoring system's architecture: where the checks run (e.g., as part of the ETL/ELT job, using a tool like Great Expectations or dbt tests), and where results are stored. Fourth, outline a tiered alerting strategy: PagerDuty for critical failures like pipeline stoppage or major volume drops, and Slack/email for warnings like minor distributional shifts.

The mistakes people make

A major red flag is listing checks without context. For example, just saying "I'd check for nulls." A better answer is "I'd check the null percentage in the user_id column, because a spike above 0.5% indicates a critical upstream problem." Another weak answer is proposing a system that is too complex or expensive for the problem, or one that alerts on everything, creating alert fatigue. Candidates who only focus on application-level bugs and not the data itself are also missing the point.

What usually comes next

How would you handle a backfill or historical data restatement? How would you differentiate between a warning and a critical alert? How would you version your data quality rules? What if a check fails—do you block the pipeline or let the data flow with a warning? (The answer depends on the criticality and the specific check).

A concrete example

For a daily sales pipeline feeding a dashboard, a key freshness check would be to ensure the max(transaction_date) is yesterday's date by 8 AM local time. A volume check would be to ensure count(transactions) is within 15% of the same day last week. If the volume drops by 50%, this is a critical alert that pages the on-call engineer. If it drops by 10%, it's a warning to a team Slack channel.