Design a Schema Validation System for Analytics Events

What's really being asked

This question tests your understanding of data governance trade-offs. The interviewer is looking for a nuanced answer that balances the need for pristine data with practical realities like developer experience, client-side performance, and the cost of server-side processing. It's about designing a robust, multi-layered system, not just a simple validation function.

The full answer

A good answer covers four key areas in order. First, establish a source of truth by defining a centralized, version-controlled 'Tracking Plan' or schema registry (e.g., using JSON Schema, Protobuf). Second, discuss client-side enforcement, which should be non-blocking and primarily for developer feedback in dev builds to catch errors early. Third, detail server-side enforcement, which is the critical, authoritative gate where every event is validated upon ingestion. This is non-negotiable for data integrity. Fourth, explain failure handling: invalid events are never silently dropped but are routed with error details to a 'violations' stream or dead-letter queue (e.g., a separate Kafka topic) for analysis.

The mistakes people make

A major red flag is suggesting enforcement only on the client. It is naive to trust the client, as it can be bypassed, old versions exist, and malicious actors can send anything. Another weak answer is enforcing only on the server; this is better but misses the chance for fast developer feedback, slowing down iteration. The worst answer is to simply drop invalid events. This constitutes data loss and prevents you from analyzing the root cause of the failure. Finally, suggesting that production client-side validation should block event submission is a performance anti-pattern; analytics should never break core application functionality.

What usually comes next

Expect questions about schema evolution and versioning (support both old and new schemas during a transition period). You might be asked how to manage the cost of the system (server-side compute, efficient dead-letter storage). Another common follow-up is how to handle a bug causing 100% of events to fail, which tests your knowledge of alert rate-limiting or circuit breakers to prevent alert storms.

A concrete example

A user submits a 'Product Viewed' event with a 'price' property as a string "29.99" instead of the number 29.99 required by the Tracking Plan. The client-side dev build console logs an error. The event is still sent. The server-side ingestion service validates it, sees the type mismatch, and routes the original event plus an error object to the 'analytics-violations' Kafka topic. It also increments a Prometheus counter validation_errors{event="Product Viewed", reason="type_mismatch"}. An analyst can later query the violations topic, and an engineer is alerted if this error rate spikes above 5% in a 5-minute window.