Explain event schemas and the purpose of a schema registry

WHAT THIS TESTS: This question isn't about simply defining a schema. It tests your experience with the operational pain of data entropy in a scaled engineering organization. The interviewer wants to know if you can connect a technical tool (a schema registry) to its business value: enabling team autonomy, ensuring data quality, and preventing cascading failures in distributed systems. It's a test of senior-level thinking about contracts and governance over simple definitions.

A GOOD ANSWER COVERS: A strong answer addresses three points in order. First, define an event schema as a formal, machine-readable contract for event data, specifying fields, types, and validation rules (e.g., using Avro, Protobuf, or JSON Schema). Second, describe a schema registry as a centralized, API-driven service that stores these schemas and, crucially, enforces compatibility rules (like backward, forward, or full compatibility) when schemas evolve. This allows producers and consumers to evolve independently. Third, detail the specific problems that arise without one: "poison pill" messages that crash downstream consumers when a producer makes a breaking change; brittle analytics where dashboards and reports break silently; and decreased developer velocity because every data format change requires manual, cross-team coordination.

COMMON WRONG ANSWERS: A common mistake is to give a textbook definition of a schema but fail to explain the role of the registry as an active enforcement mechanism. A major red flag is describing the registry as just a "wiki for schemas" or a documentation tool; its primary value is automated governance and compatibility checking. Another weak answer focuses only on producers and consumers, forgetting the impact on the data analysis and business intelligence side, which is often the most painful and visible consequence of poor data quality. Finally, failing to mention compatibility types (backward, forward) shows a lack of depth.

LIKELY FOLLOW-UPS: Expect questions that dig into the details of compatibility. For example, "Explain the difference between backward and forward compatibility, and when you'd choose one over the other." Or, "Imagine you're introducing a schema registry into a company with 100 existing Kafka topics that are un-schematized. What's your rollout strategy?" They might also ask about specific formats: "Why might you choose Avro over JSON Schema for a high-throughput system?"

ONE CONCRETE EXAMPLE: Imagine an OrderPlaced event. The initial schema (V1) is { "order_id": "string", "total_price_cents": 10000 }. A producer team wants to add an optional discount_code. With a registry set to backward compatibility, they can register V2: { "order_id": "string", "total_price_cents": 10000, "discount_code": "string" (optional) }. Existing consumers won't break. Without a registry, another team might unilaterally change total_price_cents to a float total_price_dollars: 100.00. A downstream fraud detection service expecting an integer for cents would now fail to parse the message or, worse, interpret 100.00 as 100 cents, mis-evaluating a 50,000 order as a $500 one and letting it pass.