Schema-on-read in data lakes
understanding deferred schema application.
structure is applied at query time not ingest, enabling flexible raw storage and ML, but costing query-time validation and risking data swamps.
WHAT THIS TESTS This checks whether you understand that schema-on-read defers structure to query time, and can weigh its benefits against its real costs, rather than treating it as schemaless chaos.
A GOOD ANSWER COVERS Schema-on-read means data is stored in its raw form and the schema, the interpretation of fields and types, is applied only when the data is read or queried. The same underlying file can be parsed under different schemas by different consumers. This is the defining trait of a data lake because lakes accept any format on cheap object storage and avoid forcing a model at ingest, the opposite of a warehouse's schema-on-write.
ADVANTAGES Ingestion is fast and cheap because no transformation or validation is required up front. It is flexible: you can store data whose future use is unknown and decide later how to interpret it. It naturally supports semi-structured and unstructured data and exploratory machine learning where rigid schemas would be premature.
DISADVANTAGES Validation and parsing happen at query time, so every read pays a cost and bad data is discovered late rather than rejected on entry. Without governance, cataloging, and quality checks, the lake degrades into a data swamp where consumers cannot trust or find data. Query performance can suffer compared to a pre-modeled warehouse.
COMMON WRONG ANSWERS Saying schema-on-read means there is no schema; there is one, applied later. Claiming it is always superior to schema-on-write, ignoring validation and performance costs.
LIKELY FOLLOW-UPS How does a data catalog help. How do table formats like Iceberg add schema enforcement to lakes. When is schema-on-write better.
ONE CONCRETE EXAMPLE Raw JSON event logs land in S3 untouched. Months later, an analyst defines a table over them with a schema picking out user_id and event_type, and a data scientist reads the same files with a broader schema including nested device fields, each imposing structure at read time.
Read the original → dremio.com
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.