tezvyn:

ELT: Load Raw Data First, Transform It Later

AI-drafted, machine-checkedSource: Wikipedia: Extract, load, transformbeginner

ELT pipelines load raw data directly into a data lake *before* any transformation. This speeds up ingestion and lets you figure out the data's structure later.

THE MENTAL MODEL: Extract, Load, Transform (ELT) is a data pipeline approach that prioritizes speed and flexibility by changing the order of operations. Unlike its predecessor, ETL (Extract, Transform, Load), ELT loads raw data into storage first and worries about transforming it later, at the moment it's needed.

HOW IT WORKS: The process has three stages. First, data is EXTRACTED from its source systems. Second, it is immediately LOADED into a data lake or modern cloud data warehouse in its original, raw format. Third, the TRANSFORMATION happens on-demand when a user or application queries the data. This is often called "schema-on-read," because the structure is applied at query time, not when the data is stored.

WHEN TO USE IT: ELT shines in big data environments that use a data lake. It's ideal when you need to ingest massive volumes of data quickly and when the final use cases or required data structures are not yet known. This gives data scientists and analysts the flexibility to explore the raw data for new insights without waiting for engineering to build a transformation pipeline.

WHEN NOT TO USE IT: ELT is a poor choice if your data processing engine lacks power. Because transformations run at query time, an under-resourced system will lead to very slow query performance. If you need highly structured, standardized data for consistent reporting and your transformation logic is stable, a traditional ETL approach might be more efficient.

ONE CANONICAL EXAMPLE: A company wants to analyze user clickstream data from its mobile app. Using an ELT model, they dump the raw JSON event data directly into a cloud data warehouse. Later, an analyst can write a SQL query that parses the JSON, filters for specific events, and joins it with user data on the fly to calculate a new metric. The data was available for querying moments after being generated, not days later.

Read the original → en.wikipedia.org

Get five bites like this every day.

Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.