Feeding large object-store data into training
ML data loading efficiency.
stream data instead of copying it all to disk, use streaming/pipe modes, shard and prefetch in parallel, and pack many small images into larger files.
downloading the whole 1TB to local disk first.
What's really being asked
This evaluates whether you can keep an expensive accelerator fed by designing an efficient data pipeline from object storage rather than letting IO stall training.
The full answer
The naive approach of downloading the entire terabyte to the instance's local disk before training wastes time, requires large attached storage, and delays the start. Instead, stream the data. Managed services offer modes for this: SageMaker provides pipe mode and fast file mode that stream objects from S3 on demand, and Vertex AI supports similar streaming and mounted access, so the job reads as it trains. Shard the dataset across data-loading workers and across distributed training nodes so reads happen in parallel and each worker pulls a distinct slice. Use prefetching and a data loader that overlaps IO and decoding with GPU compute, so the next batch is ready before the current one finishes. Crucially, address the many-small-files issue: a terabyte of individual image files means a huge number of small object reads, each with request overhead, so pack images into larger sequential record files such as TFRecord, WebDataset tar shards, or RecordIO, which lets the loader stream large contiguous reads and shuffle within shards. Also colocate the training job in the same region as the bucket to avoid transfer cost and latency.
The mistakes people make
Downloading all data to local disk first. Reading millions of tiny image objects one by one with no packing, drowning in request overhead. Forgetting to shard across workers, serializing IO. Ignoring prefetch so the GPU idles waiting on data. Training in a different region from the bucket.
What usually comes next
How do you shuffle when streaming. What file size to target for shards. How do you monitor whether IO is the bottleneck.
A concrete example
You repackage a terabyte of JPEGs into WebDataset tar shards of a few hundred megabytes each, stream them with fast file mode, and let multiple loader workers prefetch and decode in parallel. The GPU stays near full utilization instead of waiting on per-image S3 requests.
Interview question
Why repackage a terabyte of individual image files into larger record shards before training?
- a.It reduces per-object request overhead and enables efficient streaming readsCorrect
- b.It eliminates the need to shard across workers
- c.It is required before any model can read images
- d.It compresses the images to a smaller total size
Why? this is the answer
Packing many tiny files into large shards turns millions of small, high-overhead object reads into efficient sequential streams. It does not inherently shrink the data or remove the benefit of sharding across workers.
Just read this? Test yourself on what you have been reading.
Read the original → aws.amazon.com
- #cloud
- #machine-learning
- #data-loading
- #sagemaker
- #training
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on cloud — each one lists the topics its interview covers.
See open roles