Design petabyte-scale distributed training
WHAT IT TESTS: end-to-end big-data ML architecture. OUTLINE: object storage with columnar formats, distributed preprocessing, a data-parallel framework with efficient sharded loading, and managed orchestration.
Design auto drift detection and retraining
WHAT IT TESTS: closed-loop MLOps design. OUTLINE: capture inputs and predictions, compute data and concept drift metrics on a schedule, alert on threshold breach, and trigger a retraining and redeploy pipeline.
How would you speed up slow single-GPU training?
WHAT IT TESTS: knowledge of scaling training. OUTLINE: vertical scaling to bigger or multi-GPU instances, then data-parallel or model-parallel distributed training across nodes.
How would you build CI/CD for an ML model?
WHAT IT TESTS: MLOps maturity beyond app deployment. OUTLINE: data and model versioning, automated training plus evaluation gates, model registry, deployment with monitoring and retraining triggers.
Inference performance bottlenecks on Lambda
WHAT IT TESTS: serverless ML serving limits. OUTLINE: cold starts loading the model, memory and CPU limits, no GPU, and package size dominate; mitigate with provisioned concurrency, loading the model once outside the handler, smaller models, and right-sized…
Deploying a real-time inference endpoint
WHAT IT TESTS: model serving operations. OUTLINE: package the model artifact and inference code in a container, choose instance type and autoscaling, configure the endpoint with health checks, and plan safe rollout like canary plus monitoring.
Feeding large object-store data into training
WHAT IT TESTS: ML data loading efficiency. OUTLINE: stream data instead of copying it all to disk, use streaming/pipe modes, shard and prefetch in parallel, and pack many small images into larger files. RED FLAG: downloading the whole 1TB to local disk first.
Pre-built AI service vs custom model
WHAT IT TESTS: buy-versus-build judgment for ML. OUTLINE: choose a managed service for speed, no ML expertise, and common tasks; build custom for domain-specific needs, control, or cost at scale. RED FLAG: always building custom when a managed API suffices.
Schema evolution without rewriting history
WHAT IT TESTS: schema evolution strategy. OUTLINE: use a table format with metadata-level evolution, add a new column rather than mutating the old, and reconcile types at read time; avoid rewriting petabytes.
The small files problem in data lakes
WHAT IT TESTS: lake performance pathology. OUTLINE: too many tiny files inflate metadata and per-file overhead, slowing queries; caused by streaming micro-batches and over-partitioning; fix with compaction and table formats like Iceberg, Delta, or Hudi.
Idempotency in data ingestion pipelines
WHAT IT TESTS: reliability under retries. OUTLINE: idempotency means re-running a step yields the same result with no duplicates; it matters because retries and at-least-once delivery are inevitable; achieve it with deduplication keys or upserts.
Handling late data in streaming windows
WHAT IT TESTS: event-time stream processing. OUTLINE: use event-time windows with watermarks to bound lateness, allow a grace period before finalizing, and route data later than that to a side output.
Diagnosing and fixing data skew in Spark
WHAT IT TESTS: distributed processing skew. OUTLINE: this is data skew, caused by uneven key distribution concentrating rows on few partitions; mitigate with salting, broadcast joins, repartitioning, or adaptive execution. RED FLAG: just adding more executors.
Partitioning order events in a data lake
WHAT IT TESTS: partition design for query pruning. OUTLINE: partition by the columns queries filter on, typically date hierarchy and category, balancing granularity to avoid too many tiny files. RED FLAG: partitioning on high-cardinality keys like order ID.
CSV vs JSON vs Parquet for analytics
WHAT IT TESTS: file format tradeoffs. OUTLINE: CSV and JSON are row-based, human-readable, and bulky; columnar Parquet/ORC compress well and read only needed columns; choose columnar for analytics. RED FLAG: defaulting to CSV for large analytical workloads.
ETL versus ELT in cloud data platforms
WHAT IT TESTS: data pipeline pattern fluency. OUTLINE: ETL transforms before loading into the target; ELT loads raw first then transforms in the warehouse, leveraging cheap storage and elastic compute.
Data lake versus data warehouse
WHAT IT TESTS: storage architecture fundamentals. OUTLINE: lakes store raw, schema-on-read data of any type cheaply; warehouses store curated, schema-on-write structured data for fast SQL; choose a lake for varied raw data and ML.
Resilient stateful batch on Spot Instances
WHAT IT TESTS: fault tolerance on interruptible compute. OUTLINE: externalize state and checkpoint to durable storage, react to interruption and rebalance notices to drain gracefully, diversify instance pools.
Designing a multi-account cloud chargeback model
WHAT IT TESTS: cost allocation architecture. OUTLINE: account-per-team or mandatory cost-allocation tags enforced by SCPs and tag policies, plus a pipeline over the cost and usage report grouped by tag/account. RED FLAG: relying on voluntary tagging.
Savings Plans vs Reserved Instances for mixed compute
WHAT IT TESTS: commitment-discount strategy. OUTLINE: Compute Savings Plans cover EC2, Fargate, and Lambda flexibly; EC2 Instance Plans and RIs trade flexibility for slightly deeper discounts. RED FLAG: over-committing or ignoring usage variability.