Risks of optimizing recommendations only for CTR
WHAT IT TESTS: metric design and Goodhart's law. OUTLINE: CTR-only invites clickbait, low satisfaction, and long-term churn; add counter-metrics like dwell time, satisfaction, retention, and diversity.
Federated learning architecture, risks, and defenses
WHAT IT TESTS: privacy-preserving distributed training. OUTLINE: devices train locally and send updates not data, a server aggregates; gradients still leak, enabling inversion or membership inference; defend with secure aggregation and DP.
Differential privacy, epsilon, and noisy aggregates
WHAT IT TESTS: rigorous privacy guarantees. OUTLINE: define DP as bounded output change when one record is added or removed, explain epsilon as the privacy-accuracy knob, add calibrated noise scaled to sensitivity.
k-anonymity and its limits against linkage attacks
WHAT IT TESTS: anonymization fundamentals. OUTLINE: define k-anonymity via indistinguishable quasi-identifier groups, apply generalization and suppression, then show homogeneity or linkage attacks break it.
Right to be forgotten and machine unlearning
WHAT IT TESTS: deletion in trained models. OUTLINE: delete raw data everywhere, then remove influence via full retraining, SISA sharded retraining, or approximate unlearning; note cost and verification.
Quantifying performance disparity across subgroups
WHAT IT TESTS: fairness evaluation beyond accuracy. OUTLINE: compute per-group precision, recall, FPR, FNR, compare via fairness metrics; visualize with grouped bars or per-group confusion matrices. RED FLAG: trusting one global accuracy number.
Programmatically detect and redact PII in text
WHAT IT TESTS: PII handling in text pipelines. OUTLINE: regex for structured PII plus NER for names and places, redact or tokenize, then validate recall. RED FLAG: relying on regex alone or trusting one pass without measuring misses.
Minimizing model prediction latency end to end
WHAT IT TESTS: latency optimization across layers. OUTLINE: shrink the model via quantization, pruning, distillation, compilation; cut infra latency via caching, edge, faster runtimes. RED FLAG: only buying bigger hardware or ignoring accuracy tradeoffs.
Zero-downtime model updates with blue-green or canary
WHAT IT TESTS: safe rollout design. OUTLINE: blue-green swaps full traffic after validation, canary ramps a small slice; both need health, latency, and quality monitoring plus instant rollback. RED FLAG: only watching system metrics, not model quality.
Why version data and model artifacts, not just code
WHAT IT TESTS: reproducibility discipline. OUTLINE: code alone cannot reproduce a model; data and artifact versioning enable rollback, debugging, audit. RED FLAG: thinking Git on code suffices, or ignoring rollback and regulatory traceability.
CPU versus GPU serving: cost, latency, throughput
WHAT IT TESTS: inference hardware tradeoffs. OUTLINE: GPUs win on throughput for batched parallel work but cost more; CPUs suit low-volume or small models. RED FLAG: claiming GPU is always faster or ignoring batching and utilization.
ML CI/CD versus traditional software CI/CD
WHAT IT TESTS: MLOps pipeline understanding. OUTLINE: validates code plus data plus the model, auto-trains and evaluates, adds continuous training and monitoring. RED FLAG: describing only code CI/CD and ignoring data, model gates, and retraining.
What a feature store solves: skew and consistency
WHAT IT TESTS: feature-store value. OUTLINE: central repository of computed features, one definition serving training and inference, reuse across models. RED FLAG: calling it just a database or ignoring the offline-online consistency guarantee.
Detecting and responding to model and concept drift
WHAT IT TESTS: production monitoring. OUTLINE: define drift, pick a metric like PSI or falling AUC against labels, then investigate, retrain, validate. RED FLAG: assuming static accuracy or retraining blindly without diagnosing the cause.
Batch prediction versus online real-time prediction
WHAT IT TESTS: serving pattern selection. OUTLINE: batch is scheduled bulk scoring, online is low-latency per-request scoring; contrast latency, freshness, cost; give a use case each. RED FLAG: confusing batch with retraining or claiming online is always best.
Deploy a saved model as a REST prediction service
WHAT IT TESTS: end-to-end deployment basics. OUTLINE: load the artifact, wrap it in a predict API, containerize, host with autoscaling, add monitoring. RED FLAG: jumping to model training or forgetting preprocessing parity and input validation.
Explain k-means user segments to a marketing team
WHAT IT TESTS: turning clustering output into actionable personas. OUTLINE: name each segment, profile its defining traits, show size and value, recommend an action. RED FLAG: explaining centroids and inertia instead of who the segments are.
Two ways accurate data can still mislead in a chart
WHAT IT TESTS: visualization honesty. OUTLINE: name distortions like truncated axes or cherry-picked ranges, give the fix for each, explain why each misleads. RED FLAG: listing chart types without the perceptual mechanism.
Reconcile rising sign-ups with falling revenue per user
WHAT IT TESTS: business judgment and data storytelling. OUTLINE: reconcile the metrics via total revenue and segment mix, frame the tradeoff, recommend an action. RED FLAG: declaring one team wrong instead of synthesizing both.
Visualizing long-term trend versus seasonality
WHAT IT TESTS: matching visualization to the analytical question. OUTLINE: a line chart over the full three years, often with a moving average, shows the long-term trend; a seasonal plot overlaying each year by month, or a month-of-year box plot, reveals…