More in AI & ML — page 8
What a feature store solves: skew and consistency
WHAT IT TESTS: feature-store value. OUTLINE: central repository of computed features, one definition serving training and inference, reuse across models. RED FLAG: calling it just a database or ignoring the offline-online consistency guarantee.
Detecting and responding to model and concept drift
WHAT IT TESTS: production monitoring. OUTLINE: define drift, pick a metric like PSI or falling AUC against labels, then investigate, retrain, validate. RED FLAG: assuming static accuracy or retraining blindly without diagnosing the cause.
Batch prediction versus online real-time prediction
WHAT IT TESTS: serving pattern selection. OUTLINE: batch is scheduled bulk scoring, online is low-latency per-request scoring; contrast latency, freshness, cost; give a use case each. RED FLAG: confusing batch with retraining or claiming online is always best.
Deploy a saved model as a REST prediction service
WHAT IT TESTS: end-to-end deployment basics. OUTLINE: load the artifact, wrap it in a predict API, containerize, host with autoscaling, add monitoring. RED FLAG: jumping to model training or forgetting preprocessing parity and input validation.
Explain k-means user segments to a marketing team
WHAT IT TESTS: turning clustering output into actionable personas. OUTLINE: name each segment, profile its defining traits, show size and value, recommend an action. RED FLAG: explaining centroids and inertia instead of who the segments are.
Two ways accurate data can still mislead in a chart
WHAT IT TESTS: visualization honesty. OUTLINE: name distortions like truncated axes or cherry-picked ranges, give the fix for each, explain why each misleads. RED FLAG: listing chart types without the perceptual mechanism.
Reconcile rising sign-ups with falling revenue per user
WHAT IT TESTS: business judgment and data storytelling. OUTLINE: reconcile the metrics via total revenue and segment mix, frame the tradeoff, recommend an action. RED FLAG: declaring one team wrong instead of synthesizing both.
Visualizing long-term trend versus seasonality
WHAT IT TESTS: matching visualization to the analytical question. OUTLINE: a line chart over the full three years, often with a moving average, shows the long-term trend; a seasonal plot overlaying each year by month, or a month-of-year box plot, reveals…
Diagnosing Spark executor OutOfMemoryError
WHAT IT TESTS: systematic Spark memory debugging. OUTLINE: check the Spark UI for skew and spills, inspect executor memory and partition count, find culprits like wide collect, huge shuffles, or skewed keys, and fix via more partitions, memory tuning, or…
Spark RDDs, DataFrames, and Datasets
WHAT IT TESTS: knowledge of Spark's APIs and the optimizer. OUTLINE: RDDs are low-level typed object collections with no built-in optimization; DataFrames are named columns optimized by Catalyst and Tungsten; Datasets add compile-time type safety in…
Data skew in Spark and salting
WHAT IT TESTS: diagnosing and fixing skewed distributed work. OUTLINE: data skew is uneven key distribution sending most rows to one partition and straggler task; salting appends a random suffix to hot keys to spread them across partitions, joining in two…
Spark broadcast join versus shuffle join
WHAT IT TESTS: Spark join optimization. OUTLINE: a broadcast join sends the small table to every executor so the large table joins locally with no shuffle of its rows; the default sort-merge join shuffles both tables across the network, which is costly.
The MapReduce paradigm explained
WHAT IT TESTS: distributed batch processing basics. OUTLINE: map applies a function to each input record emitting key-value pairs in parallel; a shuffle groups values by key; reduce aggregates each key's values into a result.
HDFS purpose and fault tolerance
WHAT IT TESTS: distributed storage fundamentals. OUTLINE: HDFS stores huge files across many commodity machines as large blocks, replicating each block across nodes for fault tolerance; unlike NTFS or ext4 it is distributed, write-once, and optimized for…
Spark transformations versus actions
WHAT IT TESTS: Spark's lazy execution model. OUTLINE: transformations like map and filter are lazy and build a lineage DAG returning a new RDD; actions like count or collect trigger execution and return a value to the driver.
Mode collapse in GANs and how to fix it
WHAT IT TESTS: understanding GAN training failures. OUTLINE: mode collapse is the generator producing few similar outputs, missing data diversity to fool the discriminator; fixes include Wasserstein loss, minibatch discrimination, unrolled GANs, and feature…
Why Transformers use multi-head attention
WHAT IT TESTS: understanding of attention design. OUTLINE: a single head averages into one representation subspace; multiple heads attend in parallel to different subspaces, letting the model capture diverse relations like syntax and coreference at once, then…
Exploration versus exploitation: epsilon-greedy and UCB
WHAT IT TESTS: balancing trying new actions against using known good ones. OUTLINE: exploit current best to earn reward, explore to discover better options; epsilon-greedy explores randomly with probability epsilon; UCB explores by an uncertainty bonus…
RL components and how Q-learning works
WHAT IT TESTS: foundational RL vocabulary and the Q-learning update. OUTLINE: agent acts on the environment, observes state and reward, seeking to maximize cumulative discounted reward; Q-learning iteratively updates Q(s,a) toward reward plus discounted best…
Analyzing skewed revenue-per-user experiments
WHAT IT TESTS: handling skewed metrics in A/B tests. OUTLINE: heavy tails inflate variance and slow significance, and the mean is dominated by whales; mitigate via winsorization or capping, log transforms, CUPED variance reduction, or bootstrap and rank tests.