Concepts in AI & ML, page 16
Action Recognition: Teaching Machines to Understand 'Doing'
Action recognition teaches computers to understand *what* is happening in a video by analyzing motion over time. It's used for fall detection in healthcare, sports analysis, and security. The footgun is confusing it with single-frame object detection.
MMLU Benchmark
MMLU (Measuring Massive Multitask Language Understanding) is a popular benchmark for evaluating large language models. Its influence is shown by its many spin-offs, making it a foundational tool for comparing AI capabilities.
A/A Testing: Sanity-Checking Your Experiments
A/A testing is a fire drill for your A/B testing system. You run two identical versions of a page to ensure your tools are working correctly before a real experiment. The biggest footgun is panicking at a false positive; 1 in 20 tests will show.
Lucas-Kanade Method: Tracking Pixel Patches, Not Points
The Lucas-Kanade method tracks motion by assuming a small patch of pixels moves as a single unit. This overcomes the ambiguity of tracking one pixel. It's fundamental to video stabilization and feature tracking.
Inference Throughput: How Many Predictions Per Second?
Inference throughput measures how many predictions your system can make per second, not how fast a single one is. It's the system's total capacity, critical for high-volume tasks like recommendation engines. The footgun is confusing it with latency.
Why Elo Ratings for LLMs Can Be Misleading
Elo ranks LLMs like chess players, but models have fixed skills, not dynamic ones. This method powers leaderboards but produces volatile scores, meaning a model's rank can be an unstable estimate of its true, unchanging ability.
Statistical Power: Is Your Test Strong Enough to Find a Real Effect?
Statistical power is your experiment's ability to detect a real effect. A low-power test is like a fishing net with holes too big—the fish swims right through. This is critical for A/B tests. The footgun is concluding "no effect" from a weak test.
Horn-Schunck: Assuming Smooth Motion to See Clearly
The Horn-Schunck method estimates video motion by assuming neighboring pixels move similarly. This global 'smoothness' constraint solves the aperture problem, where local views give ambiguous motion data.
LLM-as-a-Judge: Using Models to Grade Models
Instead of paying humans to rate AI outputs, LLM-as-a-Judge uses a powerful 'judge' model to do it automatically. This is used to evaluate chatbot responses or summarization quality, but the main footgun is assuming the judge model is unbiased or perfectly…
SUTVA: The Assumption That Your Treatment Isn't Leaking
SUTVA assumes your treatment on one person doesn't spill over to affect others, and that the treatment is consistent for all. It's a key assumption for A/B tests, but is violated when one person's vaccine protects their unvaccinated neighbor.
Kalman Filters: Predicting Through Noise
A Kalman filter predicts an object's next position, then corrects that prediction with noisy real-world measurements. It's used in video tracking to smooth an object's path between frames or in robotics to fuse imperfect sensor data.
ML Monitoring Dashboards: Your Model's Health Chart
An ML monitoring dashboard is a health chart for your production model, showing how its performance decays. It tracks silent failures like data drift or concept drift, where user behavior changes and makes your model obsolete.
HumanEval: Testing if AI-Generated Code Actually Works
HumanEval is a benchmark that tests if an LLM's generated code is functionally correct, not just syntactically valid. It's used to compare models like Codex by having them solve programming puzzles.
Propensity Score Matching: Simulating a Randomized Trial
Propensity Score Matching creates a 'fair' comparison from observational data, mimicking a randomized trial. It's used to estimate a treatment's effect by matching treated individuals with similar untreated ones.
Mean Shift: A Mode-Seeking Algorithm
Mean shift is a technique for finding the peaks, or modes, of a density function. It's a non-parametric method used for cluster analysis in domains like computer vision and image processing to locate the densest areas in a feature space.
Population Stability Index (PSI): Quantifying Data Drift
The Population Stability Index (PSI) gives you a single number to quantify data drift between training and live data. It's used in MLOps to monitor model health, especially in finance. The footgun is ignoring a high PSI, which signals silent prediction decay.
The AI Alignment Problem
AI alignment is about making sure an AI pursues our intended goals, not just the literal instructions. It's critical for autonomous systems in medicine or finance. The footgun is assuming a clear objective prevents unintended, harmful outcomes.
Causal DAGs: A Map for Cause and Effect
A Causal DAG is a map of your assumptions about what causes what. It helps you spot hidden "confounder" variables that create misleading correlations. Use it before an analysis to decide which variables to control for, ensuring you measure a true effect.
Model Cards: The 'Nutrition Label' for AI Models
A model card is the nutrition label for an AI model, summarizing its ingredients, intended use, and risks. Found in model repos, it details training data, performance, and ethical guardrails.
Artificial Neural Networks: Learning from Examples
Think of an ANN as a digital brain that learns from examples, not explicit code. It's a network of simple nodes that adjust their connections to spot patterns. They power image recognition and language translation, but are only as good as their training data.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles