Concepts in AI & ML, page 5
AdamW: Decoupling Weight Decay for Better Generalization
AdamW fixes a flaw in the Adam optimizer by decoupling weight decay from the gradient update, improving model generalization. It's a go-to for training large networks like Transformers. The footgun is thinking it's the same as Adam with L2 regularization.
Chain Rule: Unpacking Nested Rates of Change
The chain rule is like Russian nesting dolls for rates of change. To find the derivative of a nested function, you multiply the derivatives of the 'outer' and 'inner' functions. It's the engine behind backpropagation in neural networks.
Morphological Transformations: Shaping Images with Kernels
Think of it as sanding or stamping an image with a small shape (kernel) to modify object boundaries. It's used for noise removal or separating touching objects. The footgun: transformations alter object size, so you often need a second operation to compensate.
MLE: Find the Parameters That Make Data Likely
MLE tunes your model until observed data looks inevitable. Use it to fit distributions to logs, traffic, or errors. The footgun: it assumes your distribution family is correct; under a wrong model, it finds the best-fitting wrong answer with high confidence.
Data Lake: A Single Repository for Raw Data
A data lake is a central repository that stores vast amounts of raw data in its native format. It acts as a single source for analytics and machine learning, but without proper management it can become a useless "data swamp".
Learning Rate Scheduling: A Gearbox for Model Training
Think of a learning rate schedule as a training 'gearbox,' starting fast and slowing for precision. It's used when fine-tuning large models to adapt them without breaking them.
Canny Edge Detector: Finding True Edges
The Canny detector finds true edges by identifying sharp, connected intensity changes. It's used in object detection and feature extraction where clean edge maps are vital.
SVD: Eigendecomposition for Any Matrix
SVD treats any matrix as rotation, then scaling, then rotation. It generalizes eigendecomposition beyond square normal matrices to any real or complex matrix.
Feature Hashing: The Hashing Trick for ML
Feature hashing turns features into vector indices without a lookup table, trading perfect accuracy for speed and memory. It's used for high-cardinality data like user IDs or in online systems.
Data Parallelism: One Task, Many Data Chunks
Data parallelism splits a huge dataset across multiple processors, each running the same task on its own chunk. It's how large models are trained on massive datasets, with each GPU handling a different batch of data.
Affine Transformations: Move, Scale, and Rotate Without Bending
An affine transformation is a way to move, scale, rotate, or shear an object while keeping all lines straight and parallel lines parallel. It's used everywhere in computer graphics.
Online vs. Offline Feature Serving: Two Speeds for ML Data
Offline serving provides large batches of historical data for model training; online serving provides low-latency features for live predictions. This dual system in a feature store prevents training-serving skew, ensuring model consistency from lab to…
What is a Large Language Model (LLM)?
An LLM is a neural network trained on vast text data to process and generate human-like language. It's the core technology behind chatbots, enabling them to generate, summarize, and translate text. Its reliability depends entirely on its training data.
The Jacobian Matrix: A Derivative for Multiple Dimensions
The Jacobian matrix is the multi-dimensional version of a derivative. It's a grid of partial derivatives showing how a small change in each input locally affects each output of a vector function. Don't confuse the matrix with its determinant.
Laplacian of Gaussian (LoG) for Blob Detection
LoG finds blobs by blurring an image then finding points of maximum curvature, like finding the top of a hill by looking where the slope changes fastest. It's used in medical imaging and astronomy. The footgun: the blur size dictates the blob size you can.
LLM Scaling Laws: Match Parameters to Tokens
For a fixed compute budget, model size and training data must grow equally. When choosing between a larger model or more tokens for the same FLOPs, more data usually wins. The footgun is scaling parameters alone, which undertrains the model and wastes compute.
Feature Backfilling: Populating Historical Data for ML
Feature backfilling computes a new feature's values for historical data. It's how you generate a complete training dataset after defining a new signal, like a user's 7-day purchase history. The footgun is using future data, causing data leakage.
Harris Corner Detector: Finding Sharp Changes in Images
The Harris detector finds corners by looking for points where image intensity changes sharply in all directions. It's used in image stitching and object tracking to find stable feature points. A key weakness is its sensitivity to image scale.
Recursive Feature Elimination: Survival of the Fittest Features
RFE runs a tournament for your features, repeatedly training a model and dropping the weakest ones. It's used to simplify models by selecting a core subset of impactful features. The main footgun: RFE's output is only as good as the model used for ranking.
Pipeline Parallelism: An Assembly Line for Your Model
Think of training a huge model like an assembly line. Pipeline parallelism splits a model's layers into stages across multiple GPUs, allowing you to train models too large for one device.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles