Concepts in AI & ML, page 29
Image Thresholding: Separating Foreground from Background
Image thresholding turns a grayscale image into black and white by setting a brightness cutoff. It's used to isolate features for analysis, like finding text on a page.
Perspective Transformation: Making 3D Look Right on a 2D Screen
Perspective transformation is like looking through a camera lens; it makes distant objects smaller to create the illusion of depth on a flat screen. It's essential for 3D games and computer vision.
BRIEF: Fast, Compact Binary Feature Descriptors
BRIEF describes image features as a compact binary string instead of a complex vector. This makes it extremely fast for real-time matching on low-power devices. The footgun: its raw form isn't rotation-invariant, trading that robustness for raw speed.
Sliding Windows: Scanning Images for Objects
A sliding window scans an image with a fixed-size box to find objects. At each location, a classifier checks the window's contents. Combined with an image pyramid, it can detect objects at various scales, localizing exactly where they are.
Anchor Boxes: Pre-defined Guesses for Object Detection
Anchor boxes are predefined 'template' boxes of various sizes and shapes. Object detection models use them as a starting point, predicting how to shift and scale these templates to fit actual objects, making detection faster.
Watershed Algorithm: Segmenting by Topography
The watershed algorithm treats an image as a landscape where pixel brightness is height. It finds the 'ridges' separating distinct 'valleys,' effectively segmenting touching objects. Its main footgun is extreme sensitivity to noise, causing over-segmentation.
U-Net: Encoder-Decoder for Image Segmentation
U-Net segments images by first compressing them to capture context, then expanding to localize features precisely. It excels in biomedical imaging where annotated data is scarce.
Self-Attention: How Models Weigh Word Importance
Self-attention lets a model weigh the importance of all words in a sequence simultaneously, asking "which other words are most relevant?" It's the core of Transformers, enabling parallel processing for tasks like translation, unlike sequential RNNs.
Autoregressive Models: Generating Images One 'Word' at a Time
Autoregressive models generate images sequentially, like writing a sentence word by word. Each new pixel or patch depends on what came before. This creates high-fidelity images but is much slower than one-shot models, a key trade-off in generative AI.
Semantic Scene Classification: Understanding Context, Not Just Objects
Scene classification tells you the context of an image ("this is a forest"), not just the objects in it ("there's a tree"). It's used by self-driving cars to identify a highway vs. a residential street and by apps to organize photos.
3D Object Detection: Seeing in Depth, Not Just Pixels
3D object detection adds depth to a 2D flat view, understanding an object's true size, distance, and orientation. It's vital for autonomous cars and robotics that need spatial awareness.
Meta-Learning: Learning How to Learn
Meta-learning is 'learning to learn.' Instead of training on data, it learns from the performance of other models, using metadata from experiments to improve the learning process itself. This helps algorithms become more flexible and solve new problems faster.
Neural Network Pruning: Making Models Smaller and Faster
Neural network pruning makes models smaller and faster by removing unimportant connections, like trimming a bonsai tree. It's essential for deploying large models on devices with limited memory, like phones.
Image Convolution: A Sliding Feature Detector
An image convolution is a sliding filter that scans an image to detect features like edges or textures. It's the core building block of modern computer vision, used in image classification and object detection.
Non-Maximum Suppression: One Box Per Object
Non-Maximum Suppression (NMS) ensures each detected object gets just one bounding box. It sorts all proposed boxes by confidence, keeps the best one, and discards others that overlap it too much.
COCO: The Messy Real-World Vision Benchmark
COCO is the standard benchmark for detecting overlapping objects in cluttered scenes. Use it to test object detectors and segmentation. Strong scores here do not mean your model works on specialized domains like medical or satellite imagery.
RoPE: Encoding Position with Rotation
Rotary Position Embedding (RoPE) encodes position by rotating token embeddings, where the angle depends on the token's absolute spot in the sequence. This is used in Transformers like Llama to handle long contexts, as the attention score naturally becomes a function of relative distance. The main footgun is assuming standard position embeddings extrapolate; RoPE is designed for sequence length flexibility, unlike many absolute position encodings which fail on longer inputs.
Instruction Tuning: Teaching Models to Follow Orders
Instruction tuning teaches a language model to generalize by finetuning it on a massive collection of tasks described in plain English. This transforms a raw pretrained model, which just predicts the next word, into one that can follow commands on unseen tasks without any examples (zero-shot). The footgun is mistaking this for simple finetuning on one task; its power comes from the sheer diversity of instructional tasks used during training.
Speculative Decoding: Faster LLM Inference, Same Results
Speculative decoding accelerates LLM inference by using a small, fast "draft" model to predict a sequence of tokens. The large, accurate model then validates this entire sequence in a single parallel pass, instead of generating one token at a time. This is used to get 2-3x speedups on production models without retraining. The common misconception is that it's a lossy approximation; in reality, it produces bit-for-bit identical output to the original model.
Constitutional AI: Teaching an AI Right from Wrong
Constitutional AI teaches a model to be harmless by making it follow a set of principles—a constitution—instead of relying on human-labeled examples of bad behavior. This self-correction process, called Reinforcement Learning from AI Feedback (RLAIF), is used to align powerful models, enabling them to refuse harmful requests while explaining their reasoning. The entire system's safety, however, hinges on the quality and completeness of the initial human-written constitution.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles