Intermediate concepts in AI & ML, page 15
Watershed Algorithm: Segmenting by Topography
The watershed algorithm treats an image as a landscape where pixel brightness is height. It finds the 'ridges' separating distinct 'valleys,' effectively segmenting touching objects. Its main footgun is extreme sensitivity to noise, causing over-segmentation.
U-Net: Encoder-Decoder for Image Segmentation
U-Net segments images by first compressing them to capture context, then expanding to localize features precisely. It excels in biomedical imaging where annotated data is scarce.
Self-Attention: How Models Weigh Word Importance
Self-attention lets a model weigh the importance of all words in a sequence simultaneously, asking "which other words are most relevant?" It's the core of Transformers, enabling parallel processing for tasks like translation, unlike sequential RNNs.
Autoregressive Models: Generating Images One 'Word' at a Time
Autoregressive models generate images sequentially, like writing a sentence word by word. Each new pixel or patch depends on what came before. This creates high-fidelity images but is much slower than one-shot models, a key trade-off in generative AI.
Semantic Scene Classification: Understanding Context, Not Just Objects
Scene classification tells you the context of an image ("this is a forest"), not just the objects in it ("there's a tree"). It's used by self-driving cars to identify a highway vs. a residential street and by apps to organize photos.
3D Object Detection: Seeing in Depth, Not Just Pixels
3D object detection adds depth to a 2D flat view, understanding an object's true size, distance, and orientation. It's vital for autonomous cars and robotics that need spatial awareness.
Meta-Learning: Learning How to Learn
Meta-learning is 'learning to learn.' Instead of training on data, it learns from the performance of other models, using metadata from experiments to improve the learning process itself. This helps algorithms become more flexible and solve new problems faster.
Neural Network Pruning: Making Models Smaller and Faster
Neural network pruning makes models smaller and faster by removing unimportant connections, like trimming a bonsai tree. It's essential for deploying large models on devices with limited memory, like phones.
Image Convolution: A Sliding Feature Detector
An image convolution is a sliding filter that scans an image to detect features like edges or textures. It's the core building block of modern computer vision, used in image classification and object detection.
Non-Maximum Suppression: One Box Per Object
Non-Maximum Suppression (NMS) ensures each detected object gets just one bounding box. It sorts all proposed boxes by confidence, keeps the best one, and discards others that overlap it too much.
COCO: The Messy Real-World Vision Benchmark
COCO is the standard benchmark for detecting overlapping objects in cluttered scenes. Use it to test object detectors and segmentation. Strong scores here do not mean your model works on specialized domains like medical or satellite imagery.
VAEs: Generating New Data by Learning Its Essence
A Variational Autoencoder (VAE) learns the *essence* of data, not just how to copy it. Instead of compressing an input to a single point, it maps it to a fuzzy region in a "concept space," allowing you to generate new, similar data by sampling from that region. This is key for creating novel images or music. The footgun is expecting sharp outputs; VAEs often produce blurrier results than models like GANs.
Tool Use: Giving LLMs Access to External Systems
Tool use lets an LLM call external functions, like a brain accessing a calculator or the internet. This is the core mechanism behind AI agents that can search the web, run code, or query a database to answer questions. The biggest footgun is assuming the model will always generate a valid function call; without enforcing a strict schema to match your function's expected input, your agent can fail unpredictably.
RAG: Giving Language Models an Open-Book Exam
Retrieval-Augmented Generation (RAG) gives a language model an open-book exam instead of forcing it to memorize everything. It combines a model's reasoning ability with a searchable external knowledge base. This grounds LLM responses in specific, up-to-date information, like a support bot using a product manual. The footgun is forgetting that the quality of the retrieved information directly limits the quality of the final answer.
Chain-of-Thought: Making LLMs 'Show Their Work'
Chain-of-thought prompting makes an LLM 'show its work' by generating intermediate reasoning steps before the final answer. This simple few-shot technique dramatically improves performance on complex tasks like math word problems or commonsense questions, especially for very large models. The common footgun is applying it to smaller models, where it can actually degrade performance instead of helping, as the reasoning ability hasn't yet emerged.
Generative Adversarial Networks (GANs): An AI Arms Race
Think of a GAN as an AI arms race between two networks: a forger and a detective. The forger network (Generator) creates fake data, like images or audio, while the detective network (Discriminator) tries to spot the fakes. This competition forces the forger to create increasingly realistic outputs. The main footgun is training instability—if one network overpowers the other too early, the whole system fails to learn and produces garbage.
Diffusion Models: Generating Data by Reversing Noise
Think of diffusion models as learning to reverse a "random walk." They take a clean data point, gradually add noise until it's unrecognizable, and then train a model to reverse that process step-by-step. This allows them to start with pure noise and guide it back into a coherent sample that resembles the original dataset. The footgun is that this multi-step reversal makes generation computationally intensive compared to single-pass models.
Perplexity: Measuring a Model's Uncertainty
Perplexity frames a model's uncertainty as the effective number of choices it's considering. For a fair die with six outcomes, the perplexity is 6, reflecting perfect confusion among six options. When evaluating language models, a lower perplexity score indicates a better ability to predict a sequence of text. The footgun is judging the score in a vacuum; a 'good' perplexity is always relative to the task's inherent randomness.
BLEU Score: Judging Translation by Human Overlap
The BLEU score judges a machine translation by how closely its text matches a professional human translation. It's a popular, automated, and inexpensive way to benchmark translation systems, like comparing different versions of a model. The main footgun is that a high score indicates high textual overlap, not necessarily better fluency or meaning, as it's just a proxy for human judgment.
RNNs: Neural Networks with Short-Term Memory
A Recurrent Neural Network (RNN) processes sequences by keeping a running memory of what it's seen. It feeds its own output from one step back into the next, like someone reading a sentence one word at a time. This is ideal for sequential data like text or time series where context is key. The main footgun is its notoriously short memory; information from early in a long sequence often gets lost.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles