Advanced concepts in AI & ML, page 8
Visual Commonsense Reasoning (VCR): From Recognition to Cognition
VCR pushes AI from simple object recognition to human-like reasoning by asking not just 'what' is in an image, but 'why.' Models must select both the correct answer and the correct rationale, exposing models that guess answers based on shallow correlations.
Slash Your Cloud Bill by Taming Data Egress Costs
Data egress is the tax for moving data out of a cloud provider's network, a common cost in MLOps when moving models or datasets. To save money, keep compute and data in the same region. The footgun is forgetting that traffic between regions also counts.
Differential Privacy: Anonymize Data with Math
Differential Privacy adds mathematical noise to data queries, making it impossible to know if one person's data is included. Tech giants use it to learn from user behavior without seeing individual activity.
Federated Learning: Train Models on Decentralized Data
Federated learning trains a shared model by sending the model to the data, not the other way around. It's used for training on sensitive, decentralized data like phone keyboards. The main footgun is that non-uniform data across clients can skew the model.
Counterfactual Fairness: What if You Were Different?
Asks "what if?" for fairness: would your model's decision change if only a sensitive attribute like race were different? It's used to audit models for hidden bias in areas like hiring.
SimCLR: Learning Powerful Vision Features Without Labels
SimCLR learns image features from unlabeled data by teaching a model that two augmentations of one image are similar, and all other images are different. It's used to pre-train models on vast, unlabeled datasets.
Homomorphic Encryption: Compute Without Decrypting
Homomorphic encryption lets you perform computations on data while it's still encrypted. This allows a third party, like a cloud provider, to process your sensitive data without ever seeing the raw information, ensuring privacy.
Momentum Contrast (MoCo): A Dynamic Dictionary for Unsupervised Learning
MoCo learns visual features without labels by treating contrastive learning as a dynamic dictionary lookup. A momentum-updated encoder creates a large, consistent set of keys on-the-fly, enabling powerful pre-training on unlabeled data for downstream vision…
Masked Autoencoders: Learning Vision by Filling in the Blanks
Masked Autoencoders (MAEs) teach models vision by playing "fill-in-the-blanks" with images, masking most of an image (e.g., 75%) and learning to reconstruct it. This is used for self-supervised pre-training of large Vision Transformers on unlabeled data.
MAML: Learning to Learn Quickly
MAML trains a model to be easy to fine-tune, finding an initial set of parameters that serve as a great starting point for many new tasks. It's used in few-shot learning where models must adapt with minimal data.
Seam Carving: Resizing Images Without Distortion
Seam carving resizes images by removing or adding pixel "seams" of low importance, not by squashing or cropping the whole frame. It's used in content-aware tools to change aspect ratios without distorting key subjects like faces or buildings.
Photometric Stereo: Reconstructing Shape from Light
Photometric stereo deduces an object's 3D shape by watching how its surface reflects light from different directions. It’s used in industrial inspection and computer graphics to capture detailed surface geometry. The footgun is assuming it works on any object.
Light Field Photography: Capturing Light's Direction
A light field camera captures not just what light hits the sensor, but where it came from. It records both the intensity and direction of every ray, unlike conventional cameras that only see intensity. The footgun is thinking it's just a better 2D camera.
Coded Aperture: Imaging Without a Lens
A coded aperture images radiation that can't be focused, like X-rays. Instead of a lens, it uses a patterned mask to cast a complex shadow, which is then mathematically decoded into an image. It's crucial for X-ray astronomy.
Knowledge Distillation: Shrinking Models, Not Performance
Knowledge Distillation trains a small "student" model on the nuanced outputs of a large "teacher" model. This is how huge, accurate models are shrunk to run on phones. The footgun is assuming performance is identical; there's always a trade-off.
TensorRT: From Trained Model to Production Speed
TensorRT is a compiler that turns a trained model into a specialized, high-speed engine for a specific NVIDIA GPU. It's used to deploy models in production where low latency is critical.
FPGA in Computer Vision
An FPGA is reconfigurable silicon wired into a custom digital circuit rather than programmed as instructions, letting a vision pipeline like demosaicing and feature extraction run in dedicated hardware with low latency and high throughput per watt.
RoPE: Encoding Position with Rotation
Rotary Position Embedding (RoPE) encodes position by rotating token embeddings, where the angle depends on the token's absolute spot in the sequence. This is used in Transformers like Llama to handle long contexts, as the attention score naturally becomes a function of relative distance. The main footgun is assuming standard position embeddings extrapolate; RoPE is designed for sequence length flexibility, unlike many absolute position encodings which fail on longer inputs.
Instruction Tuning: Teaching Models to Follow Orders
Instruction tuning teaches a language model to generalize by finetuning it on a massive collection of tasks described in plain English. This transforms a raw pretrained model, which just predicts the next word, into one that can follow commands on unseen tasks without any examples (zero-shot). The footgun is mistaking this for simple finetuning on one task; its power comes from the sheer diversity of instructional tasks used during training.
Speculative Decoding: Faster LLM Inference, Same Results
Speculative decoding accelerates LLM inference by using a small, fast "draft" model to predict a sequence of tokens. The large, accurate model then validates this entire sequence in a single parallel pass, instead of generating one token at a time. This is used to get 2-3x speedups on production models without retraining. The common misconception is that it's a lossy approximation; in reality, it produces bit-for-bit identical output to the original model.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles