Intermediate everything in AI & ML, page 20
Online vs. Offline Feature Serving: Two Speeds for ML Data
Offline serving provides large batches of historical data for model training; online serving provides low-latency features for live predictions. This dual system in a feature store prevents training-serving skew, ensuring model consistency from lab to…
Feature Hashing: The Hashing Trick for ML
Feature hashing turns features into vector indices without a lookup table, trading perfect accuracy for speed and memory. It's used for high-cardinality data like user IDs or in online systems.
Data Augmentation: Getting More from Your Data
Data augmentation creates 'new' training data by making small, realistic changes to your existing data. It's used to fight overfitting in ML models when a dataset is small, teaching the model to generalize rather than memorize.
DVC: Git for Data and ML Models
DVC extends Git to version large data files and models without bloating your repo. It stores small pointer files in Git that reference large files in cloud storage.
Great Expectations: Unit Tests for Your Data
Great Expectations brings unit testing to your data, letting you assert what a dataset should look like. It validates data within a pipeline, preventing bad data from corrupting models or reports.
Data Schema Evolution: Changing Your Data's Blueprint
Schema evolution is like updating a building's blueprint while it's occupied. You must change your data's structure without breaking apps or losing data. It's key for adding features that need new DB columns.
ELT: Load Raw Data, Transform in Place
ELT flips the data pipeline: load raw data first, then use the data warehouse's own power to transform it. It's used in ML feature pipelines. The footgun is assuming it's ETL; with ELT, the transformation logic is coupled to the warehouse's SQL engine.
CD4ML: Automating ML from Data to Deployment
CD4ML extends CI/CD to manage ML's three axes of change: code, data, and models. It automates the entire lifecycle, enabling reliable updates for systems like sales forecasting.
Data Versioning: Git for Your Datasets
Think of data versioning as Git for datasets. It tracks changes to your data, allowing you to reproduce ML experiments or roll back to a previous state. The footgun is using regular Git, which chokes on the large binary files common in ML.
ML Experiment Tracking: Your Model's Lab Notebook
Think of it as a lab notebook for your models, logging every parameter and result. It's essential when tuning hyperparameters or comparing architectures, preventing you from losing track of what worked.
Open LLM Leaderboard: Separating Hype from Performance
The Open LLM Leaderboard is the de facto scoreboard for open-source models, providing reproducible benchmarks to cut through marketing hype. It helps you compare models on standardized tests, but remember that a high rank doesn't guarantee performance on your…
AI Coding Assistants: Your LLM Pair Programmer
AI coding assistants are like pair programmers powered by large language models. They assist in tasks across the software lifecycle, from code generation and testing to debugging and documentation. The key footgun is over-reliance; they assist, not replace.
AI's Dual-Use Problem: Good Tools, Bad Outcomes
AI models built for good can be easily repurposed for harm. A language model that helps with coding can also generate malware. The footgun is assuming good intentions prevent misuse; the risk is in the capability, not the creator's intent.
Quantization-Aware Training (QAT): Forcing Models to Learn While Quantized
QAT forces a model to "learn its own compression" by simulating quantization during training. This lets you shrink LLMs to aggressive low-bit formats (like 4-bit) where simpler post-training methods fail.
Dynamic Batching: Balancing LLM Throughput and Latency
Dynamic batching groups LLM requests like a bus that leaves on a schedule or when full, whichever comes first. This improves throughput in inference servers by avoiding long waits. The footgun: all requests in a batch are still held hostage by the slowest one.
Model Pruning: Making LLMs Smaller, Not Dumber
Model pruning is surgical weight loss for an LLM, removing neurons or layers to reduce its size. It's used to create smaller, faster versions of models like LLaMA for efficient deployment. The footgun: naive pruning can cripple the model's core capabilities.
Modality Gap: When Multimodal LLMs Don't Trust Their Senses
A multimodal LLM has a modality gap when it trusts one input type (like text) over another (like images), even with identical information. This bias causes performance drops, like ignoring visual data if conflicting text is present.
Full Fine-Tuning: Updating Every Model Parameter
Full fine-tuning updates all weights of a pre-trained model on your new data, unlike methods that only change a small fraction. Use it to deeply embed new knowledge, but beware: it's costly and risks making the model forget its original general skills.
LLMs as Tool Makers: Write Once, Solve Many
LLMs can create their own tools, not just use them. A powerful model writes a reusable function once, and a cheaper model calls it many times. This gives top-tier results at a lower cost for repetitive tasks.
Reflection: Teaching LLM Agents to Learn from Mistakes
Reflection gives an agent an "inner monologue" to learn from its mistakes. An Actor model attempts a task, an Evaluator scores it, and a Self-Reflection model generates linguistic feedback for the next try.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles