Concepts in AI & ML, page 11
Feature Engineering: Better Inputs, Better Models
Feature engineering preps raw data for a model, like a chef preps ingredients. It transforms raw inputs into a more effective set of predictive signals. The footgun is creating irrelevant features, which can harm model performance more than using raw data.
Backpropagation: How Neural Networks Learn from Mistakes
Backpropagation is how a network learns from its mistakes. It works backward from the output error, calculating how much each weight contributed and adjusting it. This is the core training loop for most deep learning models.
All-Reduce: Synchronizing Parallel Workers
All-Reduce lets parallel workers agree on a global result. Each worker contributes data, an operation (like sum) runs on all data, and every worker gets the final answer. It's the core of distributed ML training, used to average gradients across GPUs.
Self-Querying Retriever: Let an LLM Write Its Own Filters
A self-querying retriever uses an LLM to turn a natural language question into a structured query with metadata filters. It lets users ask things like "Find documents about Python from before 2020," which a simple vector search can't do.
Box-Cox Transformation: Forcing Skewed Data to Look Normal
The Box-Cox transformation is a statistical lens that reshapes skewed data to better resemble a normal distribution. It helps meet the assumptions of models like linear regression, but it only works on positive data and complicates direct interpretation of…
Data Augmentation: Getting More Images for Free
Data augmentation creates "fake" training data by modifying existing images—flipping, rotating, or color-shifting them. This fights overfitting when your dataset is small, forcing the model to generalize.
Microsoft DeepSpeed: Training Massive Models Across GPUs
DeepSpeed trains models too big for one GPU by partitioning model states across many devices. It's essential for training foundation models like BLOOM, but its complexity is overkill for smaller models and misconfiguration can harm performance.
HyDE: Find Documents by Embedding a Fake Answer
Instead of embedding a short query, HyDE uses an LLM to generate a full, hypothetical answer document. This vector, representing an ideal answer, is then used to find similar real documents, improving zero-shot retrieval. The generated document is fictional.
Target Encoding: Replacing Categories with Target Stats
Target encoding replaces a category (e.g., "USA") with a statistic from your target variable (e.g., average sales). It's ideal for high-cardinality features where one-hot encoding is impractical. The footgun is data leakage, which causes severe overfitting.
Dropout: Forcing a Network to Generalize
Dropout prevents overfitting by randomly zeroing out a fraction of neurons during training. This forces the network to learn more robust features instead of relying on specific neurons. It's a standard regularizer for large, dense layers.
Git-Based CI Triggers: Automating on Events
Think of Git events like push or pull_request as the "play" button for your automation. This is how CI systems automatically run tests on new code. The footgun is using broad triggers, like push on all branches, which causes costly and redundant runs.
Graph RAG: Answering Questions with Connected Facts
Graph RAG answers complex questions by exploring a map of connected facts (a knowledge graph) instead of just searching flat text. Use it for queries needing synthesis, like finding drugs for a disease made by companies in a specific country.
Feature Selection: Making Models Better With Less Data
Feature selection improves models by giving them less data, finding signal by removing noise. Use it to speed up training, simplify models for easier interpretation, and avoid performance degradation from having too many input features.
AlexNet: The CNN That Sparked the Deep Learning Boom
AlexNet is the blueprint that proved deep CNNs could master image recognition, kicking off the modern AI boom. Its architecture is foundational for modern computer vision. The footgun is thinking it was just bigger; its novelty was combining new techniques.
Unit Testing ML: Beyond Standard Code Checks
Unit testing for ML isn't just about code logic; it's about checking data, models, and infrastructure in isolation. Use it to validate data transformers, check model prediction shapes, or confirm a function handles nulls.
LLM Agents: Giving Models Tools and a Plan
An LLM agent can choose a tool, inspect its result, and decide what to do next. Retrieval supplies information; an agent may use it while carrying out a task. A separate planning or memory module is not mandatory.
Data Warehouse: The Central Repository for Analytics
A data warehouse is a central repository for historical analysis, integrating data from many systems. It's used for reporting and complex queries to find business insights, not for day-to-day transactions.
Batch Normalization: Stabilizing Neural Network Training
Batch Normalization regulates data flow in a neural network by re-centering and re-scaling inputs to each layer. This stabilizes deep network training, allowing higher learning rates.
Task Decomposition: Teaching LLMs to Plan
Task decomposition for an LLM agent is like writing a recipe: break a big goal into a checklist of small, executable steps. It's vital for complex requests like planning a trip, but a bad initial plan can cause cascading failures that doom the entire process.
Data Quality Management: Is Your Data Fit for Use?
Data quality management ensures data is "fit for purpose." It's vital when training ML models or creating financial reports, as outcomes depend on data reliability. The footgun is treating quality as a one-time project, not a continuous process.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles