Intermediate everything in AI & ML, page 21
LangChain Agents: Giving LLMs a Toolkit
A LangChain Agent is an LLM given a toolkit and a goal. The agent's 'harness' prompts the model to pick tools, call them in a loop, and reason about the results until the task is complete. Use it to query databases or call external APIs.
Agent Memory: Short-Term vs. Long-Term Recall
Agent memory gives an LLM a sense of history, separating fleeting conversation context from persistent knowledge. Short-term memory tracks the current chat, while long-term memory recalls user facts across sessions.
Agentic Reasoning: LLMs that Plan, Act, and Learn
Agentic reasoning treats an LLM as an autonomous agent that interacts with its environment. It plans tasks, uses tools like APIs, and learns from feedback to solve complex problems. The footgun is assuming its plans are optimal or actions are always correct.
Cross-Encoder Re-ranking: Accuracy Over Speed
A cross-encoder re-ranks search results by reading the query and each document together, allowing it to spot subtle connections. It's the second, high-precision step in a search pipeline, re-ordering a small list of candidates.
Evaluating RAG Systems: Metrics for Retrieval and Generation
Evaluating a RAG system means grading its two parts: retrieval and generation. Metrics like relevance check if the right documents were found, while faithfulness and accuracy check if the final answer correctly uses those documents.
Document Chunking: Slicing Text for LLMs
Think of chunking as preparing text "bites" for an LLM. It breaks large documents into smaller, meaningful segments to fit a model's context window and improve search. It's essential for Retrieval-Augmented Generation (RAG) and semantic search.
Model Merging: Combine LLM Skills Without Retraining
Model merging blends specialized LLMs into one, like creating a custom alloy from different metals. It's used to combine a coding expert with a legal expert, for example, without costly retraining.
Amazon Bedrock: One API for Many AI Models
Amazon Bedrock is an API gateway for foundation models, letting you switch AI providers without rewriting code. It's used to build generative AI apps while avoiding vendor lock-in.
MaaS: Renting AI Brains via API
Model-as-a-Service (MaaS) is like renting a pre-trained AI expert via an API. Instead of building and training your own models, you pay to use powerful, ready-made ones for tasks like text generation or image analysis.
The Llama Model Family: Open-Source AI for Production
Think of Llama not as one model, but a family of open-source AIs you can run anywhere. Use it for cost-effective, fine-tuned applications like internal search or when you need full control. The biggest mistake is mis-sizing the model for your task.
The EU AI Act: Risk-Based AI Regulation
The EU AI Act isn't a blanket ban but a risk-based framework. It sorts AI into tiers—from unacceptable to minimal risk—and applies rules proportionally, affecting any company with AI users in the EU. The footgun is assuming it only applies to EU companies.
Fairness Metrics: Quantifying AI's Impact on People
Fairness metrics translate "fairness" into a measurable score, checking if a model treats groups equitably. They are crucial for models in hiring or lending.
FlashAttention: Faster, Memory-Efficient Exact Attention
FlashAttention is an IO-aware algorithm that computes exact attention faster and with less memory. It avoids slow GPU memory transfers, making it a key optimization for training and serving large models on modern GPUs.
ONNX Runtime: Run Any AI Model, Anywhere
ONNX Runtime is a universal engine for AI models, letting you run them efficiently on any hardware, from cloud GPUs to a user's browser. It's used to deploy models for fast inference on servers or mobile devices.
Post-Training Quantization: Shrink Models Without Retraining
Post-Training Quantization (PTQ) shrinks a pre-trained model by converting its weights to lower precision, like turning a WAV file into an MP3. Use it to run large models on consumer GPUs without costly retraining.
ML Interpretability: Cracking Open the Black Box
ML interpretability cracks open the 'black box' to explain *why* a model made a specific decision. It's essential in high-stakes fields like finance or medicine to ensure automated decisions are fair. The footgun is trusting accuracy alone.
Adversarial Attacks: Tricking LLMs into Misbehaving
Adversarial attacks are inputs designed to trick an LLM, bypassing its safety alignment. This is how "jailbreaks" coax models into generating harmful content. The footgun is assuming safety training makes a model foolproof; it just makes attacks more subtle.
LLM Guardrails: Keeping Model Outputs on Track
LLM guardrails are safety policies that steer model outputs, acting like bumpers in a bowling alley to prevent responses from going off-topic, leaking data, or generating harmful content. They are crucial for topic control and preventing prompt injections.
LLM Red Teaming: Adversarial Security Testing
LLM Red Teaming is a simulated attack where you proactively try to break your own AI to find security flaws. It's used to test for vulnerabilities like prompt injection or data leakage, which traditional security tools miss.
HellaSwag: A Benchmark Designed to Fool LLMs
HellaSwag is a commonsense benchmark designed to fool language models. It asks an AI to pick the most logical sentence ending, but the wrong answers are specifically generated to trick machines, not humans. It's used to test for true contextual understanding.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles