LLM
157 bites tagged LLM — interview questions with model answers, and 60-second explainers.
When to choose RAG over fine-tuning
RAG for fresh, factual, citable knowledge that changes often, fine-tuning for behavior, style, or format the model must internalize. matching technique to the kind of adaptation needed.
Differential privacy vs utility in LLM fine-tuning
Clipping plus calibrated noise per step, smaller epsilon means stronger privacy but degraded accuracy, tracking the privacy budget across epochs. understanding DP-SGD's noise-for-privacy bargain.
Rule-based versus model-based LLM guardrails
A guardrail is a programmatic check constraining LLM I/O; rule-based uses regex or blocklists, model-based uses a classifier like a moderation model to detect harmful content. practical safety controls.
Teacher-student knowledge distillation
A small student learns to mimic a large teacher's soft probability outputs, not just hard labels; goal is a compact model retaining most capability. grasp of model compression via distillation.
How the KV cache speeds transformer generation
Cache stores past keys and values so each new token only computes its own K, Q, V instead of recomputing all prior tokens, cutting per-step cost from quadratic to linear. understanding attention during decoding.
Model quantization benefits and risks
Quantization stores weights and activations in lower precision like INT8 or INT4; benefits are smaller memory and faster, cheaper inference; risk is accuracy loss. deployment-efficiency fundamentals.
Prompt injection versus jailbreak, and defenses
Injection hijacks the model via untrusted data overriding developer instructions; jailbreak coaxes a model past its safety policy. Defense: separate trusted instructions from untrusted data and filter. precise security distinctions.
Red teaming LLMs and harmful output categories
Red teaming deliberately probes a model to elicit harmful behavior; categories include toxic or hateful content and dangerous instructions like weapons or malware. understanding adversarial safety testing.
Setting up an LLM-as-a-judge evaluation
Define rubric, prefer pairwise comparison, randomize order, calibrate against humans; control position, verbosity, and self-preference bias. rigorous LLM-as-judge design.
How MMLU works and the contamination problem
MMLU is multiple-choice across 57 subjects scored by accuracy; contamination means test items leaked into pretraining, inflating scores. understanding of LLM benchmarks and their flaws.
How to evaluate a RAG system end to end
Measure retrieval (recall, precision, MRR, NDCG) and generation (faithfulness, answer relevance) separately, plus end-to-end correctness. ability to decompose RAG evaluation. judging only final answer quality with one number.
Zero-Shot, Few-Shot, and Chain-of-Thought Trade-offs
Zero-shot is cheap but weak on reasoning, few-shot adds demos at token cost, CoT boosts multi-step accuracy but spends the most tokens and latency. judgment about prompting strategies.
Hallucination Detection in LLMs
Hallucination detection is the set of techniques for flagging when a language model states something fluent but false or unsupported, using signals like self-consistency, model uncertainty, and grounding against retrieved evidence to catch fabrications before…
GitHub Copilot tunes context windows and routing
GitHub Copilot is improving how it handles context windows and routes requests between models. Better token efficiency and smarter routing mean faster, more relevant code suggestions for developers using the tool daily.
Most LLM Apps Need Workflows Not Agent Frameworks
Most LLM apps ship faster and more reliably as deterministic workflows than autonomous agents. Plain Python with structured outputs and local functions beats CrewAI and LangGraph for debugging. Map control flow in code before importing any agent framework.
How would you architect a multi-turn conversational RAG system?
This tests memory and query reformulation design beyond single-turn RAG. A strong answer covers 5-10 turn windows, LLM-based rewriting with coreference resolution, hybrid fallbacks, and summarized memory.
Identify RAG latency bottlenecks and propose optimizations
This tests systems thinking across the RAG pipeline. A strong answer names four bottlenecks—embedding, search, chunking, and generation—and pairs each with caching, index tuning, and distillation. Red flag: GPU scaling without indexing fixes.
Why does your RAG ignore or contradict retrieved context?
Tests separation of retrieval failures from generation grounding in RAG. Strong answers trace symptoms to root causes like bad chunks, prompt ordering, or parametric knowledge override, then outline systematic debugging. Do not just say hallucination.
NIST AI RMF for LLM Deployment
The NIST AI RMF is a pre-flight checklist for organizational AI risk, not just code bugs. Teams use it to justify LLM deployment across legal, security, and fairness dimensions.
Tool Definition Schema: Contracts for LLM Actions
A tool schema is JSON that tells an LLM what actions it can take. Use it when you want the model to call APIs instead of just chatting. The model only emits arguments; it never runs the tool, and vague descriptions cause silent failures.
Design an LLM ad copy system with human-in-the-loop
LoRA on approved copy, inference guardrails, human review, feedback as preference pairs for RLHF. Architecture for fine-tuning, guardrails, and human feedback loops. Treating review as static gate, not training signal.
LLM Inference Caching: Pay for Computation Once
LLM inference caching reuses past computations to cut costs and latency. It avoids reprocessing shared system prompts or serves full answers for common queries without hitting the model. The footgun: semantic caches can return a "similar" but incorrect answer.
Microsoft DeepSpeed: Training Massive Models Across GPUs
DeepSpeed trains models too big for one GPU by partitioning model states across many devices. It's essential for training foundation models like BLOOM, but its complexity is overkill for smaller models and misconfiguration can harm performance.
Open LLM Leaderboard: Separating Hype from Performance
The Open LLM Leaderboard is the de facto scoreboard for open-source models, providing reproducible benchmarks to cut through marketing hype. It helps you compare models on standardized tests, but remember that a high rank doesn't guarantee performance on your…
Get LLM bites daily.
Five a day, five minutes, offline. With quizzes so it sticks.
Open testing — you’ll join as an early tester.