Intermediate everything in LLMs & Generative AI, page 3
Mixture of Experts architecture and routing
Many expert FFNs per layer, a router picks top-k experts per token, only those compute so active params are far fewer than total.
Direct versus indirect injection and agent defenses
Direct injection comes from the user prompt; indirect hides in third-party data the agent ingests like web pages.
Detecting RAG hallucinations with a confidence score
Decompose the answer into claims, verify each against retrieved context with NLI or an LLM judge, aggregate into a faithfulness confidence score, and flag unsupported claims.
What memory problem PagedAttention solves
Pre-allocating contiguous max-length cache per sequence wastes memory through internal and external fragmentation; PagedAttention stores KV in fixed non-contiguous blocks like OS paging.
PTQ versus QAT for INT8 quantization
PTQ quantizes a trained model with light calibration, fast and cheap but more accuracy loss; QAT simulates quantization during training, higher accuracy but costly.
Prompt injection versus jailbreak, and defenses
Injection hijacks the model via untrusted data overriding developer instructions; jailbreak coaxes a model past its safety policy. Defense: separate trusted instructions from untrusted data and filter.
Designing input and output guardrails for a chatbot
Input guardrails filter or classify user prompts (injection, off-topic, PII) before the model; output guardrails validate responses for toxicity, leakage, and policy before sending.
Reference-free evaluation for open-ended dialogue
ROUGE punishes valid paraphrases; use reference-free LLM-as-judge or learned scorers rating coherence, relevance, and groundedness.
How MMLU works and the contamination problem
MMLU is multiple-choice across 57 subjects scored by accuracy; contamination means test items leaked into pretraining, inflating scores.
How to evaluate a RAG system end to end
Measure retrieval (recall, precision, MRR, NDCG) and generation (faithfulness, answer relevance) separately, plus end-to-end correctness.
How FID is calculated versus Inception Score
FID fits Gaussians to Inception features of real and fake images then measures Frechet distance; it uses real references and detects mode collapse.
Aligning text and image representations
Contrastive learning like CLIP pulls matched image-text pairs together and pushes mismatches apart; alternatively projection layers map one modality into a frozen model's space.
How Stable Diffusion generates images
The text encoder turns the prompt into embeddings, the U-Net predicts noise to remove conditioned on those embeddings, and the scheduler controls how noise is stepped down over iterations in…
Designing a Visual Question Answering system
Encode the image with a vision backbone, encode the question with a text encoder, fuse them via cross-attention into a joint representation, then decode or classify the answer.
Evaluating image generation: FID and IS
FID compares feature distributions of real and generated images, lower is better; Inception Score rewards confident, diverse classes but ignores real data.
Evaluating a RAG system end to end
Measure retrieval with context recall or precision, and generation with faithfulness and answer relevance, attributing failures to the right stage.
Reward models in RLHF and PPO
It learns from human preference comparisons to score responses, then supplies the reward signal that PPO maximizes while a KL penalty keeps the policy near the reference.
Zero-Shot, Few-Shot, and Chain-of-Thought Trade-offs
Zero-shot is cheap but weak on reasoning, few-shot adds demos at token cost, CoT boosts multi-step accuracy but spends the most tokens and latency.
Why Multi-Head Attention
Multiple heads attend to different subspaces and relations in parallel, which one big head averages away.
Attention in Sequence-to-Sequence Models
Attention computes per-step weighted sums over all encoder states, fixing the information bottleneck for long inputs.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles