Top 30 Advanced LLMs & Generative AI Interview Questions and Answers
30 advanced multiple-choice LLMs & Generative AI interview questions, the deep end: internals, failure modes, and the design calls a senior engineer is expected to defend. They come from 30 bites in the LLMs & Generative AI library, the hardest slice of the 145 LLMs & Generative AI interview questions in the library. Answer them here or read straight down. Every question carries the correct option, why it is correct, and a link to the bite it came from.
Large language models, chatbots, agents, prompt engineering
30 questions. Pick an answer, or open “Show the answer” to read it.
Answers are graded in your browser. Nothing is saved, and no XP or streak is earned here. The app keeps score.
Question 1 of 30
Why are positional encodings necessary before the first self-attention layer in a standard Transformer?
Show the answer
Answer: a · Because self-attention computes unordered pairwise dot products, leaving attention scores unchanged when tokens are permuted
Self-attention is permutation-invariant: swapping tokens does not change their dot-product attention scores, so the model receives no sequence-order signal without positional encodings. Distractor A represents the common misconception that feed-forward layers can learn implicit order, but they process each position independently and cannot recover shuffled order.
Read the full bite: Explain positional encodings in Transformers and their necessity
Question 2 of 30
Which statement accurately captures the computational and memory complexity of exact self-attention as sequence length grows?
Show the answer
Answer: a · FLOPs grow as O(n^2) from the QK^T attention score matrix, while activation memory can be reduced to O(n) by recomputing rather than storing the full matrix.
Exact self-attention must perform O(n^2) FLOPs to compute the attention score matrix, but activation memory is not fundamentally quadratic: by recomputing scores instead of materializing the full n×n matrix in HBM, memory can be reduced to O(n). Option C is tempting because standard implementations often do store the full matrix, leading many to assume that O(n^2) memory is unavoidable even without approximation.
Read the full bite: Why is self-attention O(n^2) and what are the implications?
Question 3 of 30
Why is layer normalization preferred over batch normalization inside Transformer blocks for NLP?
Show the answer
Answer: c · Layer norm normalizes per token over features, independent of batch size and sequence length
Layer norm uses per-token feature statistics, so variable sequence lengths and small or padded batches do not destabilize it, and it matches at train and inference. Batch norm's batch-wise statistics are exactly what makes it unreliable here.
Read the full bite: Layer Norm and Residuals in Transformer Blocks
Question 4 of 30
What is the core reason FlashAttention is faster than a naive attention implementation?
Show the answer
Answer: c · It is IO-aware: tiling and kernel fusion avoid writing the full attention matrix to slow HBM
FlashAttention computes exact attention but minimizes slow HBM traffic by tiling into SRAM and fusing kernels with an online softmax. It does not approximate, sparsify, or reduce the arithmetic complexity; it cuts memory movement.
Question 5 of 30
Which design best avoids redundant demonstrations while keeping retrieval latency acceptable at scale?
Show the answer
Answer: a · Fetch a larger candidate set using ANN, apply a diversity reranker, then assemble a token-bounded prompt asynchronously.
Fetching a larger ANN candidate set and reranking for diversity prevents redundant examples while keeping latency low, and asynchronous assembly protects inference time. Option D is tempting but exact flat search is O(N) and too slow for online serving, and it lacks any diversity mechanism.
Read the full bite: Design dynamic few-shot example retrieval from a vector database
Question 6 of 30
Zeroing the KL-divergence penalty in RLHF PPO most directly leads to which failure mode?
Show the answer
Answer: a · The policy over-optimizes the proxy reward model, yielding adversarial outputs that score highly but are incoherent to humans.
Without the KL trust-region anchor, the policy drifts from the reference distribution and exploits weaknesses in the learned reward model, producing text that maximizes proxy reward but degrades in human judgment. This is distinct from overfitting the training corpus or benign gradient instability; the core failure is reward hacking caused by misalignment between the proxy and true human preferences.
Read the full bite: What does the KL-divergence penalty do in RLHF PPO, and if zeroed?
Question 7 of 30
What is the central mathematical idea that lets DPO align a model without a separate reward model?
Show the answer
Answer: d · The optimal KL-constrained policy lets the reward be re-expressed via the policy and reference probabilities
DPO exploits the closed-form optimal policy of the KL-constrained objective to fold the reward into the policy-reference probability ratio, removing the reward network. It still uses rejected responses and standard gradient descent.
Read the full bite: Direct Preference Optimization explained
Question 8 of 30
Why is a cross-encoder re-ranker applied only to a shortlist of candidates rather than the entire corpus?
Show the answer
Answer: c · It scores each query-document pair with joint attention, which is accurate but too slow at corpus scale
Cross-encoders jointly attend over the pair for high precision but cannot be precomputed, making full-corpus scoring infeasible, so they rescore a shortlist. They are more accurate than bi-encoders, not less.
Read the full bite: Hybrid search and re-ranking for retrieval
Question 9 of 30
In a production RAG pipeline, which optimization best demonstrates systems-level thinking about retrieval latency?
Show the answer
Answer: d · Tune HNSW index parameters and implement hybrid dense-plus-sparse retrieval with BM25 pruning
Tuning HNSW and adding BM25 hybrid pruning directly addresses vector search as a tunable distributed component rather than a black box. Option A is tempting because scaling GPUs is a common reflex, but it ignores that retrieval and embedding can consume 30 to 50 percent of total latency while failing to address index configuration or chunking strategy.
Read the full bite: Identify RAG latency bottlenecks and propose optimizations
Question 10 of 30
Why is a hybrid query reformulation pipeline—using both rules and an LLM—preferred over an LLM-only rewriter in a multi-turn RAG system?
Show the answer
Answer: b · Rules handle high-volume simple references with low latency, while the LLM fallback handles complex coreference without rewriting every query
The card describes LLM rewriting as effective for coreference resolution but advocates a hybrid to route common cases to fast rule-based matching and reserve LLM calls for complex references, managing latency and cost. Option C is tempting but wrong because the card explicitly credits LLMs with resolving pronouns; the hybrid exists for efficiency, not because LLMs lack capability.
Read the full bite: How would you architect a multi-turn conversational RAG system?
Question 11 of 30
Why is a carefully written system prompt insufficient as the primary defense against indirect prompt injection in a tool-using agent?
Show the answer
Answer: c · Models can be jailbroken, so prompts are guidance, not an enforceable security boundary
Prompt instructions can be overridden by injected content, so real boundaries come from sandboxing, least privilege, and approval gates. A prompt does not sandbox tools or stop inference-time injection.
Question 12 of 30
In this booking agent, which action should require explicit user confirmation before execution?
Show the answer
Answer: d · Creating the calendar event, because it mutates external state and notifies others
The booking call is the irreversible, externally visible action, so it warrants confirmation. Contact lookup and date computation are read-only or internal and can run without interrupting the user.
Read the full bite: Designing an agent that resolves ambiguity
Question 13 of 30
What allows DDIM to sample in far fewer steps than DDPM using the very same trained network?
Show the answer
Answer: c · It uses a non-Markovian deterministic process consistent with DDPM's learned marginals, enabling step skipping
DDIM exploits that DDPM training only fixes the marginals, so a deterministic non-Markovian process can reuse the same network and skip steps. It needs no retraining and removes, rather than adds, stochastic noise.
Question 14 of 30
In training-free diffusion inpainting, what happens to the unmasked region during each denoising step?
Show the answer
Answer: c · It is overwritten with the original image diffused to the current noise level to preserve known pixels
Re-injecting the correctly noised original into the unmasked region forces known pixels to stay correct while only the mask is generated. Regenerating everything would discard the original content and cause seams.
Question 15 of 30
Why does adding temporal attention layers fix flicker that simply generating each frame with the same image model cannot?
Show the answer
Answer: a · Temporal attention lets each frame attend to others, so content stays consistent across time
Cross-frame attention lets frames exchange information so identities and textures remain stable, which independent per-frame generation cannot do since those frames never communicate. Temporal layers complement, not replace, spatial ones.
Question 16 of 30
What is the core architectural difference between how LLaVA and Flamingo connect vision to the LLM?
Show the answer
Answer: b · LLaVA projects image features into input tokens; Flamingo inserts gated cross-attention layers into a frozen LLM
LLaVA maps vision features into the embedding space and feeds them as input tokens, while Flamingo adds gated cross-attention inside a frozen LLM. Neither feeds raw pixels nor retrains the LLM from scratch.
Question 17 of 30
Why is sampling batches in proportion to each dataset's raw size a poor strategy when training a multimodal model?
Show the answer
Answer: a · A large dataset can dominate batches, starving rare modalities and degrading underrepresented skills
Size-proportional sampling lets the biggest corpus swamp the batch, causing modality collapse and forgetting of underrepresented data, so explicit mixing weights are preferred. The issue is balance and quality, not implementation or memory.
Read the full bite: Batching strategy for multimodal training
Question 18 of 30
You run an LLM judge comparing two answers and always place your model's answer first. What confound does this introduce?
Show the answer
Answer: b · Position bias: the judge may systematically favor whichever answer appears first regardless of quality
Fixed ordering lets the judge's tendency to favor a slot masquerade as quality, which is position bias, fixed by swapping order. Verbosity bias is about length, and the other options describe unrelated phenomena.
Question 19 of 30
Why is tensor parallelism typically kept within a single node while pipeline parallelism spans nodes?
Show the answer
Answer: b · Tensor parallelism all-reduces every layer needing fast interconnect; pipeline parallelism only passes activations at stage boundaries
Tensor parallelism's frequent high-volume all-reduces demand NVLink-class links, so it stays intra-node; pipeline parallelism's lighter boundary communication tolerates slower cross-node links. Neither replicates the full model, and they are commonly combined.
Read the full bite: Tensor versus pipeline parallelism for large models
Question 20 of 30
What shared insight lets GPTQ and AWQ preserve accuracy at INT4 where naive rounding fails?
Show the answer
Answer: c · Weights contribute unequally to output, so quantization should protect or compensate the important ones using activation statistics
Both methods exploit unequal weight importance, GPTQ compensating via second-order info and AWQ protecting salient activation-aligned channels. They are post-training, do not retrain like QAT, and do not keep everything in FP16.
Question 21 of 30
In DP-SGD, what is the relationship between the epsilon parameter and the strength of the privacy guarantee?
Show the answer
Answer: a · A smaller epsilon gives a stronger privacy guarantee but typically lowers utility
Smaller epsilon bounds privacy loss more tightly, requiring more noise and usually reducing accuracy. Larger epsilon is a weaker guarantee, so the first option reverses the relationship.
Read the full bite: Differential privacy vs utility in LLM fine-tuning
Question 22 of 30
Why does speculative decoding speed up generation without changing the model's output distribution?
Show the answer
Answer: c · The large target model still verifies every drafted token, accepting only matching ones
A small draft model proposes tokens, but the target model verifies them in parallel and accepts only those it would have produced, so outputs are unchanged. It is unrelated to temperature or layer pruning.
Read the full bite: Three techniques to cut LLM inference latency
Question 23 of 30
When debugging an agentic system built with a tool-calling framework, what is a primary challenge a senior engineer is likely to encounter?
Show the answer
Answer: b · Frameworks often abstract away the specific prompts sent to the LLM and its raw tool-calling outputs.
The card explicitly states that 'Framework Obfuscation' is a major failure mode, where frameworks hide underlying prompts and model responses, making debugging difficult. While LLM non-determinism (Option A) is a general challenge, the card specifically highlights framework abstraction as the primary debugging issue in this context.
Read the full bite: How do agents use tool-calling and what can go wrong?
Question 24 of 30
Which scenario most compellingly justifies choosing LoRA over full fine-tuning for adapting a large language model?
Show the answer
Answer: b · B. The need is to quickly develop and deploy multiple distinct task-specific models, each requiring minimal GPU memory and storage for efficient management.
Option B correctly identifies LoRA's strength in enabling efficient, rapid development and deployment of multiple specialized models under resource constraints. While LoRA does not add inference latency (Option D), its primary justification often lies in its significant resource savings and ability to manage many adaptations, and its training pipeline is simpler, not more complex.
Read the full bite: When would you use LoRA vs full fine-tuning?
Question 25 of 30
In a RAG system, which retrieval method would be most effective for accurately finding a document containing a specific, unique product identifier like "SKU-XYZ-789"?
Show the answer
Answer: b · Sparse retrieval, as it excels at exact keyword matching using inverted indices.
Sparse retrieval is explicitly stated to excel at keyword-heavy queries and identifiers, making it ideal for exact matches like product SKUs. Dense retrieval, conversely, fails on queries requiring exact keyword matches because its semantic embedding can obscure specific identifiers, making option A a common misconception.
Read the full bite: Trade-offs between dense and sparse retrieval in RAG?
Question 26 of 30
Which statement accurately describes a key distinction in how RLHF and DPO achieve preference alignment for LLMs?
Show the answer
Answer: a · A. RLHF relies on an intermediate reward model to guide reinforcement learning, whereas DPO directly optimizes the LLM's policy using a specialized loss function.
Option A is correct because the card explicitly states RLHF involves training a separate reward model and using reinforcement learning, while DPO 'bypasses the explicit reward model and RL training' by optimizing the LLM directly. Option B is incorrect as both RLHF and DPO are for preference alignment, not task teaching (which is SFT) or exclusively conversational fluency.
Read the full bite: Explain Supervised Fine-Tuning, RLHF, and DPO
Question 27 of 30
What is the primary reason Mixture-of-Experts (MoE) models are considered 'larger' in parameter count but 'cheaper to run' during inference?
Show the answer
Answer: a · A) For each input, only a small, dynamically selected subset of experts is activated, significantly reducing computational operations (FLOPs).
The efficiency of MoE models stems from sparse activation: a router dynamically selects and activates only a small subset of experts for each input token, drastically reducing the floating-point operations (FLOPs) required during inference. Option B is incorrect because the efficiency comes from *not* running all experts, unlike a simple ensemble where all components might be processed.
Read the full bite: Why are MoE models larger but cheaper to run?
Question 28 of 30
Which strategy offers the most comprehensive and robust defense against hallucination in a production LLM application?
Show the answer
Answer: b · B. Combining Retrieval-Augmented Generation (RAG) with low decoding temperature, structured output validation, and user feedback mechanisms.
The card emphasizes a multi-layered, systemic approach. Option B correctly combines RAG (data grounding), low temperature (model configuration), structured output validation, and user feedback (application safeguards), which are all critical components. Option A is insufficient as prompt engineering alone is described as a weak defense.
Read the full bite: How to reduce hallucination in a production LLM application?
Question 29 of 30
What is the primary reason the KV cache is crucial for efficient autoregressive LLM inference?
Show the answer
Answer: a · It avoids recomputing Key and Value tensors for previously processed tokens in the self-attention mechanism.
The KV cache's core function is to store the Key and Value tensors for all tokens processed so far, preventing their redundant recomputation at each new token generation step during autoregressive inference. Option D is incorrect because the O(N) complexity per step applies to subsequent token generation, not the initial prompt encoding itself.
Read the full bite: What is the KV cache and why does it matter for serving LLMs?
Question 30 of 30
Which architectural feature primarily enables transformers to mitigate the vanishing gradient problem for long-range dependencies in sequences?
Show the answer
Answer: a · A. The multi-head self-attention mechanism, which creates direct, parallel gradient paths between any token pair.
The correct answer is A because the transformer's parallel self-attention mechanism creates direct, O(1) gradient paths between any two tokens, regardless of their distance in the sequence, thereby preventing gradients from vanishing over long dependencies. Option B is a tempting distractor as residual connections do help with vanishing gradients, but primarily with respect to network *depth*, not the *sequence length* problem addressed by self-attention.
Read the full bite: What is the vanishing gradient problem and how do transformers avoid it?
Could you explain these out loud?
That is what an interview actually tests. Tezvyn gives you questions like these with what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.