Top 30 LLMs & Generative AI Interview Questions and Answers
30 multiple-choice questions on LLMs & Generative AI, of the kind that come up in a technical interview, drawn from 30 bites in the LLMs & Generative AI library. Answer them here or read straight down. Every question carries the correct option, why it is correct, and a link to the bite it came from.
Large language models, chatbots, agents, prompt engineering
30 questions. Pick an answer, or open “Show the answer” to read it.
Answers are graded in your browser. Nothing is saved, and no XP or streak is earned here. The app keeps score.
Question 1 of 30
Why does replacing sigmoid with ReLU in a deep network reduce vanishing gradients?
Show the answer
Answer: c · ReLU has a derivative of 1 for positive inputs, so it does not shrink gradients there
ReLU's unit derivative in its active region avoids the sub-one multiplications that cause sigmoid's gradients to vanish across depth. ReLU is unbounded above, does not normalize, and does not amplify gradients beyond one.
Question 2 of 30
Which property of one-hot vectors prevents a model from inferring that 'excellent' and 'outstanding' are similar?
Show the answer
Answer: a · They are mutually orthogonal, giving every pair zero cosine similarity
One-hot vectors are orthogonal by design, so every pair has zero cosine similarity and shares no geometric structure, meaning the model receives no hint that excellent and outstanding are related. Distractor A mistakes a practical symptom (huge size) for the root representational limitation.
Question 3 of 30
When training a large transformer, training loss falls while validation loss rises. Which action most directly addresses the root cause?
Show the answer
Answer: b · Apply early stopping or add regularization like weight decay
The divergence signals overfitting, where the model memorizes training noise instead of generalizable patterns. Early stopping and regularization directly combat this, whereas longer training or more parameters typically worsens the gap, and a lower learning rate does not directly fix memorization.
Read the full bite: Validation loss increases while training loss decreases: what is this?
Question 4 of 30
In full BPTT, what is the primary reason memory usage grows linearly with sequence length T?
Show the answer
Answer: c · All intermediate hidden states from the forward pass must be kept for the backward pass.
During BPTT the backward pass flows through every time step, so every forward hidden state must be retained, yielding O(T) memory. Option D repeats the common misconception that training memory is constant like inference memory, while option A confuses unrolling with parameter duplication.
Read the full bite: Explain BPTT and its computational and memory challenges for long sequences
Question 5 of 30
What core limitation of a vanilla RNN encoder-decoder does the attention mechanism specifically address?
Show the answer
Answer: a · Compressing the entire input into one fixed-size vector loses detail on long sequences
Attention removes the single fixed-context-vector bottleneck by letting the decoder weight all encoder states per step. Parallelism and positional encodings are Transformer concerns, not what classic seq2seq attention was introduced to fix.
Read the full bite: Attention in Sequence-to-Sequence Models
Question 6 of 30
Why are positional encodings necessary before the first self-attention layer in a standard Transformer?
Show the answer
Answer: a · Because self-attention computes unordered pairwise dot products, leaving attention scores unchanged when tokens are permuted
Self-attention is permutation-invariant: swapping tokens does not change their dot-product attention scores, so the model receives no sequence-order signal without positional encodings. Distractor A represents the common misconception that feed-forward layers can learn implicit order, but they process each position independently and cannot recover shuffled order.
Read the full bite: Explain positional encodings in Transformers and their necessity
Question 7 of 30
Beyond modeling long-range dependencies, what key training advantage does self-attention have over an LSTM?
Show the answer
Answer: c · It can process all sequence positions in parallel rather than step by step
Self-attention computes all token interactions simultaneously, enabling parallel training that sequential LSTMs cannot match. It actually needs added positional encodings and is quadratic, not cheaper, in sequence length.
Read the full bite: Self-Attention versus Recurrent Architectures
Question 8 of 30
Why does self-attention use three separate learned projections of the same input rather than the raw embeddings directly?
Show the answer
Answer: a · It allows the model to learn which token features to use for matching versus which to pass forward as content
The correct answer reflects that learned projections decouple the matching process from content retrieval, letting the model decide which aspects of a token to use for scoring and which to propagate forward. The distractor describing a decoder query with encoder key and value defines cross-attention, not self-attention, where all three matrices are derived from the same input sequence.
Read the full bite: Explain Q, K, and V matrices in self-attention
Question 9 of 30
Why does multi-head attention generally not cost much more than a single full-width attention head?
Show the answer
Answer: c · The model dimension is split across heads, so each head operates in a smaller subspace
The total dimension is partitioned among heads, keeping aggregate compute comparable to one full head while gaining diverse attention patterns. Heads run in parallel with their own projections, not shared weights, and each still uses softmax.
Question 10 of 30
During parallel teacher-forced training, what problem does the causal mask in decoder self-attention solve?
Show the answer
Answer: c · It stops position i from attending to future target tokens, preventing the model from copying answers
The causal mask forces token i to attend only to prior positions, preserving the autoregressive property and stopping the model from cheating by looking at future target tokens during parallel training. Option D describes the padding mask, which is a common misconception.
Read the full bite: What is masked in decoder self-attention and why?
Question 11 of 30
Which statement accurately captures the computational and memory complexity of exact self-attention as sequence length grows?
Show the answer
Answer: a · FLOPs grow as O(n^2) from the QK^T attention score matrix, while activation memory can be reduced to O(n) by recomputing rather than storing the full matrix.
Exact self-attention must perform O(n^2) FLOPs to compute the attention score matrix, but activation memory is not fundamentally quadratic: by recomputing scores instead of materializing the full n×n matrix in HBM, memory can be reduced to O(n). Option C is tempting because standard implementations often do store the full matrix, leading many to assume that O(n^2) memory is unavoidable even without approximation.
Read the full bite: Why is self-attention O(n^2) and what are the implications?
Question 12 of 30
Why is layer normalization preferred over batch normalization inside Transformer blocks for NLP?
Show the answer
Answer: c · Layer norm normalizes per token over features, independent of batch size and sequence length
Layer norm uses per-token feature statistics, so variable sequence lengths and small or padded batches do not destabilize it, and it matches at train and inference. Batch norm's batch-wise statistics are exactly what makes it unreliable here.
Read the full bite: Layer Norm and Residuals in Transformer Blocks
Question 13 of 30
What fundamentally distinguishes BERT's masked language modeling from GPT's causal language modeling?
Show the answer
Answer: d · MLM predicts masked tokens using bidirectional context; CLM predicts the next token from left context only
MLM reconstructs hidden tokens using context from both sides, suiting understanding, while CLM's left-only next-token prediction suits generation. Both are self-supervised, and it is CLM, not MLM, that generates autoregressively.
Question 14 of 30
Under Chinchilla's compute-optimal scaling laws, how should a fixed FLOP budget be divided between model parameters and training tokens?
Show the answer
Answer: b · Scale model parameters and training tokens equally, resulting in smaller models trained on much more data.
Chinchilla showed that for a fixed compute budget, parameters and training tokens should be scaled equally, meaning smaller models must be trained on proportionally more data to be compute-optimal. Option D describes the pre-Chinchilla approach that produced undertrained models, while option C incorrectly suggests holding model size fixed rather than scaling both dimensions together.
Read the full bite: How did Chinchilla change compute allocation between model size and data size?
Question 15 of 30
In a canonical 3D hybrid strategy for a massive model, why is tensor parallelism placed within a node while pipeline parallelism spans across nodes?
Show the answer
Answer: c · Tensor parallelism requires frequent all-reduce on activations that saturates high-latency inter-node links, whereas pipeline parallelism uses coarser point-to-point communication.
Tensor parallelism performs all-reduce on activations every layer, so keeping it inside a node avoids saturating slower inter-node bandwidth, while pipeline stages only exchange activations via point-to-point between passes. Option A is tempting because it mentions NVLink, but pipeline parallelism can also use fast links; the placement is driven by tensor parallelism's intense communication pattern, not pipeline's inability to use NVLink.
Read the full bite: Explain data, tensor, and pipeline parallelism and hybrid training strategy
Question 16 of 30
In a long pre-training run, the loss suddenly spikes. Which action should you take FIRST before applying any mitigation?
Show the answer
Answer: c · Check the gradient norm logs to confirm whether a gradient explosion occurred
The card emphasizes checking gradient norm logs first to confirm an explosion before deciding on recovery actions like rollback or hyperparameter changes. Option A is tempting because rolling back is a crucial recovery step, but doing so before confirming the root cause is premature and skips the diagnostic phase.
Read the full bite: What causes sudden loss spikes in long pre-training runs?
Question 17 of 30
When adapting a foundation model for high-stakes scientific reasoning, which approach best balances capability gains with bias control according to the described best practices?
Show the answer
Answer: d · Apply domain-adaptive pretraining on curated scientific literature, followed by instruction tuning and reasoning-based distillation, validating each component with ablations
The correct approach combines curated domain-adaptive pretraining, specialized instruction tuning, and reasoning distillation with ablation validation. The code-heavy distractor is wrong because although code strengthens general reasoning, scientific specialization requires curated literature and quality scoring, not raw unfiltered papers.
Read the full bite: How does pre-training dataset composition influence capabilities and biases?
Question 18 of 30
What is the core reason FlashAttention is faster than a naive attention implementation?
Show the answer
Answer: c · It is IO-aware: tiling and kernel fusion avoid writing the full attention matrix to slow HBM
FlashAttention computes exact attention but minimizes slow HBM traffic by tiling into SRAM and fusing kernels with an online softmax. It does not approximate, sparsify, or reduce the arithmetic complexity; it cuts memory movement.
Question 19 of 30
What must a few-shot classification prompt include that a zero-shot prompt must not?
Show the answer
Answer: d · Two to four labeled demonstrations showing feedback mapped to categories
Few-shot prompts rely on in-context learning through explicit input-output demonstrations, which zero-shot prompts omit by definition. Task instructions, output formats, and category labels are components present in both approaches, so selecting them confuses auxiliary structure with the defining presence of labeled examples.
Question 20 of 30
How does Chain-of-Thought prompting change the way a model solves a multi-step problem compared with standard prompting?
Show the answer
Answer: a · It makes the model output intermediate reasoning steps that become additional context for the final answer.
CoT prompting works by inducing the model to generate intermediate reasoning steps as tokens, creating additional context that the model attends to when producing the final answer. Distractor A is wrong because it anthropomorphizes the model as performing deeper internal thinking, whereas the actual mechanism is simply conditioning the final prediction on an explicit externalized reasoning trace.
Read the full bite: Explain Chain-of-Thought prompting, its reasoning mechanism, and ideal use cases
Question 21 of 30
Which approach to selecting and ordering demonstrations is most likely to improve text-to-SQL execution accuracy?
Show the answer
Answer: a · Retrieve candidates by SQL syntax similarity, balance pattern diversity across joins and aggregations, order simple to complex, and anchor with schema context.
Effective text-to-SQL prompting requires retrieving examples by SQL syntax, ensuring diverse query patterns, scaffolding from simple to complex, and grounding each example in schema context. The most tempting distractor relies on natural language embedding similarity, which often retrieves semantically close but syntactically irrelevant demonstrations, and front-loading complex SQL overwhelms the model rather than scaffolding it.
Read the full bite: How do you select in-context examples for text-to-SQL prompts?
Question 22 of 30
An engineer needs to improve an LLM's structured JSON extraction rate using only prompt changes. Which approach should they prioritize?
Show the answer
Answer: d · Include the exact JSON schema with empty values and add one to three valid input-output examples
Embedding an exact JSON skeleton and providing few-shot exemplars are the two prompt-based techniques that constrain the output distribution by making the desired token sequence highly probable. Chain-of-thought reasoning is a tempting distractor because it often increases verbosity and introduces stray tokens that break JSON validity.
Read the full bite: Describe two prompt-based techniques to ensure valid LLM JSON output
Question 23 of 30
For a multi-step reasoning task, which factor most increases chain-of-thought prompting's cost and latency relative to zero-shot?
Show the answer
Answer: d · The many additional output tokens generated for the reasoning steps
Chain-of-thought generates extra output tokens for intermediate reasoning, and sequential decoding makes output length the dominant cost and latency driver. It needs no fine-tuning or external tool calls per step.
Read the full bite: Zero-Shot, Few-Shot, and Chain-of-Thought Trade-offs
Question 24 of 30
Which design best avoids redundant demonstrations while keeping retrieval latency acceptable at scale?
Show the answer
Answer: a · Fetch a larger candidate set using ANN, apply a diversity reranker, then assemble a token-bounded prompt asynchronously.
Fetching a larger ANN candidate set and reranking for diversity prevents redundant examples while keeping latency low, and asynchronous assembly protects inference time. Option D is tempting but exact flat search is O(N) and too slow for online serving, and it lacks any diversity mechanism.
Read the full bite: Design dynamic few-shot example retrieval from a vector database
Question 25 of 30
Which statement best describes how supervised fine-tuning updates a pre-trained language model?
Show the answer
Answer: b · It updates existing weights by predicting the next token only on curated assistant completions, masking the prompt.
Supervised fine-tuning updates existing weights by continuing next-token prediction on curated prompt-completion data, computing loss only on assistant tokens while masking prompts. The most tempting distractor is wrong because SFT deliberately uses curated demonstrations and a small learning rate, not raw internet text like pre-training.
Read the full bite: Describe supervised fine-tuning for a pre-trained language model
Question 26 of 30
A team wants their LLM to reliably cite this week's internal sales figures. Why is fine-tuning the base model a poor primary tool for this requirement?
Show the answer
Answer: c · Fine-tuning mainly steers existing capabilities and is unreliable for injecting fresh, specific facts
Fine-tuning adapts and steers learned behaviors rather than dependably storing new facts, so retrieval is better for fresh data. It does not cost more than pre-training, nor does it wipe out language ability.
Question 27 of 30
When adapting a large model to dozens of tasks on a tight compute budget, why is LoRA strongly preferred over full fine-tuning?
Show the answer
Answer: a · It trains only a small number of extra parameters per task, keeping one shared base model and sharply reducing compute, storage, and serving costs while matching performance.
LoRA adds lightweight adapters per task around a single frozen base model, drastically cutting compute, storage, and inference costs while maintaining comparable performance. Distractor B is wrong because the card explicitly flags the belief that full fine-tuning always outperforms PEFT as a major red flag, and storing separate full checkpoints for many tasks is prohibitively expensive.
Read the full bite: Full fine-tuning or LoRA on a tight compute budget?
Question 28 of 30
Why does LoRA reduce GPU memory usage during fine-tuning compared to full fine-tuning?
Show the answer
Answer: b · It freezes the original weight matrix W and trains only low-rank matrices A and B, so gradients and optimizer states are only stored for A and B.
LoRA keeps the original weight matrix W frozen and only trains the small low-rank matrices A and B, which means gradients and optimizer states like those in Adam are only maintained for A and B, drastically reducing memory usage. The most tempting distractor claims the update is merged during training, but merging BA into W actually happens after training to preserve inference speed, not to save memory during fine-tuning.
Read the full bite: How does LoRA work and why is it memory-efficient?
Question 29 of 30
In the RLHF pipeline, why is a separate reward model trained on human preference comparisons before the final RL stage?
Show the answer
Answer: d · To generalize discrete human comparisons into a continuous scalar score that drives automated policy optimization
The reward model captures human preference numerically as a scalar so that automated RL optimization can occur. Option C is a common misconception: the reward model is not the final deployed policy; stage three produces the aligned language model that users interact with.
Read the full bite: Walk through RLHF's three stages, outputs, and purposes.
Question 30 of 30
During PPO-based RLHF, why is a KL-divergence penalty against the reference model added to the reward signal?
Show the answer
Answer: c · To prevent the policy from exploiting the reward model and degenerating into reward-hacking text
The KL term keeps the policy close to the trusted reference so it cannot drift into degenerate outputs that game the reward model. It has nothing to do with relabeling samples or removing preference data.
Could you explain these out loud?
That is what an interview actually tests. Tezvyn gives you questions like these with what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.