Top 30 LLMs & Generative AI Concepts Quiz
30 multiple-choice questions on the LLMs & Generative AI fundamentals, drawn from 30 bites in the LLMs & Generative AI library. Answer them here or read straight down. Every question carries the correct option, why it is correct, and a link to the bite it came from.
Large language models, chatbots, agents, prompt engineering
30 questions. Pick an answer, or open “Show the answer” to read it.
Answers are graded in your browser. Nothing is saved, and no XP or streak is earned here. The app keeps score.
Question 1 of 30
Which situation best illustrates why a loss function should not be the only measure of a model's real-world effectiveness?
Show the answer
Answer: a · The model achieves a low loss score, but its predictions are biased or provide no practical value to users.
The card explicitly states that a low loss score doesn't guarantee real-world usefulness, as outputs can still be nonsensical, biased, or unhelpful. This highlights that loss functions are for optimization, not the final arbiter of a model's real-world usefulness. Option C describes a scenario where a loss function isn't typically used, rather than a limitation of relying on it as a sole metric when it is applied.
Read the full bite: Loss Function: Quantifying 'How Wrong' a Model Is
Question 2 of 30
What fundamental capability do activation functions primarily provide to neural networks?
Show the answer
Answer: c · They introduce non-linear transformations, allowing the network to model complex, non-linear relationships.
The card emphasizes that activation functions introduce non-linearity, which is crucial for neural networks to learn complex patterns beyond simple linear relationships. Without non-linearity, a multi-layered network would collapse into a single linear model. Other options describe secondary effects or unrelated concepts.
Read the full bite: Activation Functions: Making Neural Networks Nonlinear
Question 3 of 30
Which statement best describes the fundamental way regularization helps a model avoid overfitting?
Show the answer
Answer: b · It encourages the model to learn smaller, less extreme parameter values.
Regularization adds a penalty to the loss function for large parameter values, which encourages the model to learn simpler, less extreme weights, thus preventing it from memorizing noise. Option A describes a beneficial outcome of regularization, but not its direct mechanism; it doesn't explicitly "ignore" data points but rather reduces their influence by constraining parameter magnitudes.
Read the full bite: Regularization: Penalizing Complexity to Prevent Overfitting
Question 4 of 30
What is the fundamental principle behind how Word2Vec represents word meaning?
Show the answer
Answer: c · Words with similar meanings are mapped to points that are spatially close in a multi-dimensional vector space.
The core idea of Word2Vec is that words with similar meanings are represented by vectors that are close to each other in a multi-dimensional space, capturing semantic similarity through spatial proximity. Option A describes a method Word2Vec aims to improve upon, as it fails to capture semantic relationships.
Read the full bite: Word2Vec: Word Meaning as a Point in Space
Question 5 of 30
What is the most significant consequence of the vanishing gradient problem in deep neural networks?
Show the answer
Answer: b · Early layers of the network fail to learn effective features from the input data.
The card states that vanishing gradients cause 'the gradients for the earliest layers become so small they effectively vanish, and those layers stop learning,' meaning they cannot learn effective features. Option D is incorrect because vanishing gradients lead to a failure to learn, not typically overfitting. Options C and D are general problems but not the specific, direct consequence on the learning of early layers.
Question 6 of 30
For which scenario would an LSTM be preferred over a traditional RNN?
Show the answer
Answer: c · Tasks requiring memory of context from distant points in a long sequence.
LSTMs are specifically designed to overcome the vanishing gradient problem in traditional RNNs, enabling them to maintain and utilize information from far back in a sequence. For short-term dependencies, a simpler RNN might be more efficient, and for large-scale parallel tasks, Transformers are often preferred.
Read the full bite: LSTMs: Giving Neural Networks a Longer Memory
Question 7 of 30
What is the primary limitation of a basic Seq2Seq model when dealing with very long input sequences?
Show the answer
Answer: a · The fixed-size context vector acts as an information bottleneck, leading to the forgetting of early input details.
The card states that the "primary weakness of basic Seq2Seq is its reliance on a single, fixed-size context vector" which "becomes an information bottleneck" for "very long inputs," causing the model to "forget details from the beginning of the input." Option D describes the problem that Seq2Seq was designed to solve, not its limitation.
Read the full bite: Seq2Seq: Turning One Sequence Into Another
Question 8 of 30
How does self-attention primarily determine the relevance of other words to a given word in a sequence?
Show the answer
Answer: d · By comparing the current word's Query vector with every other word's Key vector.
Self-attention calculates relevance by measuring the similarity between a word's Query vector and other words' Key vectors. This process allows it to directly identify and weigh the importance of all other words, unlike sequential processing or fixed context windows.
Read the full bite: Self-Attention: The Transformer's Core Idea
Question 9 of 30
What distinct stabilization mechanism does Layer Normalization provide in deep networks, complementing residual connections?
Show the answer
Answer: a · It normalizes feature activations within a layer, maintaining a stable scale for subsequent layers.
Layer Normalization's primary role is to rescale activations within a layer to a stable distribution (mean 0, std 1), which stabilizes the inputs to the next layer. Option C describes the function of residual connections, which provide direct paths for information and gradients.
Read the full bite: Residual Connections & Layer Norm: The Transformer's Stabilizers
Question 10 of 30
Which scenario best highlights the necessity of cross-attention in a neural network?
Show the answer
Answer: d · Translating a sentence where the decoder needs to align specific output words with relevant input words.
Cross-attention is crucial in sequence-to-sequence tasks like translation, enabling the decoder to dynamically focus on relevant parts of the encoder's output. Option A describes the information bottleneck that cross-attention was designed to overcome, not its necessity.
Read the full bite: Cross-Attention: How Models Connect Two Ideas
Question 11 of 30
What fundamental limitation necessitates the tokenization step for feeding text into a Transformer model?
Show the answer
Answer: b · Transformer models are mathematical functions that exclusively process numerical tensors.
The card explicitly states that "Neural networks, including Transformers, are mathematical functions that operate on numbers, not raw text strings." Thus, text must be converted into a numerical tensor format. While tokenization helps manage vocabulary and prepares for embeddings, the core reason is the model's numerical input requirement.
Read the full bite: Transformer Preprocessing: From Text to Tensors
Question 12 of 30
Before the softmax function is applied, what does the Transformer's final linear layer directly output?
Show the answer
Answer: c · Raw, un-normalized scores (logits) for each word in the vocabulary.
The linear layer's specific function is to convert the decoder's abstract vector into raw, un-normalized scores (logits) for every word. The softmax function then takes these logits and transforms them into a probability distribution, not the linear layer itself.
Read the full bite: Transformer: The Final Linear and Softmax Layers
Question 13 of 30
For which task would a Causal Language Model be the most suitable choice?
Show the answer
Answer: c · Extending a partial sentence into a complete, coherent paragraph
Causal Language Models are designed for open-ended text generation, predicting the next word based only on preceding words, making them ideal for continuing or extending text. Tasks like sentiment analysis, information extraction, or summarization typically require understanding the entire input, which is a limitation for CLMs.
Read the full bite: Causal Language Modeling: The Autocomplete Engine
Question 14 of 30
For what primary purpose is Common Crawl most effectively utilized?
Show the answer
Answer: b · Offering a massive, raw corpus of web content for large-scale research and model training.
The card states Common Crawl is "the raw material for training many LLMs and for academic research on web-scale data" and is ideal for projects that "can handle the scale and messiness of raw web data." It explicitly notes it is "not a search engine" and is "famously noisy," requiring extensive cleaning, making options A, B, and D incorrect.
Read the full bite: Common Crawl: A Free Snapshot of the Entire Web
Question 15 of 30
What problem does loss scaling primarily address in mixed-precision training?
Show the answer
Answer: d · The potential for small gradient values to become zero in FP16.
Loss scaling's primary purpose is to inflate small gradient values before converting them to FP16, preventing them from becoming zero (underflow), which would otherwise cause training to fail. While a master copy of weights is kept in FP32 for stability, loss scaling directly ensures the gradients themselves are numerically viable.
Read the full bite: Mixed-Precision Training: Faster Training with Less Memory
Question 16 of 30
What is the core mechanism by which AdamW ensures L2 regularization is consistently applied, unlike the original Adam?
Show the answer
Answer: c · It applies the weight decay as a direct subtraction from the weights, independent of the gradient's adaptive scaling.
The card states AdamW applies weight decay by directly subtracting a fraction of the weight's value in a separate step, decoupling it from the adaptive learning rate mechanism. Option A is incorrect because AdamW doesn't just reorder the application within the gradient calculation; it completely separates it and applies it directly to the weights.
Read the full bite: AdamW: Decoupling Weight Decay for Better Generalization
Question 17 of 30
When is a learning rate schedule particularly crucial for achieving state-of-the-art results?
Show the answer
Answer: d · When fine-tuning large, complex models such as transformers.
C is correct because the card states schedules are "standard practice for training and fine-tuning large models, especially transformers... essential for achieving state-of-the-art results." B is a tempting distractor, but the card explicitly warns against using external schedules with optimizers like AdaFactor due to potential conflicts.
Read the full bite: Learning Rate Scheduling: A Gearbox for Model Training
Question 18 of 30
For which type of computational problem is data parallelism most appropriate?
Show the answer
Answer: b · Dividing a large dataset among multiple processors, each running the same program on its assigned portion.
Data parallelism is designed for scenarios where a single, computationally intensive operation needs to be performed on a massive dataset, with each processor handling a different chunk. Option A describes task parallelism, while Option C highlights a situation where data parallelism would be inefficient due to communication overhead.
Read the full bite: Data Parallelism: One Task, Many Data Chunks
Question 19 of 30
According to the card, what is the most accurate mental model for understanding how a Large Language Model (LLM) processes information?
Show the answer
Answer: b · A sophisticated pattern-matching engine that predicts word sequences based on learned statistical relationships.
The card describes an LLM as a "sophisticated pattern-matching engine for words and ideas" that learns "statistical relationships between words" to predict text. Option C is incorrect because the card explicitly states an LLM is "not a database of facts".
Question 20 of 30
Under a fixed FLOP budget, a team must choose between a 100B-parameter model on 200B tokens and a 50B-parameter model on 400B tokens. What should they expect?
Show the answer
Answer: b · The 50B model will likely match or outperform the 100B model while costing less to serve
The card states that for a fixed FLOP budget, scaling parameters and tokens equally is optimal, so the smaller model on more data can match or beat the larger one while being cheaper to serve. Option D reflects the outdated assumption that parameters alone drive performance, while D incorrectly assumes overfitting rather than undertraining is the risk.
Read the full bite: LLM Scaling Laws: Match Parameters to Tokens
Question 21 of 30
What is the primary advantage of employing pipeline parallelism in deep learning model training?
Show the answer
Answer: d · It enables the training of models whose total parameter count exceeds the memory capacity of a single GPU.
B is correct as the card states pipeline parallelism is for models 'too large to fit into the memory of a single accelerator.' A is incorrect because, despite using parallelism, the card notes that communication overhead and pipeline bubbles can make it slower for models that fit on a single GPU, so its primary benefit isn't universal speedup.
Read the full bite: Pipeline Parallelism: An Assembly Line for Your Model
Question 22 of 30
What is the primary motivation for employing Tensor Parallelism in large language models?
Show the answer
Answer: c · To process a single, excessively large model layer by distributing its internal components across multiple GPUs.
Tensor Parallelism is specifically designed to enable the execution of individual model layers that are too large to fit into a single GPU's memory by splitting the layer's components. Option A describes pipeline parallelism, which distributes entire layers, not parts of a single layer.
Read the full bite: Tensor Parallelism: Split Layers, Not Just Models
Question 23 of 30
Which scenario most clearly demonstrates the value of prompt engineering?
Show the answer
Answer: b · Generating a consistent series of structured product descriptions for an e-commerce platform.
Prompt engineering is essential for tasks requiring reliable, repeatable, and specific outputs, such as generating consistent content for an application. Casual brainstorming or one-off requests do not typically require the detailed guidance that prompt engineering provides.
Read the full bite: Prompt Engineering: Guiding LLMs to Better Answers
Question 24 of 30
When is zero-shot prompting the most appropriate technique to use?
Show the answer
Answer: d · When performing a simple sentiment analysis on a product review.
Zero-shot prompting is ideal for simple, common tasks like sentiment analysis that leverage the model's pre-existing knowledge without needing examples. Tasks requiring new, specific formatting rules (option A) are generally too novel for zero-shot and would likely require examples.
Question 25 of 30
Which statement accurately describes how in-context learning influences a large language model's behavior?
Show the answer
Answer: d · It temporarily guides the model's output for a specific query by providing examples within the current prompt.
In-context learning temporarily guides the model's behavior for a specific task by providing examples within the prompt, without making any permanent changes to its underlying weights. It does not involve permanent modification of parameters or deep, lasting knowledge acquisition, which would require fine-tuning or retraining.
Read the full bite: In-Context Learning: Teaching a GPT Without Retraining
Question 26 of 30
In which situation should you avoid few-shot prompting?
Show the answer
Answer: d · A clearly described one-sentence task is running near the model's token limit
The card states that few-shot prompting should be avoided when a zero-shot instruction is sufficient and when the context window is nearly full, because examples then add cost and bias without benefit. Distractor D describes a valid use case, as adapting niche style without retraining is exactly where few-shot prompting excels.
Question 27 of 30
For which type of task is Chain-of-Thought (CoT) prompting most beneficial?
Show the answer
Answer: b · Solving complex problems that require multiple logical steps
CoT prompting is designed to improve LLM performance on tasks requiring complex, multi-step reasoning, such as arithmetic word problems or logical puzzles. It is explicitly stated that CoT should be avoided for simple factual recall questions as it adds unnecessary latency and cost without improving accuracy.
Read the full bite: Chain-of-Thought Prompting: Making LLMs 'Show Their Work'
Question 28 of 30
In which situation is Least-to-Most Prompting the most appropriate strategy?
Show the answer
Answer: c · When the problem's complexity far exceeds that of the few-shot examples, and it can be solved incrementally.
The card explicitly states Least-to-Most Prompting is for when "the problem's complexity far exceeds that of your few-shot examples" and can be broken into "a clear sequence of smaller, dependent steps." Option B describes a feature also present in Chain-of-Thought, but not the unique advantage of Least-to-Most for harder problems.
Read the full bite: Least-to-Most Prompting: Solving Hard Problems Incrementally
Question 29 of 30
For complex reasoning tasks, what is the main benefit of using self-consistency over standard greedy decoding in LLMs?
Show the answer
Answer: d · It enables the model to overcome early reasoning errors by exploring multiple solution paths and selecting the most consistent outcome.
Self-consistency's primary benefit is to prevent the model from getting 'locked into a flawed reasoning path early on' by generating and evaluating a diverse set of reasoning paths, then selecting the most frequent answer. Option C is incorrect because self-consistency samples a 'diverse set' of paths, not an exhaustive search, and does not guarantee the 'absolute optimal solution' but rather a more robust one.
Read the full bite: Self-Consistency: Majority Rules for LLM Reasoning
Question 30 of 30
For which type of problem is Tree of Thoughts (ToT) most beneficial compared to a single-path reasoning approach?
Show the answer
Answer: a · Problems where initial decisions are critical and backtracking from errors is necessary.
The card states ToT excels "where initial decisions are pivotal and the ability to backtrack from a dead end is necessary." This directly matches option A. Option C is incorrect because the card explicitly mentions ToT's "massive overhead in computation, cost, and latency."
Read the full bite: Tree of Thoughts: LLM Reasoning Beyond a Single Path
Could you explain these out loud?
That is what an interview actually tests. Tezvyn gives you questions like these with what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.