Top 30 Intermediate LLMs & Generative AI Concepts Quiz
30 intermediate multiple-choice LLMs & Generative AI concept questions, the mechanics underneath the basics: how the pieces relate and where the usual mental model stops holding. They come from 30 bites in the LLMs & Generative AI library, the middle slice of the 168 LLMs & Generative AI concept questions in the library. Answer them here or read straight down. Every question carries the correct option, why it is correct, and a link to the bite it came from.
Large language models, chatbots, agents, prompt engineering
30 questions. Pick an answer, or open “Show the answer” to read it.
Answers are graded in your browser. Nothing is saved, and no XP or streak is earned here. The app keeps score.
Question 1 of 30
Which statement best describes the fundamental way regularization helps a model avoid overfitting?
Show the answer
Answer: b · It encourages the model to learn smaller, less extreme parameter values.
Regularization adds a penalty to the loss function for large parameter values, which encourages the model to learn simpler, less extreme weights, thus preventing it from memorizing noise. Option A describes a beneficial outcome of regularization, but not its direct mechanism; it doesn't explicitly "ignore" data points but rather reduces their influence by constraining parameter magnitudes.
Read the full bite: Regularization: Penalizing Complexity to Prevent Overfitting
Question 2 of 30
What is the fundamental principle behind how Word2Vec represents word meaning?
Show the answer
Answer: c · Words with similar meanings are mapped to points that are spatially close in a multi-dimensional vector space.
The core idea of Word2Vec is that words with similar meanings are represented by vectors that are close to each other in a multi-dimensional space, capturing semantic similarity through spatial proximity. Option A describes a method Word2Vec aims to improve upon, as it fails to capture semantic relationships.
Read the full bite: Word2Vec: Word Meaning as a Point in Space
Question 3 of 30
How does self-attention primarily determine the relevance of other words to a given word in a sequence?
Show the answer
Answer: d · By comparing the current word's Query vector with every other word's Key vector.
Self-attention calculates relevance by measuring the similarity between a word's Query vector and other words' Key vectors. This process allows it to directly identify and weigh the importance of all other words, unlike sequential processing or fixed context windows.
Read the full bite: Self-Attention: The Transformer's Core Idea
Question 4 of 30
What distinct stabilization mechanism does Layer Normalization provide in deep networks, complementing residual connections?
Show the answer
Answer: a · It normalizes feature activations within a layer, maintaining a stable scale for subsequent layers.
Layer Normalization's primary role is to rescale activations within a layer to a stable distribution (mean 0, std 1), which stabilizes the inputs to the next layer. Option C describes the function of residual connections, which provide direct paths for information and gradients.
Read the full bite: Residual Connections & Layer Norm: The Transformer's Stabilizers
Question 5 of 30
What problem does loss scaling primarily address in mixed-precision training?
Show the answer
Answer: d · The potential for small gradient values to become zero in FP16.
Loss scaling's primary purpose is to inflate small gradient values before converting them to FP16, preventing them from becoming zero (underflow), which would otherwise cause training to fail. While a master copy of weights is kept in FP32 for stability, loss scaling directly ensures the gradients themselves are numerically viable.
Read the full bite: Mixed-Precision Training: Faster Training with Less Memory
Question 6 of 30
What is the core mechanism by which AdamW ensures L2 regularization is consistently applied, unlike the original Adam?
Show the answer
Answer: c · It applies the weight decay as a direct subtraction from the weights, independent of the gradient's adaptive scaling.
The card states AdamW applies weight decay by directly subtracting a fraction of the weight's value in a separate step, decoupling it from the adaptive learning rate mechanism. Option A is incorrect because AdamW doesn't just reorder the application within the gradient calculation; it completely separates it and applies it directly to the weights.
Read the full bite: AdamW: Decoupling Weight Decay for Better Generalization
Question 7 of 30
When is a learning rate schedule particularly crucial for achieving state-of-the-art results?
Show the answer
Answer: d · When fine-tuning large, complex models such as transformers.
C is correct because the card states schedules are "standard practice for training and fine-tuning large models, especially transformers... essential for achieving state-of-the-art results." B is a tempting distractor, but the card explicitly warns against using external schedules with optimizers like AdaFactor due to potential conflicts.
Read the full bite: Learning Rate Scheduling: A Gearbox for Model Training
Question 8 of 30
For which type of computational problem is data parallelism most appropriate?
Show the answer
Answer: b · Dividing a large dataset among multiple processors, each running the same program on its assigned portion.
Data parallelism is designed for scenarios where a single, computationally intensive operation needs to be performed on a massive dataset, with each processor handling a different chunk. Option A describes task parallelism, while Option C highlights a situation where data parallelism would be inefficient due to communication overhead.
Read the full bite: Data Parallelism: One Task, Many Data Chunks
Question 9 of 30
Which statement accurately describes how in-context learning influences a large language model's behavior?
Show the answer
Answer: d · It temporarily guides the model's output for a specific query by providing examples within the current prompt.
In-context learning temporarily guides the model's behavior for a specific task by providing examples within the prompt, without making any permanent changes to its underlying weights. It does not involve permanent modification of parameters or deep, lasting knowledge acquisition, which would require fine-tuning or retraining.
Read the full bite: In-Context Learning: Teaching a GPT Without Retraining
Question 10 of 30
In which situation should you avoid few-shot prompting?
Show the answer
Answer: d · A clearly described one-sentence task is running near the model's token limit
The card states that few-shot prompting should be avoided when a zero-shot instruction is sufficient and when the context window is nearly full, because examples then add cost and bias without benefit. Distractor D describes a valid use case, as adapting niche style without retraining is exactly where few-shot prompting excels.
Question 11 of 30
For which type of task is Chain-of-Thought (CoT) prompting most beneficial?
Show the answer
Answer: b · Solving complex problems that require multiple logical steps
CoT prompting is designed to improve LLM performance on tasks requiring complex, multi-step reasoning, such as arithmetic word problems or logical puzzles. It is explicitly stated that CoT should be avoided for simple factual recall questions as it adds unnecessary latency and cost without improving accuracy.
Read the full bite: Chain-of-Thought Prompting: Making LLMs 'Show Their Work'
Question 12 of 30
A developer needs to adapt a large foundation model for a specific task using limited computational resources. Which method is most appropriate?
Show the answer
Answer: a · Parameter-Efficient Fine-Tuning (PEFT)
PEFT is specifically designed to adapt large models for specific tasks on a budget by freezing most of the original model and only training a small number of new parameters. Full fine-tuning (B) is too resource-intensive for limited hardware, and training a new model from scratch (D) is even more demanding and inefficient for adapting an existing foundation model.
Read the full bite: PEFT: Fine-Tune Large Models on a Budget
Question 13 of 30
Which statement accurately describes a key outcome of instruction fine-tuning for a base LLM?
Show the answer
Answer: d · It teaches the model to generate responses that align with user intent and specific task formats.
Instruction fine-tuning is designed to align the model's behavior with user intent, teaching it to follow directions and produce compliant, helpful responses in specific formats. The card explicitly states that it does not inject new factual knowledge; rather, it teaches conversational style and format.
Read the full bite: Instruction Fine-Tuning: Teaching LLMs to Follow Orders
Question 14 of 30
For which task would a Reward Model likely be considered an inefficient or unnecessary approach?
Show the answer
Answer: d · Fine-tuning a code generation model to produce executable and bug-free code.
The card explicitly states that reward modeling should be avoided when a clear, objective, and cheaper metric exists, citing code executability as an example. The other options involve subjective qualities like helpfulness, creativity, or ethics, for which reward models are specifically designed.
Read the full bite: Reward Modeling: Teaching an LLM What 'Good' Means
Question 15 of 30
Which statement best describes how Dense Passage Retrieval (DPR) primarily achieves semantic search for question answering?
Show the answer
Answer: b · It transforms questions and document passages into numerical vectors and identifies relevant passages by vector similarity.
DPR's core mechanism is to convert both questions and document passages into dense numerical vectors using separate encoder models. It then finds semantically relevant passages by identifying the closest vectors in a shared 'meaning space'. Option C is incorrect because DPR is a retrieval system, not a generative one, and it uses two distinct encoders for queries and documents.
Read the full bite: Dense Passage Retrieval (DPR): Semantic Search for QA
Question 16 of 30
Which scenario best illustrates the primary benefit of employing hybrid search in a retrieval system?
Show the answer
Answer: c · A user queries "troubleshooting error 0x80070005 in billing system" and expects results that mention the exact code AND conceptually related solutions.
Hybrid search excels in scenarios like option C, where it combines the precision of keyword matching for specific terms (like an error code) with the conceptual understanding of vector search for broader topics (like troubleshooting solutions). Option B describes a simple exact match where pure keyword search is more efficient, and option D is a pure conceptual search where pure vector search suffices.
Read the full bite: Hybrid Search: Combining Keyword and Vector Search
Question 17 of 30
When is context stuffing the most appropriate method for an LLM to access new information?
Show the answer
Answer: c · When a model needs to answer a one-off question using specific, external documents.
Context stuffing is designed for providing temporary, task-specific information for one-off queries, acting like short-term memory. It is explicitly stated as unsuitable for permanent learning or for data exceeding the context window.
Read the full bite: Context Stuffing: Giving LLMs Short-Term Memory
Question 18 of 30
How does the placement of critical information within a long prompt typically affect an LLM's ability to retrieve it?
Show the answer
Answer: b · Information placed at the beginning or end of the context yields the best retrieval accuracy.
The card states that LLMs exhibit a "U-shaped" performance curve, meaning accuracy is highest when key information is at the beginning or end of the context. Option C is a tempting distractor because it acknowledges a decline, but it incorrectly suggests a linear decrease and misses the improved performance at the very end due to the recency effect.
Read the full bite: LLMs Get 'Lost in the Middle' of Long Contexts
Question 19 of 30
For which type of task is task decomposition most crucial for an LLM agent?
Show the answer
Answer: a · Developing a multi-stage marketing campaign strategy from scratch
Task decomposition is essential for complex, multi-step goals that require planning and state management, such as developing a multi-stage strategy. While summarizing a lengthy paper involves complex input, it is typically a single-turn task for an LLM, not requiring the LLM to decompose the summarization process itself.
Read the full bite: Task Decomposition: Teaching LLMs to Plan
Question 20 of 30
Which of the following best describes a fundamental limitation of current text-to-image models?
Show the answer
Answer: c · They struggle with tasks demanding precise logical consistency or accurate object enumeration.
The card explicitly states that these models 'struggle with rendering legible text, counting objects correctly' and 'don't truly understand prompts,' indicating a limitation in logical consistency. Option D is incorrect because the card clarifies it 'is not a tool for retrieving existing images; it generates new ones.'
Read the full bite: Text-to-Image Synthesis: From Prompt to Picture
Question 21 of 30
Which characteristic most directly indicates a generative model is experiencing mode collapse?
Show the answer
Answer: d · The model produces a narrow range of similar outputs, despite some appearing very convincing.
Mode collapse is characterized by the generator producing a limited set of outputs, even if those specific outputs are high quality, failing to capture the full data distribution. Option A is incorrect because the generator might minimize its loss for the few modes it produces, not necessarily see a continuous increase.
Read the full bite: Mode Collapse: When Your AI Gets Stuck in a Rut
Question 22 of 30
Why is Fréchet Inception Distance (FID) preferred over pixel-by-pixel comparisons for evaluating generative image models?
Show the answer
Answer: c · It captures the high-level semantic features and overall realism of image distributions.
FID is preferred because it captures high-level semantic features and overall realism, which pixel-by-pixel comparisons fail to do effectively. The card explicitly states that FID evaluates a distribution, not single images, meaning it does not guarantee the quality of every individual generated image.
Read the full bite: Fréchet Inception Distance (FID): Grading AI Art
Question 23 of 30
What is the fundamental purpose of a joint embedding space in AI?
Show the answer
Answer: d · To map diverse data types into a common semantic space where conceptual similarity is quantifiable.
A joint embedding space's core purpose is to represent different data types in a shared coordinate system where their semantic similarity can be measured. Option B is incorrect because the card explicitly states it's not for perfect, lossless translation.
Read the full bite: Joint Embedding Space: A Rosetta Stone for AI
Question 24 of 30
For which task would multimodal fusion be most essential for achieving a comprehensive and nuanced understanding?
Show the answer
Answer: d · Answering a question about the content of an image, given both the image and a natural language question.
Multimodal fusion is critical for tasks like Visual Question Answering (VQA), where understanding requires combining information from distinct modalities like images and text. The other options are primarily unimodal tasks that can be effectively addressed using a single data type.
Read the full bite: Multimodal Fusion: Combining Senses for AI
Question 25 of 30
What fundamental capability distinguishes a Large Multimodal Model (LMM) from combining separate, specialized AI models for text, image, and audio?
Show the answer
Answer: d · LMMs learn a shared conceptual understanding that links information across different data types.
The card states an LMM is a 'single neural network trained to understand the relationships between different types of data' and 'learns a shared conceptual space' to reason across modalities. Option B is incorrect because the card explicitly states that for purely text-based tasks, a standard LLM is often more efficient and sufficient.
Read the full bite: Large Multimodal Models (LMMs): Beyond Text
Question 26 of 30
What is the primary advantage of converting raw audio into a spectrogram for AI models?
Show the answer
Answer: c · It transforms audio into a visual format that explicitly displays frequency content over time, making it suitable for image-processing AI.
The core purpose of a spectrogram is to convert the rich, hidden frequency information of audio into a visual, time-frequency representation, which is ideal for AI models that excel at image processing, such as CNNs. Option D describes the primary information found in a raw audio waveform, not a spectrogram, and the card states that raw waveforms might be better when precise timing of amplitude peaks is crucial.
Read the full bite: Audio Spectrograms: Turning Sound into Images for AI
Question 27 of 30
Which of the following best describes the fundamental operation of a Text-to-Speech (TTS) system?
Show the answer
Answer: b · Converting written language into an audible, artificially generated voice.
A Text-to-Speech (TTS) system's core function is to take written text and transform it into artificial human speech, as described in option B. Option C describes speech recognition, which the card explicitly states is the reverse process and not what TTS does.
Read the full bite: Text-to-Speech (TTS): Turning Text into Spoken Audio
Question 28 of 30
A generative model creates individually clear images, but all belong to one narrow class. How would its Inception Score likely be affected?
Show the answer
Answer: c · The Inception Score would be low due to insufficient diversity, despite high individual image quality.
The Inception Score evaluates both the quality (distinctness) and diversity of generated images. While the model excels in quality, its lack of diversity by generating images from only one narrow class would significantly lower its overall score, as explicitly stated in the card's canonical example. Options focusing solely on individual image quality miss the critical diversity component.
Read the full bite: Inception Score: Judging AI Art for Quality and Variety
Question 29 of 30
What is the key characteristic of the incorrect answer options in the HellaSwag benchmark that makes it challenging for language models?
Show the answer
Answer: a · They are engineered to appear statistically probable to models while being nonsensical to human understanding.
The card explains that HellaSwag's wrong answers are "specifically generated to trick machines" and are "plausible-sounding nonsense that LLMs tend to generate," which are "ridiculous to humans but often misclassified." This directly describes option A. Option B, while a general LLM challenge, does not capture the unique adversarial design of HellaSwag's distractors, which focuses on commonsense meaning rather than just syntax.
Read the full bite: HellaSwag: A Benchmark Designed to Fool LLMs
Question 30 of 30
Why is LLM Red Teaming considered essential for AI systems, beyond traditional security practices?
Show the answer
Answer: d · It specifically targets and identifies novel, model-specific vulnerabilities that traditional tools miss.
The card states that LLM Red Teaming was created to address "unique, model-specific vulnerabilities" and "new classes of risks that standard security tools were not designed to find." Distractor B focuses on general functionality and accuracy, which is not the primary security-focused purpose of red teaming.
Read the full bite: LLM Red Teaming: Adversarial Security Testing
Could you explain these out loud?
That is what an interview actually tests. Tezvyn gives you questions like these with what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.