Top 30 Intermediate LLMs & Generative AI Interview Questions and Answers

30 questions. Pick an answer, or open “Show the answer” to read it.

Answers are graded in your browser. Nothing is saved, and no XP or streak is earned here. The app keeps score.

1. Question 1 of 30

   When training a large transformer, training loss falls while validation loss rises. Which action most directly addresses the root cause?

   • aTrain for more steps to allow validation loss to converge
   • bApply early stopping or add regularization like weight decay
   • cIncrease model capacity to better fit the training data
   • dLower the learning rate to stabilize the optimization landscape
   Show the answer

   Answer: b · Apply early stopping or add regularization like weight decay

   The divergence signals overfitting, where the model memorizes training noise instead of generalizable patterns. Early stopping and regularization directly combat this, whereas longer training or more parameters typically worsens the gap, and a lower learning rate does not directly fix memorization.

   Read the full bite: Validation loss increases while training loss decreases: what is this?

2. Question 2 of 30

   In full BPTT, what is the primary reason memory usage grows linearly with sequence length T?

   • aThe recurrent weights are duplicated T times across the unrolled computational graph.
   • bThe backward pass requires storing T separate Jacobian matrices, one per time step.
   • cAll intermediate hidden states from the forward pass must be kept for the backward pass.
   • dOnly the current hidden state is needed, so memory stays constant and linear growth is a myth.
   Show the answer

   Answer: c · All intermediate hidden states from the forward pass must be kept for the backward pass.

   During BPTT the backward pass flows through every time step, so every forward hidden state must be retained, yielding O(T) memory. Option D repeats the common misconception that training memory is constant like inference memory, while option A confuses unrolling with parameter duplication.

   Read the full bite: Explain BPTT and its computational and memory challenges for long sequences

3. Question 3 of 30

   What core limitation of a vanilla RNN encoder-decoder does the attention mechanism specifically address?

   • aCompressing the entire input into one fixed-size vector loses detail on long sequences
   • bThe lack of positional information about token order
   • cThe inability to process input tokens in parallel during training
   • dThe vanishing gradient within each individual recurrent cell
   Show the answer

   Answer: a · Compressing the entire input into one fixed-size vector loses detail on long sequences

   Attention removes the single fixed-context-vector bottleneck by letting the decoder weight all encoder states per step. Parallelism and positional encodings are Transformer concerns, not what classic seq2seq attention was introduced to fix.

   Read the full bite: Attention in Sequence-to-Sequence Models

4. Question 4 of 30

   Why does self-attention use three separate learned projections of the same input rather than the raw embeddings directly?

   • aIt allows the model to learn which token features to use for matching versus which to pass forward as content
   • bBecause the weight matrices are predefined linguistic features that do not require training
   • cBecause the dot product of raw embeddings would be too large and cause softmax saturation
   • dSo that the query can come from the decoder while key and value come from the encoder
   Show the answer

   Answer: a · It allows the model to learn which token features to use for matching versus which to pass forward as content

   The correct answer reflects that learned projections decouple the matching process from content retrieval, letting the model decide which aspects of a token to use for scoring and which to propagate forward. The distractor describing a decoder query with encoder key and value defines cross-attention, not self-attention, where all three matrices are derived from the same input sequence.

   Read the full bite: Explain Q, K, and V matrices in self-attention

5. Question 5 of 30

   Why does multi-head attention generally not cost much more than a single full-width attention head?

   • aSoftmax is skipped in all but one head to save computation
   • bExtra heads share a single query-key-value projection matrix
   • cThe model dimension is split across heads, so each head operates in a smaller subspace
   • dHeads are computed sequentially, reusing the same weights each time
   Show the answer

   Answer: c · The model dimension is split across heads, so each head operates in a smaller subspace

   The total dimension is partitioned among heads, keeping aggregate compute comparable to one full head while gaining diverse attention patterns. Heads run in parallel with their own projections, not shared weights, and each still uses softmax.

   Read the full bite: Why Multi-Head Attention

6. Question 6 of 30

   During parallel teacher-forced training, what problem does the causal mask in decoder self-attention solve?

   • aIt enables bidirectional context modeling over the full target sequence
   • bIt blocks the source sequence from being accessed before cross-attention
   • cIt stops position i from attending to future target tokens, preventing the model from copying answers
   • dIt hides padding tokens so batched sequences remain the same length
   Show the answer

   Answer: c · It stops position i from attending to future target tokens, preventing the model from copying answers

   The causal mask forces token i to attend only to prior positions, preserving the autoregressive property and stopping the model from cheating by looking at future target tokens during parallel training. Option D describes the padding mask, which is a common misconception.

   Read the full bite: What is masked in decoder self-attention and why?

7. Question 7 of 30

   Under Chinchilla's compute-optimal scaling laws, how should a fixed FLOP budget be divided between model parameters and training tokens?

   • aUse the largest possible model and dataset independently, ignoring the fixed compute constraint.
   • bScale model parameters and training tokens equally, resulting in smaller models trained on much more data.
   • cKeep model size fixed and increase training tokens as much as the budget allows.
   • dMaximize parameters while keeping training tokens relatively constant, as in GPT-3 and Gopher.
   Show the answer

   Answer: b · Scale model parameters and training tokens equally, resulting in smaller models trained on much more data.

   Chinchilla showed that for a fixed compute budget, parameters and training tokens should be scaled equally, meaning smaller models must be trained on proportionally more data to be compute-optimal. Option D describes the pre-Chinchilla approach that produced undertrained models, while option C incorrectly suggests holding model size fixed rather than scaling both dimensions together.

   Read the full bite: How did Chinchilla change compute allocation between model size and data size?

8. Question 8 of 30

   In a canonical 3D hybrid strategy for a massive model, why is tensor parallelism placed within a node while pipeline parallelism spans across nodes?

   • aPipeline parallelism cannot utilize NVLink bandwidth, so it is relegated to slower inter-node connections.
   • bData parallelism must always occupy the largest cluster dimension, forcing tensor and pipeline into smaller subgroups.
   • cTensor parallelism requires frequent all-reduce on activations that saturates high-latency inter-node links, whereas pipeline parallelism uses coarser point-to-point communication.
   • dTensor parallelism reduces the total number of parameters per device, making it unnecessary to scale beyond a single node.
   Show the answer

   Answer: c · Tensor parallelism requires frequent all-reduce on activations that saturates high-latency inter-node links, whereas pipeline parallelism uses coarser point-to-point communication.

   Tensor parallelism performs all-reduce on activations every layer, so keeping it inside a node avoids saturating slower inter-node bandwidth, while pipeline stages only exchange activations via point-to-point between passes. Option A is tempting because it mentions NVLink, but pipeline parallelism can also use fast links; the placement is driven by tensor parallelism's intense communication pattern, not pipeline's inability to use NVLink.

   Read the full bite: Explain data, tensor, and pipeline parallelism and hybrid training strategy

9. Question 9 of 30

   In a long pre-training run, the loss suddenly spikes. Which action should you take FIRST before applying any mitigation?

   • aRoll back to the last stable checkpoint to prevent the corrupted state from persisting
   • bSwitch from FP16 to BF16 to eliminate mixed-precision overflow as the cause
   • cCheck the gradient norm logs to confirm whether a gradient explosion occurred
   • dLower the learning rate and enable gradient clipping to stabilize the optimization
   Show the answer

   Answer: c · Check the gradient norm logs to confirm whether a gradient explosion occurred

   The card emphasizes checking gradient norm logs first to confirm an explosion before deciding on recovery actions like rollback or hyperparameter changes. Option A is tempting because rolling back is a crucial recovery step, but doing so before confirming the root cause is premature and skips the diagnostic phase.

   Read the full bite: What causes sudden loss spikes in long pre-training runs?

10. Question 10 of 30

    When adapting a foundation model for high-stakes scientific reasoning, which approach best balances capability gains with bias control according to the described best practices?

    • aMaximize broad web crawl data to increase knowledge coverage, then filter for scientific keywords at inference to control bias
    • bDump a large volume of scientific papers into the pre-training pool, since domain-specific scale consistently outperforms smaller curated datasets
    • cPrioritize code-heavy pre-training data to boost algorithmic reasoning, then add raw scientific papers without quality scoring for domain coverage
    • dApply domain-adaptive pretraining on curated scientific literature, followed by instruction tuning and reasoning-based distillation, validating each component with ablations
    Show the answer

    Answer: d · Apply domain-adaptive pretraining on curated scientific literature, followed by instruction tuning and reasoning-based distillation, validating each component with ablations

    The correct approach combines curated domain-adaptive pretraining, specialized instruction tuning, and reasoning distillation with ablation validation. The code-heavy distractor is wrong because although code strengthens general reasoning, scientific specialization requires curated literature and quality scoring, not raw unfiltered papers.

    Read the full bite: How does pre-training dataset composition influence capabilities and biases?

11. Question 11 of 30

    Which approach to selecting and ordering demonstrations is most likely to improve text-to-SQL execution accuracy?

    • aRetrieve candidates by SQL syntax similarity, balance pattern diversity across joins and aggregations, order simple to complex, and anchor with schema context.
    • bSelect many random examples to fill the context window, since increasing the number of demonstrations always improves structured generation accuracy.
    • cUse only simple single-table SELECT examples to maintain a consistent pattern, exclude schema details to minimize tokens, and list them in random order.
    • dChoose examples by matching natural language embedding similarity to the input question, ordering them from the most complex SQL patterns to the simplest.
    Show the answer

    Answer: a · Retrieve candidates by SQL syntax similarity, balance pattern diversity across joins and aggregations, order simple to complex, and anchor with schema context.

    Effective text-to-SQL prompting requires retrieving examples by SQL syntax, ensuring diverse query patterns, scaffolding from simple to complex, and grounding each example in schema context. The most tempting distractor relies on natural language embedding similarity, which often retrieves semantically close but syntactically irrelevant demonstrations, and front-loading complex SQL overwhelms the model rather than scaffolding it.

    Read the full bite: How do you select in-context examples for text-to-SQL prompts?

12. Question 12 of 30

    An engineer needs to improve an LLM's structured JSON extraction rate using only prompt changes. Which approach should they prioritize?

    • aRetry failed outputs with Python's json.loads and regex repair in a loop
    • bSwitch to a larger model and enable the API's native JSON mode
    • cTell the model to output JSON and use chain-of-thought reasoning at temperature 0.7
    • dInclude the exact JSON schema with empty values and add one to three valid input-output examples
    Show the answer

    Answer: d · Include the exact JSON schema with empty values and add one to three valid input-output examples

    Embedding an exact JSON skeleton and providing few-shot exemplars are the two prompt-based techniques that constrain the output distribution by making the desired token sequence highly probable. Chain-of-thought reasoning is a tempting distractor because it often increases verbosity and introduces stray tokens that break JSON validity.

    Read the full bite: Describe two prompt-based techniques to ensure valid LLM JSON output

13. Question 13 of 30

    For a multi-step reasoning task, which factor most increases chain-of-thought prompting's cost and latency relative to zero-shot?

    • aThe larger number of input tokens in the system prompt
    • bThe requirement to call an external tool for every reasoning step
    • cThe need to fine-tune the model before each chain-of-thought query
    • dThe many additional output tokens generated for the reasoning steps
    Show the answer

    Answer: d · The many additional output tokens generated for the reasoning steps

    Chain-of-thought generates extra output tokens for intermediate reasoning, and sequential decoding makes output length the dominant cost and latency driver. It needs no fine-tuning or external tool calls per step.

    Read the full bite: Zero-Shot, Few-Shot, and Chain-of-Thought Trade-offs

14. Question 14 of 30

    When adapting a large model to dozens of tasks on a tight compute budget, why is LoRA strongly preferred over full fine-tuning?

    • aIt trains only a small number of extra parameters per task, keeping one shared base model and sharply reducing compute, storage, and serving costs while matching performance.
    • bFull fine-tuning always outperforms LoRA, so the increased cost is justified when adapting to many tasks.
    • cLoRA primarily lowers training FLOPs but has the same storage and inference requirements as full fine-tuning because each task still needs its own complete checkpoint.
    • dYou can average full fine-tuned weights across tasks to avoid storing multiple checkpoints, making full fine-tuning as storage-efficient as LoRA.
    Show the answer

    Answer: a · It trains only a small number of extra parameters per task, keeping one shared base model and sharply reducing compute, storage, and serving costs while matching performance.

    LoRA adds lightweight adapters per task around a single frozen base model, drastically cutting compute, storage, and inference costs while maintaining comparable performance. Distractor B is wrong because the card explicitly flags the belief that full fine-tuning always outperforms PEFT as a major red flag, and storing separate full checkpoints for many tasks is prohibitively expensive.

    Read the full bite: Full fine-tuning or LoRA on a tight compute budget?

15. Question 15 of 30

    Why does LoRA reduce GPU memory usage during fine-tuning compared to full fine-tuning?

    • aIt quantizes the frozen base weights to lower precision, shrinking the model footprint.
    • bIt freezes the original weight matrix W and trains only low-rank matrices A and B, so gradients and optimizer states are only stored for A and B.
    • cIt reduces the rank of the pretrained weight matrix W itself, lowering its memory footprint.
    • dIt merges the low-rank update into W during each training step, reducing active parameters.
    Show the answer

    Answer: b · It freezes the original weight matrix W and trains only low-rank matrices A and B, so gradients and optimizer states are only stored for A and B.

    LoRA keeps the original weight matrix W frozen and only trains the small low-rank matrices A and B, which means gradients and optimizer states like those in Adam are only maintained for A and B, drastically reducing memory usage. The most tempting distractor claims the update is merged during training, but merging BA into W actually happens after training to preserve inference speed, not to save memory during fine-tuning.

    Read the full bite: How does LoRA work and why is it memory-efficient?

16. Question 16 of 30

    In the RLHF pipeline, why is a separate reward model trained on human preference comparisons before the final RL stage?

    • aTo synthesize diverse text completions that replace the base model's outputs during RL initialization
    • bTo eliminate the need for RL by enabling supervised fine-tuning directly on human-written answers
    • cTo serve as the final aligned model that is deployed to users after training
    • dTo generalize discrete human comparisons into a continuous scalar score that drives automated policy optimization
    Show the answer

    Answer: d · To generalize discrete human comparisons into a continuous scalar score that drives automated policy optimization

    The reward model captures human preference numerically as a scalar so that automated RL optimization can occur. Option C is a common misconception: the reward model is not the final deployed policy; stage three produces the aligned language model that users interact with.

    Read the full bite: Walk through RLHF's three stages, outputs, and purposes.

17. Question 17 of 30

    During PPO-based RLHF, why is a KL-divergence penalty against the reference model added to the reward signal?

    • aTo eliminate the need for any human preference data
    • bTo merge the reward model and policy into a single network
    • cTo prevent the policy from exploiting the reward model and degenerating into reward-hacking text
    • dTo let humans relabel every generated sample inside the loop
    Show the answer

    Answer: c · To prevent the policy from exploiting the reward model and degenerating into reward-hacking text

    The KL term keeps the policy close to the trusted reference so it cannot drift into degenerate outputs that game the reward model. It has nothing to do with relabeling samples or removing preference data.

    Read the full bite: Reward models in RLHF and PPO

18. Question 18 of 30

    When an LLM regresses on legacy tasks after domain-specific fine-tuning, which approach best balances new task mastery with preserving existing capabilities?

    • aFreeze the first few transformer layers and fully fine-tune all remaining parameters
    • bLower the learning rate so updates are too small to overwrite prior knowledge
    • cAdopt LoRA with a regularization term that penalizes deviation from base weights while evaluating on holdout legacy tasks
    • dTreat the regression as overfitting and expand the new domain training set
    Show the answer

    Answer: c · Adopt LoRA with a regularization term that penalizes deviation from base weights while evaluating on holdout legacy tasks

    LoRA restricts the update space, regularization prevents drift from base capabilities, and legacy evaluation detects regression early. Simply lowering the learning rate is a common misconception because gradients can still overwrite critical knowledge even at small step sizes.

    Read the full bite: What is catastrophic forgetting in LLMs and how do you mitigate it?

19. Question 19 of 30

    A RAG system contradicts retrieved evidence. Debugging reveals the correct document was retrieved but ranked below irrelevant chunks. What is the most targeted immediate fix?

    • aLabel the issue as model hallucination and add a system prompt instructing the model to trust retrieved documents
    • bLower the generation temperature to zero to force adherence to the provided context
    • cAdd a cross-encoder reranker to boost the relevant document and increase prompt emphasis on the top-ranked excerpt
    • dSwap the vector database for a more advanced alternative to improve semantic search
    Show the answer

    Answer: c · Add a cross-encoder reranker to boost the relevant document and increase prompt emphasis on the top-ranked excerpt

    The correct fix targets the diagnosed ranking failure by reordering chunks and reinforcing prompt priority, which directly resolves the contradiction. Blaming hallucination ignores the explicit retrieval trace evidence, which is the biggest debugging red flag, and merely adding a trust-context instruction does not fix the ranking inversion.

    Read the full bite: Why does your RAG ignore or contradict retrieved context?

20. Question 20 of 30

    A RAG answer is fluent but contradicts the passage that was correctly retrieved for it. Which metric most directly flags this problem?

    • aContext precision, since too much irrelevant text was retrieved
    • bFaithfulness, since the answer is not grounded in the retrieved context
    • cContext recall, since the correct passage was missing
    • dBLEU against a reference, since wording differs
    Show the answer

    Answer: b · Faithfulness, since the answer is not grounded in the retrieved context

    Faithfulness measures whether the generated answer is supported by retrieved context, so it catches a grounded-but-ignored hallucination. Recall and precision concern retrieval, which here worked correctly.

    Read the full bite: Evaluating a RAG system end to end

21. Question 21 of 30

    Which approach best handles a user question requiring both a precise database aggregation and a qualitative summary from documents in a hybrid RAG system?

    • aRetrieve text chunks first and prompt the LLM to infer the numeric aggregation from the unstructured text instead of using SQL
    • bUse one LLM prompt that simultaneously writes the SQL query and retrieves relevant text chunks without intermediate result stages
    • cUse a query planner to route the aggregation to native SQL execution and the summary request to vector search, then synthesize both outputs
    • dEmbed all database tables as text chunks and perform a single vector search across both tables and documents
    Show the answer

    Answer: c · Use a query planner to route the aggregation to native SQL execution and the summary request to vector search, then synthesize both outputs

    Native SQL preserves exact numeric aggregations and filter semantics that vector search or text inference cannot reliably replicate, while synthesis combines both sources accurately. Option D is the most tempting distractor because it mirrors standard text-only RAG, but flattening tables into chunks destroys relational operations like joins and SUMs.

    Read the full bite: How would you modify retrieval architecture for hybrid text and SQL RAG?

22. Question 22 of 30

    How does a ReAct agent handle a tool call that requires data returned by a previous tool?

    • aIt interleaves reasoning, action, and observation in a loop, appending each step to a shared trajectory.
    • bIt batches all dependent tool calls together in a single parallel request to minimize latency.
    • cIt pre-computes a static DAG of all tool calls before execution begins.
    • dIt stores observations in external variables and prompts the LLM once with a final summary.
    Show the answer

    Answer: a · It interleaves reasoning, action, and observation in a loop, appending each step to a shared trajectory.

    ReAct agents rely on an iterative thought-action-observation loop with an append-only trajectory so the LLM can dynamically replan after each live observation. A static DAG fails because it locks in execution order before any results are known, preventing adaptation to unexpected outputs.

    Read the full bite: Describe a ReAct agent architecture for multi-step dependent tool calls

23. Question 23 of 30

    Which approach should you choose when prompt engineering alone fails to produce valid JSON arguments for a complex tool schema?

    • aCombine few-shot CoT prompting, JSON Schema validation, constrained decoding, and retry loops with error feedback
    • bFine-tune the model on a small dataset of valid calls and disable all sampling randomness
    • cPrompt it to double-check its work, set temperature to zero, and extract JSON with regex
    • dUse the model's native function-calling mode and omit backend validation to keep latency low
    Show the answer

    Answer: a · Combine few-shot CoT prompting, JSON Schema validation, constrained decoding, and retry loops with error feedback

    The correct answer layers user-facing prompts with backend guardrails—schema validation, constrained decoding, and retry loops with error feedback—to provide hard guarantees that prompt engineering alone cannot. Option C is a tempting distractor because simply asking the model to be more careful and using regex extraction offers no assurance of syntactically correct or schema-adherent JSON for nested objects.

    Read the full bite: What fixes an LLM agent's incorrect JSON arguments for a complex tool?

24. Question 24 of 30

    Which description best captures mode collapse and a GAN-specific mitigation for it?

    • aMode collapse is the generator cycling between single modes without covering the full distribution, which batch normalization corrects by stabilizing activations.
    • bMode collapse is the generator producing a narrow subset of the true data distribution, which mini-batch discrimination mitigates by penalizing batch homogeneity.
    • cMode collapse is the generator overfitting to high-frequency details caused by limited data, which is best fixed by data augmentation.
    • dMode collapse is the generator memorizing specific training examples, which can be prevented by adding dropout to the generator.
    Show the answer

    Answer: b · Mode collapse is the generator producing a narrow subset of the true data distribution, which mini-batch discrimination mitigates by penalizing batch homogeneity.

    Mode collapse is structurally a loss of diversity to a few modes, not memorization or overfitting; mini-batch discrimination specifically forces the discriminator to evaluate sample diversity across an entire batch. Option A is tempting because oscillatory cycling is a real related dynamic, but batch normalization is a generic stabilization technique and does not directly penalize homogeneity the way batch-level evaluation does.

    Read the full bite: What is GAN mode collapse, its causes, and two mitigations?

25. Question 25 of 30

    When denoising in Stable Diffusion, how do text embeddings primarily influence the U-Net's intermediate feature maps?

    • aText and image tokens attend to each other through shared self-attention in the U-Net's bottleneck
    • bImage features act as Query while text embeddings provide Key and Value in cross-attention layers throughout the U-Net
    • cThe text encoder is fine-tuned to output embeddings that replace the initial latent noise input
    • dText vectors are concatenated to the latent noise before entering the first U-Net block
    Show the answer

    Answer: b · Image features act as Query while text embeddings provide Key and Value in cross-attention layers throughout the U-Net

    Text embeddings condition the U-Net through cross-attention layers where image-derived Queries attend to text-derived Keys and Values at multiple resolutions, not just at the input or bottleneck. Option D reflects the common misconception that text is concatenated to the latent noise, while option A incorrectly describes the mechanism as self-attention.

    Read the full bite: How does text guide Stable Diffusion via U-Net cross-attention?

26. Question 26 of 30

    A model suffers mode collapse, producing a few very sharp images. Why might Inception Score look acceptable while FID exposes the problem?

    • aHigher FID is better, so collapse improves the score
    • bIS compares to the real dataset, but FID does not, hiding collapse
    • cBoth metrics ignore diversity entirely
    • dIS rewards confident per-image labels and never compares to real data, while FID measures distance to the real distribution
    Show the answer

    Answer: d · IS rewards confident per-image labels and never compares to real data, while FID measures distance to the real distribution

    IS can be high for confidently classified images without referencing real data, so collapse slips through, whereas FID measures distance to the real distribution and rises when generated samples lack spread. Lower FID is better, not higher.

    Read the full bite: Evaluating image generation: FID and IS

27. Question 27 of 30

    In a VQA system, why is a cross-attention fusion stage more effective than simply concatenating the final image and question vectors?

    • aIt lets question words attend to relevant image regions, grounding the answer in specific visual content
    • bConcatenation cannot be implemented in transformers
    • cFusion is only needed when the answer set is small
    • dIt removes the need for any image encoder
    Show the answer

    Answer: a · It lets question words attend to relevant image regions, grounding the answer in specific visual content

    Cross-attention grounds individual words in the relevant image regions, which late concatenation of single vectors cannot do because it discards spatial detail. Both an image encoder and fusion remain necessary.

    Read the full bite: Designing a Visual Question Answering system

28. Question 28 of 30

    In Stable Diffusion, what does the U-Net operate on during each inference step, and how does the prompt influence it?

    • aA classifier score that selects the best of several finished images
    • bA compressed latent, predicting noise to remove while cross-attending to the text embeddings
    • cThe full-resolution pixel image, with the prompt appended as pixels
    • dThe scheduler's output directly, ignoring the text encoder
    Show the answer

    Answer: b · A compressed latent, predicting noise to remove while cross-attending to the text embeddings

    The U-Net denoises in VAE latent space and conditions on text embeddings via cross-attention, which is the efficiency core of latent diffusion. It never works on raw pixels, and decoding to pixels is the VAE's job.

    Read the full bite: How Stable Diffusion generates images

29. Question 29 of 30

    Why can't you simply take a separately pretrained image encoder and text encoder and compare their embeddings directly?

    • aImage and text embeddings always have different dimensions that cannot be compared
    • bCosine similarity is undefined for text embeddings
    • cTheir latent spaces are unaligned, so embeddings are not comparable without a learned alignment objective
    • dPretrained encoders never output embeddings, only logits
    Show the answer

    Answer: c · Their latent spaces are unaligned, so embeddings are not comparable without a learned alignment objective

    Independently trained encoders learn separate, non-comparable spaces, so alignment must be learned via contrastive training or projection layers. The barrier is unaligned geometry, not dimensionality or undefined similarity.

    Read the full bite: Aligning text and image representations

30. Question 30 of 30

    What capability does FID have that the Inception Score fundamentally lacks?

    • aFID requires no pretrained network whereas IS depends on Inception v3
    • bFID measures only per-image sharpness while IS measures diversity
    • cFID is computed without any covariance information
    • dFID compares generated features against the real data distribution, while IS uses only generated samples
    Show the answer

    Answer: d · FID compares generated features against the real data distribution, while IS uses only generated samples

    FID's defining feature is referencing real-image statistics; IS never sees real data. The covariance term in FID is precisely what encodes diversity, so the option dropping it is wrong.

    Read the full bite: How FID is calculated versus Inception Score