Interview questions in LLMs & Generative AI, page 4
LLaVA versus Flamingo vision-LLM design
LLaVA projects image features into the LLM input space and feeds them as tokens, keeping the LLM mostly intact; Flamingo inserts gated cross-attention layers inside a frozen LLM.
Batching strategy for multimodal training
Control dataset mixing ratios, use balanced sampling and per-source weighting, keep enough text-only data to avoid forgetting, and handle variable shapes via grouping or padding.
Perplexity versus BLEU for LMs
Perplexity measures intrinsic next-token prediction quality needing no references; BLEU measures n-gram overlap with reference outputs for tasks like translation.
Why human evaluation is the gold standard
Humans judge fluency, helpfulness, and correctness that n-gram or distribution metrics miss; automated scores correlate weakly with quality, are gameable, and penalize valid diverse outputs.
Standard metric for image generation quality
Name FID, explain it compares feature distributions of real and generated images via a pretrained network.
How FID is calculated versus Inception Score
FID fits Gaussians to Inception features of real and fake images then measures Frechet distance; it uses real references and detects mode collapse.
How to evaluate a RAG system end to end
Measure retrieval (recall, precision, MRR, NDCG) and generation (faithfulness, answer relevance) separately, plus end-to-end correctness.
How MMLU works and the contamination problem
MMLU is multiple-choice across 57 subjects scored by accuracy; contamination means test items leaked into pretraining, inflating scores.
Reference-free evaluation for open-ended dialogue
ROUGE punishes valid paraphrases; use reference-free LLM-as-judge or learned scorers rating coherence, relevance, and groundedness.
Setting up an LLM-as-a-judge evaluation
Define rubric, prefer pairwise comparison, randomize order, calibrate against humans; control position, verbosity, and self-preference bias.
What RLHF is and the safety problem it solves
Collect human preference rankings, train a reward model, fine-tune the policy with PPO; it aligns outputs with human intent the loss function cannot specify.
Red teaming LLMs and harmful output categories
Red teaming deliberately probes a model to elicit harmful behavior; categories include toxic or hateful content and dangerous instructions like weapons or malware.
Designing input and output guardrails for a chatbot
Input guardrails filter or classify user prompts (injection, off-topic, PII) before the model; output guardrails validate responses for toxicity, leakage, and policy before sending.
Prompt injection versus jailbreak, and defenses
Injection hijacks the model via untrusted data overriding developer instructions; jailbreak coaxes a model past its safety policy. Defense: separate trusted instructions from untrusted data and filter.
Model quantization benefits and risks
Quantization stores weights and activations in lower precision like INT8 or INT4; benefits are smaller memory and faster, cheaper inference; risk is accuracy loss.
How the KV cache speeds transformer generation
Cache stores past keys and values so each new token only computes its own K, Q, V instead of recomputing all prior tokens, cutting per-step cost from quadratic to linear.
Teacher-student knowledge distillation
A small student learns to mimic a large teacher's soft probability outputs, not just hard labels; goal is a compact model retaining most capability.
PTQ versus QAT for INT8 quantization
PTQ quantizes a trained model with light calibration, fast and cheap but more accuracy loss; QAT simulates quantization during training, higher accuracy but costly.
What memory problem PagedAttention solves
Pre-allocating contiguous max-length cache per sequence wastes memory through internal and external fragmentation; PagedAttention stores KV in fixed non-contiguous blocks like OS paging.
Tensor versus pipeline parallelism for large models
Tensor parallelism splits individual layers across GPUs needing fast interconnect; pipeline parallelism splits layers into stages across GPUs.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles