Intermediate interview questions in LLMs & Generative AI
Validation loss increases while training loss decreases: what is this?
This tests recognition of overfitting and regularization. A strong answer names it, offers early stopping, dropout or weight decay, and data augmentation or more data. A red flag is suggesting longer training or more parameters without fixing generalization.
Explain BPTT and its computational and memory challenges for long sequences
Tests whether you see RNNs as deep unrolled graphs. Good answers define BPTT as backprop over T steps, flag O(T) memory from hidden states, and note vanishing or exploding gradients. Red flag: calling memory constant or confusing BPTT with online updates.
Attention in Sequence-to-Sequence Models
Attention computes per-step weighted sums over all encoder states, fixing the information bottleneck for long inputs.
Explain Q, K, and V matrices in self-attention
This tests the information-retrieval intuition behind self-attention. Cover that Q, K, V are linear projections of one input; Q requests, K indexes, V supplies content; scores weight a sum of V.
Why Multi-Head Attention
Multiple heads attend to different subspaces and relations in parallel, which one big head averages away.
What is masked in decoder self-attention and why?
Future positions are masked so token i attends only to prior tokens. This prevents cheating during parallel teacher-forced training when the full target is visible.
How did Chinchilla change compute allocation between model size and data size?
This tests whether you know prior scaling fixed data while growing parameters, but Chinchilla showed parameters and tokens must scale equally. A good answer: double both together, so train smaller models on more data. Red flag: huge models, fixed data.
Explain data, tensor, and pipeline parallelism and hybrid training strategy
Tests communication and memory tradeoffs of core distributed training strategies. Strong answers contrast data parallelism (shard batch), tensor parallelism (shard layers, all-reduce), and pipeline parallelism (shard stages, p2p), then propose a 3D hybrid…
What causes sudden loss spikes in long pre-training runs?
Name gradient explosions, LR mismatch, FP16 overflow, and poison batches; propose norm checks, rollback, and LR cuts.
How does pre-training dataset composition influence capabilities and biases?
This probes whether you link data mix to capabilities and bias. Answer: code strengthens reasoning, web text adds noise; for science, use domain-adaptive pretraining on filtered literature, instruction tuning, and reasoning distillation, validating via…
How do you select in-context examples for text-to-SQL prompts?
What it tests: practical ICL design for structured generation. Answer outline: select examples by SQL syntax similarity plus pattern diversity, order from simple to complex, and anchor schema context. Red flag: claiming random examples work fine.
Describe two prompt-based techniques to ensure valid LLM JSON output
This tests output constriction via prompt design. First, embed an exact JSON skeleton with empty values. Second, provide few-shot exemplars mapping inputs to valid JSON. A red flag is suggesting only post-hoc regex repair or larger models.
Zero-Shot, Few-Shot, and Chain-of-Thought Trade-offs
Zero-shot is cheap but weak on reasoning, few-shot adds demos at token cost, CoT boosts multi-step accuracy but spends the most tokens and latency.
Full fine-tuning or LoRA on a tight compute budget?
This tests budget-constrained adaptation for many tasks. A strong answer picks LoRA: it trains only a small number of extra parameters, cutting compute and storage versus full fine-tuning while matching performance.
How does LoRA work and why is it memory-efficient?
LoRA freezes weights and trains A and B so delta-W equals BA, cutting trainable params 10,000x and memory 3x since only A and B get grads.
Walk through RLHF's three stages, outputs, and purposes.
Tests your grasp of the RLHF pipeline end-to-end. A strong answer lists: pretrain an instruction-following LM, train a reward model outputting a scalar preference score, then fine-tune the LM via RL.
Reward models in RLHF and PPO
It learns from human preference comparisons to score responses, then supplies the reward signal that PPO maximizes while a KL penalty keeps the policy near the reference.
What is catastrophic forgetting in LLMs and how do you mitigate it?
This tests stability-plasticity trade-offs in fine-tuning. A strong answer defines catastrophic forgetting as lost prior capabilities, cites LoRA, regularization, and continual learning.
Why does your RAG ignore or contradict retrieved context?
Tests separation of retrieval failures from generation grounding in RAG. Strong answers trace symptoms to root causes like bad chunks, prompt ordering, or parametric knowledge override, then outline systematic debugging. Do not just say hallucination.
Evaluating a RAG system end to end
Measure retrieval with context recall or precision, and generation with faithfulness and answer relevance, attributing failures to the right stage.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles