Interview questions in LLMs & Generative AI, page 6
Agent planning beyond a ReAct loop
ReAct adapts step by step but costs many calls, plan-then-execute drafts a full plan upfront for fewer calls but is brittle to surprises, hierarchical decomposition splits goals…
Detecting catastrophic forgetting in continual fine-tuning
Maintain a frozen held-out benchmark of original capabilities, evaluate after every fine-tune, track per-capability deltas, and alert on regressions.
Dynamic batching and the throughput-latency trade-off
The server groups concurrent requests into one batch to use the GPU fully, but larger batches and waiting to fill them raise per-request latency and time to first token.
Handling outlier activations in INT8 quantization
Profile activation magnitudes to find a few large-magnitude outlier channels, then keep those in higher precision while quantizing the rest, a mixed-precision decomposition.
Mitigating demographic bias in a fine-tuned chatbot
Curate or counterfactually augment training data to balance demographics, plus apply post-hoc guardrails or fairness-constrained fine-tuning.
Preprocessing conversations to protect privacy before fine-tuning
Detect and redact PII with NER plus regex, choose redaction versus pseudonymization, and validate recall.
Measuring fairness in an embedding-based text classifier
Define group definitions, apply demographic parity and equalized odds, and address noisy labels and implicit group membership.
Practical explainability for an LLM loan summary
Use attribution-by-design with grounded citations, structured rationales, and a deterministic rules layer instead of slow per-token SHAP.
Data lineage and machine unlearning for a fine-tuned LLM
Version and fingerprint datasets, record transforms and which checkpoint saw what, and enable unlearning via retraining, data sharding, or approximate gradient methods.
Designing an autonomous research-and-report agent
Planner that decomposes goals, short-term scratchpad plus long-term vector memory, structured tool calls, and a reflect-retry loop for error correction.
Why RAG persists despite million-token context windows
Cost and latency scale with context, attention degrades in the middle, and RAG adds freshness, access control, and citations.
Self-attention and the Query, Key, Value matrices
Queries score against keys via scaled dot product, softmax yields weights, and those weight the values into the output.
How Transformers encode token position
Attention is permutation-invariant, so positional encodings (sinusoidal, learned, or rotary) are added or applied.
Cross-attention versus self-attention in encoder-decoder Transformers
Cross-attention draws Queries from the decoder and Keys/Values from the encoder, letting the decoder condition on the source.
Tokens and vocabulary-size tradeoffs
A token is a subword unit; larger vocab shortens sequences but bloats the embedding matrix, smaller vocab generalizes but lengthens sequences.
Fault-tolerant checkpointing for thousand-GPU pre-training
Checkpoint weights, optimizer state, RNG, and data position together; use asynchronous sharded writes and automated detect-restart-resume.
Prompt engineering to curb extraction hallucinations
Ground strictly in source, allow null for missing fields, enforce a schema, and use few-shot examples; acknowledge prompting cannot fully eliminate it.
Self-consistency over chain-of-thought
Sample multiple CoT paths at nonzero temperature and majority-vote the final answer; cost scales with the number of samples.
Why chain-of-thought helps large models but not small ones
Small models lack reliable multi-step reasoning, so CoT just adds error-prone steps; adapt by using few-shot/fine-tuning or distillation for small tiers.
Handling a 401 error in an LLM agent's tool call
Catch the tool error, return a structured observation to the LLM, and distinguish recoverable retries from terminal failures needing re-plan or escalation.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles