Advanced everything in LLMs & Generative AI
Three techniques to cut LLM inference latency
Quantization shrinks weights with small accuracy risk, KV-cache plus continuous batching boost throughput, speculative decoding drafts tokens for lossless speedup.
Differential privacy vs utility in LLM fine-tuning
Clipping plus calibrated noise per step, smaller epsilon means stronger privacy but degraded accuracy, tracking the privacy budget across epochs.
Core insight behind GPTQ and AWQ
Not all weights matter equally; GPTQ minimizes layer output error using second-order info, AWQ protects salient weight channels tied to large activations.
Tensor versus pipeline parallelism for large models
Tensor parallelism splits individual layers across GPUs needing fast interconnect; pipeline parallelism splits layers into stages across GPUs.
Setting up an LLM-as-a-judge evaluation
Define rubric, prefer pairwise comparison, randomize order, calibrate against humans; control position, verbosity, and self-preference bias.
Batching strategy for multimodal training
Control dataset mixing ratios, use balanced sampling and per-source weighting, keep enough text-only data to avoid forgetting, and handle variable shapes via grouping or padding.
LLaVA versus Flamingo vision-LLM design
LLaVA projects image features into the LLM input space and feeds them as tokens, keeping the LLM mostly intact; Flamingo inserts gated cross-attention layers inside a frozen LLM.
Temporal consistency in video diffusion
Add temporal layers, such as temporal attention or 3D convolutions across frames, so the model attends across time and frames denoise jointly rather than independently.
Diffusion-based image inpainting design
At each denoising step keep the known region by replacing it with the noised original, let the model generate only the masked area, condition on prompt and mask.
DDIM: faster diffusion sampling
DDIM defines a non-Markovian deterministic process sharing DDPM's training, letting you skip steps and sample in far fewer iterations.
Designing an agent that resolves ambiguity
Detect ambiguity, gather evidence with the contact API, resolve relative time deterministically, ask the user only when genuinely uncertain, then confirm before the irreversible booking.
Securing tool-using LLM agents
Name indirect prompt injection, data exfiltration, and unsafe tool execution, then defend with sandboxing, least-privilege scoped tools, input/output filtering, and human-in-the-loop on risky actions.
Hybrid search and re-ranking for retrieval
Hybrid search fuses dense semantic and sparse keyword signals to catch exact terms dense misses; a cross-encoder re-ranker rescoring top-k boosts precision.
Direct Preference Optimization explained
DPO reparameterizes the RLHF reward in terms of the policy itself, turning alignment into a simple classification loss on preference pairs with no separate reward model or PPO.
FlashAttention and IO-Aware Attention
FlashAttention is IO-aware, tiling and fusing attention in fast SRAM to avoid materializing the n-by-n matrix in slow HBM.
Layer Norm and Residuals in Transformer Blocks
Residuals preserve gradient flow, layer norm stabilizes activations per token, and it beats batch norm because it is independent of batch and sequence length.
How would you architect a multi-turn conversational RAG system?
This tests memory and query reformulation design beyond single-turn RAG. A strong answer covers 5-10 turn windows, LLM-based rewriting with coreference resolution, hybrid fallbacks, and summarized memory.
Identify RAG latency bottlenecks and propose optimizations
This tests systems thinking across the RAG pipeline. A strong answer names four bottlenecks—embedding, search, chunking, and generation—and pairs each with caching, index tuning, and distillation. Red flag: GPU scaling without indexing fixes.
What does the KL-divergence penalty do in RLHF PPO, and if zeroed?
It tests RLHF reward hacking awareness. The KL penalty anchors PPO to the reference model to stop mode collapse; zeroing it causes over-optimization against the proxy reward model, yielding incoherent outputs.
Design dynamic few-shot example retrieval from a vector database
Tests RAG-style prompt engineering with semantic retrieval and latency. Use shared embeddings, approximate nearest neighbors with metadata filters, diversity reranking, and token-bounded prompt templates.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles