Interview questions in LLMs & Generative AI, page 5
Core insight behind GPTQ and AWQ
Not all weights matter equally; GPTQ minimizes layer output error using second-order info, AWQ protects salient weight channels tied to large activations.
Rule-based versus model-based LLM guardrails
A guardrail is a programmatic check constraining LLM I/O; rule-based uses regex or blocklists, model-based uses a classifier like a moderation model to detect harmful content.
Detecting RAG hallucinations with a confidence score
Decompose the answer into claims, verify each against retrieved context with NLI or an LLM judge, aggregate into a faithfulness confidence score, and flag unsupported claims.
Direct versus indirect injection and agent defenses
Direct injection comes from the user prompt; indirect hides in third-party data the agent ingests like web pages.
Differential privacy vs utility in LLM fine-tuning
Clipping plus calibrated noise per step, smaller epsilon means stronger privacy but degraded accuracy, tracking the privacy budget across epochs.
Fine-tuning vs RAG for daily-updated docs
Choose RAG because docs change daily, embed and index chunks in a vector store, retrieve top matches and inject into the prompt.
Closed API vs open-weight models for production
APIs offer top quality and zero ops but recurring per-token cost and data-sharing concerns, open weights give control, privacy, and tuning at the price of hosting and MLOps burden.
Hugging Face Hub, transformers, and datasets
The Hub hosts models and data, transformers loads models and tokenizers and provides the Trainer, datasets streams and maps preprocessing.
Mixture of Experts architecture and routing
Many expert FFNs per layer, a router picks top-k experts per token, only those compute so active params are far fewer than total.
Designing a production LLM summarization eval
A representative gold set, quality via human or LLM-as-judge plus faithfulness checks, and operational metrics like p95 latency and cost per request.
Multimodal video understanding architecture
Sample frames, encode them into visual tokens via a vision encoder and projector, concatenate with text tokens, let cross-attention fuse them.
Three techniques to cut LLM inference latency
Quantization shrinks weights with small accuracy risk, KV-cache plus continuous batching boost throughput, speculative decoding drafts tokens for lossless speedup.
LSTM vs GRU gating and trade-offs
LSTM has three gates and a separate cell state, GRU merges gates and state into two, so GRU is lighter and faster while LSTM may model long dependencies better.
Static vs contextual word embeddings
Static embeddings give one fixed vector per word ignoring context, contextual ones vary by sentence and resolve polysemy at higher compute cost.
Encoder, decoder, and encoder-decoder Transformers
Encoder-only uses bidirectional masked-token pretraining for understanding tasks, decoder-only uses causal next-token prediction for generation, encoder-decoder uses span corruption for…
Fixing a prompt that ignores key constraints
Move the critical constraint to a prominent position, state it positively and specifically, separate instructions from data with delimiters, and add a concrete example.
Diagnosing sycophancy from RLHF
Annotators reward agreeable, inoffensive answers so the reward model learns to favor them, fix by diversifying labelers, rewarding factual correctness over agreeableness, and…
When to choose RAG over fine-tuning
RAG for fresh, factual, citable knowledge that changes often, fine-tuning for behavior, style, or format the model must internalize.
Chunking and embedding a RAG corpus
Choose chunk size and overlap balancing context vs precision, prefer semantic boundaries, then pick an embedding model matching domain and dimension, and store with metadata.
Evaluating a multi-tool LLM agent
Measure end-to-end task success, plus trajectory quality like correct tool choice and arguments, efficiency via steps and cost, and robustness to errors and edge cases.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles