Easy interview questions in LLMs & Generative AI
Vanishing Gradients and Why ReLU Helps
Saturating activations shrink gradients across layers, ReLU's flat-one derivative preserves them.
Explain word embeddings and why they beat one-hot encoding for large vocabularies
Embeddings cluster similar meanings in low-dimensional space, while one-hot vectors are orthogonal, huge, and semantically blank.
Self-Attention versus Recurrent Architectures
Each token attends to all others via query-key-value, enabling parallelism and direct long-range links.
Causal versus Masked Language Modeling
Pre-training learns general language from unlabeled text; CLM predicts the next token left-to-right, MLM predicts masked tokens using both sides.
How would you construct zero-shot and few-shot prompts for feedback classification?
Tests knowledge of zero-shot versus few-shot prompt structure. Zero-shot gives instructions, labels, and format without examples; few-shot prepends 2-4 labeled demonstrations before the target input. Red flag: calling an example-containing prompt zero-shot.
Explain Chain-of-Thought prompting, its reasoning mechanism, and ideal use cases
This tests reasoning scaffolding. A good answer says CoT makes the model emit intermediate steps before the final answer, excelling at multi-step math and logic versus direct instructions.
Describe supervised fine-tuning for a pre-trained language model
Tests if you know SFT aligns a base model to instructions using curated prompt-completion data. A strong answer covers next-token prediction on completions, conversational formats, and small learning rates.
Pre-training versus fine-tuning an LLM
Pre-training is broad self-supervised next-token prediction on huge corpora at massive cost; fine-tuning adapts on small labeled data cheaply.
Describe a basic RAG architecture and its two main components
This tests retrieval-generation separation. Good answers name the retriever, which fetches relevant documents, and the generator, which synthesizes an answer using those documents plus the query.
How does function calling work in modern LLMs?
Schemas in the prompt; model emits JSON name and arguments; client executes and returns results.
Walk me through building a weather agent with get_weather
Register get_weather, let the model emit parameters, execute it yourself, feed the result back, then synthesize the answer.
Explain GAN architecture, generator and discriminator roles, and objective function
Tests adversarial training as a minimax game. Strong answers: generator maps noise z to fakes; discriminator classifies real versus fake; both optimize V(D,G)=E[log D(x)]+E[log(1-D(G(z)))].
Key latent space difference between Autoencoder and VAE, and generative use
This tests deterministic versus probabilistic latent representations. Standard autoencoders encode fixed points; VAEs encode distributions. Sampling the regularized latent distribution generates new data. Red flag: calling VAEs mere noise adders.
Explain Denoising Diffusion models and forward/reverse processes.
This tests if you see diffusion as iterative latent generation, not GANs. Forward: add Gaussian noise over T steps until data is pure noise. Reverse: a network iteratively denoises random noise into data.
Classic image captioning architecture
A CNN encoder extracts image features, a recurrent or transformer decoder generates the caption word by word, and attention lets the decoder focus on image regions per word.
Early versus late modality fusion
Early fusion merges raw or low-level features so the model learns cross-modal interactions, while late fusion processes each modality separately and combines outputs.
Perplexity versus BLEU for LMs
Perplexity measures intrinsic next-token prediction quality needing no references; BLEU measures n-gram overlap with reference outputs for tasks like translation.
Why human evaluation is the gold standard
Humans judge fluency, helpfulness, and correctness that n-gram or distribution metrics miss; automated scores correlate weakly with quality, are gameable, and penalize valid diverse outputs.
Standard metric for image generation quality
Name FID, explain it compares feature distributions of real and generated images via a pretrained network.
What RLHF is and the safety problem it solves
Collect human preference rankings, train a reward model, fine-tune the policy with PPO; it aligns outputs with human intent the loss function cannot specify.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles