All bites
The whole library, newest first. Filter by what you are here for, or pick a topic if you already know.
4330 bites
Page 42
Self-Attention versus Recurrent Architectures
Each token attends to all others via query-key-value, enabling parallelism and direct long-range links.
Explain Q, K, and V matrices in self-attention
This tests the information-retrieval intuition behind self-attention. Cover that Q, K, V are linear projections of one input; Q requests, K indexes, V supplies content; scores weight a sum of V.
Why Multi-Head Attention
Multiple heads attend to different subspaces and relations in parallel, which one big head averages away.
What is masked in decoder self-attention and why?
Future positions are masked so token i attends only to prior tokens. This prevents cheating during parallel teacher-forced training when the full target is visible.
Why is self-attention O(n^2) and what are the implications?
Tests the attention matrix bottleneck. Strong answers note QK^T yields an N×N matrix, creating quadratic compute and memory that blocks long documents and high-res images. Red flag: confusing model size with activation memory.
Layer Norm and Residuals in Transformer Blocks
Residuals preserve gradient flow, layer norm stabilizes activations per token, and it beats batch norm because it is independent of batch and sequence length.
Causal versus Masked Language Modeling
Pre-training learns general language from unlabeled text; CLM predicts the next token left-to-right, MLM predicts masked tokens using both sides.
How did Chinchilla change compute allocation between model size and data size?
This tests whether you know prior scaling fixed data while growing parameters, but Chinchilla showed parameters and tokens must scale equally. A good answer: double both together, so train smaller models on more data. Red flag: huge models, fixed data.
Explain data, tensor, and pipeline parallelism and hybrid training strategy
Tests communication and memory tradeoffs of core distributed training strategies. Strong answers contrast data parallelism (shard batch), tensor parallelism (shard layers, all-reduce), and pipeline parallelism (shard stages, p2p), then propose a 3D hybrid…
What causes sudden loss spikes in long pre-training runs?
Name gradient explosions, LR mismatch, FP16 overflow, and poison batches; propose norm checks, rollback, and LR cuts.
How does pre-training dataset composition influence capabilities and biases?
This probes whether you link data mix to capabilities and bias. Answer: code strengthens reasoning, web text adds noise; for science, use domain-adaptive pretraining on filtered literature, instruction tuning, and reasoning distillation, validating via…
FlashAttention and IO-Aware Attention
FlashAttention is IO-aware, tiling and fusing attention in fast SRAM to avoid materializing the n-by-n matrix in slow HBM.
How would you construct zero-shot and few-shot prompts for feedback classification?
Tests knowledge of zero-shot versus few-shot prompt structure. Zero-shot gives instructions, labels, and format without examples; few-shot prepends 2-4 labeled demonstrations before the target input. Red flag: calling an example-containing prompt zero-shot.
Explain Chain-of-Thought prompting, its reasoning mechanism, and ideal use cases
This tests reasoning scaffolding. A good answer says CoT makes the model emit intermediate steps before the final answer, excelling at multi-step math and logic versus direct instructions.
How do you select in-context examples for text-to-SQL prompts?
What it tests: practical ICL design for structured generation. Answer outline: select examples by SQL syntax similarity plus pattern diversity, order from simple to complex, and anchor schema context. Red flag: claiming random examples work fine.
Describe two prompt-based techniques to ensure valid LLM JSON output
This tests output constriction via prompt design. First, embed an exact JSON skeleton with empty values. Second, provide few-shot exemplars mapping inputs to valid JSON. A red flag is suggesting only post-hoc regex repair or larger models.
Zero-Shot, Few-Shot, and Chain-of-Thought Trade-offs
Zero-shot is cheap but weak on reasoning, few-shot adds demos at token cost, CoT boosts multi-step accuracy but spends the most tokens and latency.
Design dynamic few-shot example retrieval from a vector database
Tests RAG-style prompt engineering with semantic retrieval and latency. Use shared embeddings, approximate nearest neighbors with metadata filters, diversity reranking, and token-bounded prompt templates.
Describe supervised fine-tuning for a pre-trained language model
Tests if you know SFT aligns a base model to instructions using curated prompt-completion data. A strong answer covers next-token prediction on completions, conversational formats, and small learning rates.
Pre-training versus fine-tuning an LLM
Pre-training is broad self-supervised next-token prediction on huge corpora at massive cost; fine-tuning adapts on small labeled data cheaply.