Top 30 Optimization Interview Questions and Answers
30 multiple-choice questions on Optimization, drawn from 30 bites out of the 48 tagged Optimization on Tezvyn. Answer them here or read straight down. Every question carries the correct option, why it is correct, and a link to the bite it came from.
30 questions. Pick an answer, or open “Show the answer” to read it.
Answers are graded in your browser. Nothing is saved, and no XP or streak is earned here. The app keeps score.
Question 1 of 30
Which situation best illustrates why a loss function should not be the only measure of a model's real-world effectiveness?
Show the answer
Answer: a · The model achieves a low loss score, but its predictions are biased or provide no practical value to users.
The card explicitly states that a low loss score doesn't guarantee real-world usefulness, as outputs can still be nonsensical, biased, or unhelpful. This highlights that loss functions are for optimization, not the final arbiter of a model's real-world usefulness. Option C describes a scenario where a loss function isn't typically used, rather than a limitation of relying on it as a sole metric when it is applied.
Read the full bite: Loss Function: Quantifying 'How Wrong' a Model Is
Question 2 of 30
Why does solver-specific code generated by LLMs create production risk that ORPilot's JSON IR avoids?
Show the answer
Answer: c · Data or solver changes force a new LLM API call, risking nondeterministic model drift
The card explains that solver-specific code breaks when data or solvers change, forcing costly LLM regeneration and risking model drift, whereas the IR captures the mathematical structure as portable JSON that can be retargeted without another API call.
Question 3 of 30
What happens when the learning rate in gradient descent is set too high?
Show the answer
Answer: a · Steps overshoot the minimum and the loss may oscillate or diverge
Too large a step size overshoots the minimum, causing oscillation or divergence instead of convergence. A high rate does not guarantee faster or correct convergence, and it does not change how the gradient is computed.
Read the full bite: How gradient descent and the learning rate work
Question 4 of 30
What is the core reason FlashAttention is faster than a naive attention implementation?
Show the answer
Answer: c · It is IO-aware: tiling and kernel fusion avoid writing the full attention matrix to slow HBM
FlashAttention computes exact attention but minimizes slow HBM traffic by tiling into SRAM and fusing kernels with an online softmax. It does not approximate, sparsify, or reduce the arithmetic complexity; it cuts memory movement.
Question 5 of 30
What problem does loss scaling primarily address in mixed-precision training?
Show the answer
Answer: d · The potential for small gradient values to become zero in FP16.
Loss scaling's primary purpose is to inflate small gradient values before converting them to FP16, preventing them from becoming zero (underflow), which would otherwise cause training to fail. While a master copy of weights is kept in FP32 for stability, loss scaling directly ensures the gradients themselves are numerically viable.
Read the full bite: Mixed-Precision Training: Faster Training with Less Memory
Question 6 of 30
How does Gradient Descent determine the direction to update model parameters?
Show the answer
Answer: d · By taking a step in the direction opposite to the gradient of the cost function.
Gradient Descent works by iteratively taking small steps in the direction opposite to the gradient of the cost function, as the gradient points towards the steepest ascent. Option B is incorrect because following the direction of steepest increase would maximize, not minimize, the cost function.
Read the full bite: Gradient Descent: Finding the Bottom of the Hill
Question 7 of 30
What problem do Docker multi-stage builds primarily solve for application deployment?
Show the answer
Answer: b · The excessive size of Docker images due to included build-time dependencies.
The card clearly states that multi-stage builds exist because "creating small Docker images was clumsy" and they "keeps images small by excluding build-time dependencies," dramatically reducing the final image size. While consolidating Dockerfile logic (D) is a side benefit, the primary problem addressed is image bloat.
Read the full bite: Docker Multi-stage Builds: Slimmer, Faster Images
Question 8 of 30
When is a learning rate schedule particularly crucial for achieving state-of-the-art results?
Show the answer
Answer: d · When fine-tuning large, complex models such as transformers.
C is correct because the card states schedules are "standard practice for training and fine-tuning large models, especially transformers... essential for achieving state-of-the-art results." B is a tempting distractor, but the card explicitly warns against using external schedules with optimizers like AdaFactor due to potential conflicts.
Read the full bite: Learning Rate Scheduling: A Gearbox for Model Training
Question 9 of 30
Which statement accurately describes the primary role of the Jacobian matrix for a differentiable multi-variable function at a specific point?
Show the answer
Answer: d · It provides the best linear approximation of the function's local behavior.
The card states that "The Jacobian matrix is the best linear approximation of a function at a specific point," capturing its local, first-order behavior. Option B is incorrect because the card explicitly mentions that the Jacobian does not provide second-order derivative information like curvature or concavity.
Read the full bite: The Jacobian Matrix: A Derivative for Multiple Dimensions
Question 10 of 30
In Flutter, a parent widget rebuilds frequently due to state changes, but its child displays static data. Which approach correctly prevents the child's build method from running unnecessarily?
Show the answer
Answer: c · Extract the child into its own StatelessWidget and invoke it with a const constructor.
Extracting the child into its own StatelessWidget and invoking it with const creates a build boundary, allowing Flutter to reuse the existing element and skip calling build on that subtree. Wrapping it in a RepaintBoundary is a common misconception because that only reduces paint cost during rasterization, not build-phase work.
Read the full bite: How can you prevent unnecessary child rebuilds in Flutter?
Question 11 of 30
Blank cells appear during fast scrolling of fixed-height rows. Which single change most directly reduces the blanks?
Show the answer
Answer: c · Provide getItemLayout so FlatList positions cells without measuring them
getItemLayout lets FlatList compute positions instantly for known heights, cutting the lag that produces blanks. A huge windowSize trades blanks for memory pressure and new jank.
Read the full bite: Diagnosing slow FlatList scroll and blank cells
Question 12 of 30
Why does running apt-get clean in a separate RUN instruction fail to shrink the image?
Show the answer
Answer: c · The cache already persists in the earlier layer, and a later layer cannot delete data from a prior one
Each layer is additive; deleting files in a later layer leaves them in the earlier layer's size. Cleaning must happen in the same RUN that created the cache, so option C is correct.
Read the full bite: Three techniques to shrink a Docker image
Question 13 of 30
Which combination best shrinks a PyTorch serving image while preserving operational rigor and layer hygiene?
Show the answer
Answer: c · Build in a devel stage, copy artifacts to a runtime stage, and collapse cleanup into the install RUN
The correct approach uses multi-stage builds to exclude devel libraries and collapses cleanup into the installation RUN so intermediate files never commit to a layer. The most tempting distractor, docker squash, merely hides bloat rather than fixing build hygiene, and separate RUN cleanups still preserve deleted files in underlying layers.
Read the full bite: Strategies to reduce a 5GB ML Docker image size
Question 14 of 30
What is the most significant outcome of using a multi-stage Docker build for a compiled application like Go?
Show the answer
Answer: d · The resulting image is substantially smaller, containing only the necessary runtime artifacts.
Multi-stage builds are designed to produce lean images by separating the build environment from the runtime environment. This significantly reduces the final image size, as only the compiled application and its essential dependencies are included, not the entire build toolchain. Option C is incorrect because while Docker caching helps, multi-stage builds' primary benefit isn't faster build time but smaller image size and faster deployment.
Read the full bite: Multi-stage Docker Builds: Lean Images, Fast Deploys
Question 15 of 30
Which factor would most directly prevent a query optimizer from successfully pushing down a predicate to the data source?
Show the answer
Answer: b · The predicate involves a user-defined function (UDF) that the data source does not recognize.
The card explicitly states that if a predicate involves a UDF only the query engine understands, it cannot be pushed down. While other options describe scenarios where pushdown might be less beneficial or relevant, they do not directly prevent the optimizer from attempting to push down a filter the source could handle.
Read the full bite: Predicate Pushdown: Filter Data at the Source
Question 16 of 30
What mathematical principle is fundamental to backpropagation's ability to efficiently adjust neural network weights?
Show the answer
Answer: c · The chain rule of calculus for gradient computation.
The card states that backpropagation "Using the chain rule from calculus, it calculates the gradient of the loss with respect to every single weight in the network." This mathematical principle is crucial for efficiently determining how each weight contributes to the overall error. While other mathematical concepts are used in neural networks, the chain rule is central to the backward pass of backpropagation. Matrix inversion is not used for weight updates in gradient-based optimization.
Read the full bite: Backpropagation: How Neural Networks Learn from Mistakes
Question 17 of 30
What is the primary purpose of the learned scaling factor (gamma) and shifting factor (beta) in Batch Normalization?
Show the answer
Answer: c · To enable the network to learn and apply an optimal mean and variance for each layer's inputs.
After the initial normalization to mean zero and variance one, gamma and beta allow the network to learn and apply an optimal scale and mean for the activations, potentially deviating from zero and one if it aids training. Option B describes the initial normalization step, which occurs before gamma and beta are applied.
Read the full bite: Batch Normalization: Stabilizing Neural Network Training
Question 18 of 30
In a production Dockerfile, what is the primary purpose of splitting the build into multiple stages?
Show the answer
Answer: a · To keep build dependencies and compilers out of the final deployed image
Multi-stage builds isolate compilation in a builder stage and copy only runtime artifacts to the final image, eliminating compilers and dev tools that increase size and attack surface. Option D describes a common anti-pattern: keeping build tools in production for debugging, which defeats the security and size benefits of multi-stage builds.
Read the full bite: Walk me through a production-ready Dockerfile for a web app
Question 19 of 30
When using a bundle analyzer to optimize web application performance, which metric is most critical to focus on for identifying load time issues?
Show the answer
Answer: c · The gzipped size, representing the actual data transferred to the user.
The card explicitly states that the 'footgun' is to focus on gzipped size, not raw size. This is because gzipped size accurately reflects the amount of data transferred over the network, which directly impacts load times, unlike raw or parsed sizes.
Read the full bite: Bundle Analysis: An X-Ray for Your App's Weight
Question 20 of 30
When is Quantization-Aware Training the better choice over Post-Training Quantization?
Show the answer
Answer: a · When PTQ's accuracy loss is unacceptable and you can afford the extra training time and compute
QAT recovers accuracy by training the model to tolerate quantization, justified when PTQ degrades quality too much. PTQ, not QAT, is the no-training, calibration-only, fast-and-cheap option.
Question 21 of 30
A Figma prototype lags specifically when transitioning between screens. Inspection shows a single background image embedded at 6000 pixels wide, reused across forty frames, plus a background blur on every modal. What is the most effective fix?
Show the answer
Answer: a · Resize the background image to its actual display resolution and replace the blur with a semi-transparent overlay
Oversized embedded images and recomputed effects like blur are GPU costs paid on every interaction, so resizing the image and dropping the blur fixes the real bottleneck. It is not a network problem, and the effect recomputes on every transition, not just at load.
Question 22 of 30
What shared insight lets GPTQ and AWQ preserve accuracy at INT4 where naive rounding fails?
Show the answer
Answer: c · Weights contribute unequally to output, so quantization should protect or compensate the important ones using activation statistics
Both methods exploit unequal weight importance, GPTQ compensating via second-order info and AWQ protecting salient activation-aligned channels. They are post-training, do not retrain like QAT, and do not keep everything in FP16.
Question 23 of 30
What is the primary advantage of employing model compilation in machine learning workflows?
Show the answer
Answer: d · It enables a single model definition to be efficiently deployed across various hardware platforms.
Model compilation's core purpose is to translate a high-level model into an optimized, hardware-specific artifact, solving the 'write once, run anywhere' problem for efficient inference across diverse devices. While compilation involves optimization, it is explicitly for inference performance, not training acceleration or facilitating early-stage architectural changes.
Read the full bite: Model Compilation: Bridging Models and Hardware
Question 24 of 30
For which scenario would multi-model serving be the most advantageous deployment strategy?
Show the answer
Answer: b · Numerous 'long-tail' models, each serving a small user base with infrequent, unpredictable requests.
Multi-model serving is ideal for 'long-tail' use cases with many models and intermittent traffic, as it maximizes resource utilization and minimizes costs by sharing framework memory. It is not recommended for high-throughput, mission-critical models or those with conflicting dependencies.
Read the full bite: Multi-Model Serving: Packing More Models into Less RAM
Question 25 of 30
Which statement best describes the primary function of Dynamic Creative Optimization (DCO)?
Show the answer
Answer: a · It uses real-time technology to adapt and enhance ad creative elements during a campaign.
DCO's core purpose is to optimize the performance of creative content using real-time technology, allowing for dynamic adjustments to ad elements during a campaign. While other options describe aspects of programmatic advertising, they are not the specific function of DCO, which is centered on creative optimization.
Question 26 of 30
In MAML, what is actually being learned by the outer loop optimization?
Show the answer
Answer: c · A parameter initialization from which a few gradient steps adapt to new tasks
The outer loop optimizes the shared initialization so that brief task-specific inner-loop adaptation generalizes. It is not a single all-task solver, and unlike metric methods it does not learn a distance function.
Read the full bite: How does MAML's inner and outer loop work?
Question 27 of 30
What fundamentally distinguishes a multi-armed bandit from a traditional fixed-horizon A/B test?
Show the answer
Answer: d · A bandit continuously reallocates traffic toward better-performing arms during the run
Bandits adapt allocation in real time to reduce regret, balancing explore and exploit, whereas A/B holds a fixed split until the end. They still track rewards, and their adaptive nature actually makes classical significance messier, not guaranteed faster.
Read the full bite: Multi-armed bandit vs A/B testing for headlines
Question 28 of 30
What is the primary mechanism by which KV Cache accelerates large language model (LLM) text generation?
Show the answer
Answer: d · It stores the Key and Value vectors of previously generated tokens, preventing their recomputation in subsequent steps.
KV Cache speeds up generation by storing the Key and Value vectors of past tokens, so the model doesn't have to recompute them for every new token. Option B is incorrect because KV Cache focuses on storing K and V vectors of *past* tokens, not pre-computing Q vectors for all tokens.
Read the full bite: KV Cache: Don't Recompute, Just Remember
Question 29 of 30
Which scenario presents a significant challenge for effectively applying a Multi-Armed Bandit (MAB) algorithm?
Show the answer
Answer: b · The reward signal, such as a purchase, typically occurs days after the initial interaction.
MAB algorithms require rapid feedback to learn and adapt their exploration-exploitation strategy effectively; a heavily delayed reward signal hinders this learning process. Maximizing performance during runtime and dynamic traffic allocation are, in fact, primary benefits of using MABs.
Read the full bite: Multi-Armed Bandit: The Explore vs. Exploit Trade-off
Question 30 of 30
What is the core function of ONNX Runtime in the AI model lifecycle?
Show the answer
Answer: a · To serve as a universal, high-performance engine for deploying trained AI models across diverse hardware platforms.
ONNX Runtime's primary role is to act as a high-performance inference engine, allowing trained AI models (in ONNX format) to be deployed and run efficiently on various hardware. While conversion to ONNX format is a prerequisite, ONNX Runtime itself is not a conversion tool between training frameworks, nor is it for training or experimentation.
Read the full bite: ONNX Runtime: Run Any AI Model, Anywhere
Could you explain these out loud?
That is what an interview actually tests. Tezvyn gives you questions like these with what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.