How Stable Diffusion generates images

WHAT THIS TESTS The interviewer wants a clear, component-by-component account of latent diffusion inference, confirming you understand why it runs in latent space and how text conditions it.

A GOOD ANSWER COVERS The text encoder, typically a CLIP or T5 text model, converts the prompt into a sequence of embeddings that serve as conditioning. Generation starts from a tensor of random Gaussian noise in a compressed latent space, not pixel space, which is the key efficiency trick of latent diffusion. The U-Net is the denoiser: at each step it takes the current noisy latent and the timestep and predicts the noise present, conditioned on the text embeddings injected through cross-attention layers so the prediction aligns with the prompt. The scheduler, or sampler, defines the noise schedule and the update rule that uses the U-Net's noise prediction to compute the next, less-noisy latent, and it determines how many steps run and how aggressively, with options like DDIM or DPM-Solver. After the loop completes, a separate VAE decoder maps the clean latent back into a full-resolution image. Classifier-free guidance scales the influence of the prompt.

COMMON WRONG ANSWERS Saying the U-Net operates on raw pixels; in Stable Diffusion it works in VAE latent space, which is why it is efficient. Another error is conflating the scheduler with the U-Net, or omitting that text conditioning enters via cross-attention.

LIKELY FOLLOW-UPS Expect questions on classifier-free guidance scale, on the VAE's role, on why latent space saves compute, on negative prompts, and on how the scheduler trades steps for quality.

ONE CONCRETE EXAMPLE For the prompt a red bicycle at sunset, CLIP encodes the text, the pipeline starts from random latent noise, and over say thirty scheduler steps the U-Net iteratively removes noise while cross-attending to the embeddings so a bicycle and warm colors emerge in the latent, then the VAE decodes it into the final image.