How text prompts guide Stable Diffusion

WHAT THIS TESTS: Whether you can trace how a natural-language prompt becomes a control signal that steers the iterative denoiser.

A GOOD ANSWER COVERS: The prompt is first tokenized and embedded by a text encoder, commonly the text tower of a CLIP model that was pretrained to align text and image representations. The encoder outputs a sequence of contextual token embeddings, one per token, and these are typically kept frozen during diffusion training. The denoising U-Net consumes these embeddings through cross-attention layers: image features form the queries, the text embeddings form the keys and values, so at every denoising step and at several resolutions each spatial location can pull in the words most relevant to it. Because conditioning is applied at every step, the prompt continuously shapes the trajectory from noise to image. To make adherence stronger, classifier-free guidance runs the network twice, once with the prompt and once with an empty prompt, then extrapolates in the direction of the conditional prediction. In latent diffusion all of this happens in a compressed VAE latent space, and a decoder finally maps the denoised latent to pixels.

COMMON WRONG ANSWERS: Saying CLIP itself generates the image; claiming the text is concatenated to the pixel or latent tensor instead of attended to; forgetting cross-attention and asserting the embedding is only added once at the start; ignoring classifier-free guidance; confusing the CLIP image encoder with the text encoder.

LIKELY FOLLOW-UPS: Why freeze the text encoder? What does the guidance scale trade off? How would you implement negative prompts? Why use CLIP rather than a plain language model, and what changes if you swap in a larger text encoder like T5?

ONE CONCRETE EXAMPLE: For the prompt a red bicycle on a beach at sunset, the CLIP text encoder produces seventy-seven token embeddings. At each of, say, thirty denoising steps the U-Net's cross-attention lets the latent region forming the bicycle attend strongly to red and bicycle while the background attends to beach and sunset. Raising the guidance scale makes the colors and objects match the prompt more faithfully at some cost to diversity.