How text prompts guide Stable Diffusion
the text-conditioning pipeline.
a frozen text encoder turns the prompt into token embeddings, which feed the U-Net via cross-attention at each denoising step so the prompt steers generation; classifier-free guidance amplifies the…
What's really being asked
Whether you can trace how a natural-language prompt becomes a control signal that steers the iterative denoiser.
The full answer
The prompt is first tokenized and embedded by a text encoder, commonly the text tower of a CLIP model that was pretrained to align text and image representations. The encoder outputs a sequence of contextual token embeddings, one per token, and these are typically kept frozen during diffusion training. The denoising U-Net consumes these embeddings through cross-attention layers: image features form the queries, the text embeddings form the keys and values, so at every denoising step and at several resolutions each spatial location can pull in the words most relevant to it. Because conditioning is applied at every step, the prompt continuously shapes the trajectory from noise to image. To make adherence stronger, classifier-free guidance runs the network twice, once with the prompt and once with an empty prompt, then extrapolates in the direction of the conditional prediction. In latent diffusion all of this happens in a compressed VAE latent space, and a decoder finally maps the denoised latent to pixels.
The mistakes people make
Saying CLIP itself generates the image; claiming the text is concatenated to the pixel or latent tensor instead of attended to; forgetting cross-attention and asserting the embedding is only added once at the start; ignoring classifier-free guidance; confusing the CLIP image encoder with the text encoder.
What usually comes next
Why freeze the text encoder? What does the guidance scale trade off? How would you implement negative prompts? Why use CLIP rather than a plain language model, and what changes if you swap in a larger text encoder like T5?
A concrete example
For the prompt a red bicycle on a beach at sunset, the CLIP text encoder produces seventy-seven token embeddings. At each of, say, thirty denoising steps the U-Net's cross-attention lets the latent region forming the bicycle attend strongly to red and bicycle while the background attends to beach and sunset. Raising the guidance scale makes the colors and objects match the prompt more faithfully at some cost to diversity.
Interview question
What role does the CLIP text encoder play in Stable Diffusion?
- a.It classifies whether the generated image matches the prompt at the end
- b.It adds noise to the image during the forward process
- c.It converts the prompt into token embeddings that condition the U-Net via cross-attentionCorrect
- d.It directly decodes the latent into final pixels
Why? this is the answer
The text encoder turns the prompt into embeddings used as keys and values in the U-Net's cross-attention, steering each denoising step. It does not generate pixels; a separate VAE decoder does that.
Just read this? Test yourself on what you have been reading.
Read the original → en.wikipedia.org
- #stable-diffusion
- #text-to-image
- #clip
- #cross-attention
- #conditioning
Put your scrolling time to good use
Learn one idea, try a quiz and save useful cards for revision. Tezvyn makes it easy to learn and stay current in your tech field, a few minutes at a time.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles