Cross-attention versus self-attention in encoder-decoder Transformers

WHAT THIS TESTS Whether you can pinpoint the source of Q, K, and V in each attention block and explain why cross-attention is the bridge between encoder and decoder.

A GOOD ANSWER COVERS There are three attention blocks in a classic encoder-decoder. Encoder self-attention lets each source token attend to all other source tokens; Q, K, and V all come from the source, with no masking, so it is bidirectional. Decoder self-attention lets each target token attend to previously generated target tokens; Q, K, and V come from the target sequence, but a causal mask blocks attending to future positions so generation stays autoregressive. Cross-attention is the distinguishing piece: it lives in each decoder layer between the decoder self-attention and the feed-forward sublayer. Its Queries come from the decoder's current hidden states, while its Keys and Values come from the encoder's final output. This lets every decoder position look back over the entire encoded source, which is how a translation decoder aligns each output word with relevant source words. It is not causally masked over the source, since the whole source is available.

COMMON WRONG ANSWERS Saying cross-attention is the same as self-attention. Claiming all of Q, K, V come from the decoder. Applying a causal mask to the encoder Keys. Forgetting that decoder self-attention is masked while cross-attention is not. Putting cross-attention in the encoder.

LIKELY FOLLOW-UPS Why is decoder self-attention masked but cross-attention is not? Where does cross-attention sit within a decoder layer? Why do decoder-only models lack cross-attention? How does cross-attention enable alignment in translation?

ONE CONCRETE EXAMPLE Translating the house is red to French, the encoder produces hidden states for the four English tokens. As the decoder generates rouge, its cross-attention Query attends over the encoder Keys and Values and places high weight on the encoder state for red, pulling that meaning into the output, while decoder self-attention separately tracks the French words generated so far.