Self-Attention versus Recurrent Architectures

WHAT IT TESTS: a clear mental model of self-attention and why it displaced LSTMs. ANSWER OUTLINE: each token projects into query, key, and value vectors; it scores its query against all keys, softmaxes to weights, and outputs a weighted sum of values, so every token directly relates to every other. Versus recurrence this enables full parallelization over the sequence and constant-length paths between distant tokens, easing long-range dependencies. RED FLAG: confusing it with cross-attention or missing the parallelism and path-length arguments.