Transformer Encoder-Decoder Architecture
The encoder-decoder Transformer maps an input sequence into rich contextual representations with an encoder, then a decoder generates output tokens autoregressively while attending to those representations via cross-attention, making it ideal for…
The original Transformer pairs two stacks. The encoder reads the whole input at once with bidirectional self-attention, producing context-aware vectors for every input token. The decoder generates the output one token at a time using masked self-attention over what it has produced so far, plus cross-attention that lets each decoding step query the encoder's representations. This separation suits tasks that transform one sequence into another, such as translation or summarization, where the full source must inform every generated token.
Read the original → direct-llm://transformerencoderdecoderarchitecture
- #transformers
- #encoder-decoder
- #attention
- #seq2seq
- #nlp
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.