Encoder-Decoder Architecture — AI Glossary

Reference details

Related terms

• Decoder-Only Architecture
• Encoder-Only Architecture
• Cross-Attention

How it works

1. The encoder runs first and in full, with every input token attending to every other in both directions, producing one representation per input position.
2. Those representations are computed once per input and then held fixed for the entire output, which is the structural reason the architecture suits a known input and an unknown output.
3. Each decoder block carries two attention sublayers rather than one: causal self-attention over the output generated so far, then cross-attention into the encoder's representations.
4. In cross-attention the queries come from the decoder and the keys and values from the encoder, so the decoder is reading a finished object rather than extending its own sequence.
5. That split is what lets the input be processed bidirectionally while the output is still generated left to right, which no single causal stack can do.
6. The cost is that input and output are separate computations with a fixed boundary, so there is no way to revisit the input in light of what has been generated — the encoder ran before any of it existed.

Example: Bidirectional reading is the thing you give up

In an encoder-decoder, the input is read by a bidirectional encoder: every input token attends to every other, before any output exists. Translating a German sentence whose verb lands at the end, the encoder has that verb available while representing the first word. The decoder then generates while cross-attending to those finished representations. A decoder-only model reads the same input left-to-right under a causal mask, so the representation of the first word never sees the last. In practice scale has covered the gap for general generation — but the architectural difference is real, and it is why the encoder-decoder shape persists where input and output are distinct sequences.

Commonly confused with

Encoder-decoder is not the same as an encoder-only model. Encoder-only produces representations and cannot generate — it is what an embedding model or a classifier is built from. Encoder-decoder generates conditioned on a fully-read input; decoder-only generates conditioned on everything to its left, prompt and output alike.

When to use it

Reach for it when:

• Fixed transformations with a clear input and output sequence — translation, structured rewriting
• Where bidirectional input understanding matters more than open-ended generation
• Smaller task-specific models, where the shape is efficient for the job

Reach for something else when:

• General chat and agentic work, where the decoder-only ecosystem is far better served
• Anything needing in-context learning at scale — that is where decoder-only models concentrate
• Assuming cross-attention makes it immune to long-input degradation; it does not

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Encoder-Decoder Architecture — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path