Cross-Attention — AI Glossary

Reference details

Related terms

• Multi-Head Attention (MHA)
• Encoder-Decoder Architecture
• Multimodal Architecture

How it works

1. Self-attention derives queries, keys and values from one sequence. Cross-attention splits the sources: queries come from the sequence being generated, keys and values from a different, already-encoded one.
2. Each generated position therefore scores itself against every position of the other sequence, producing a weighting over the input rather than over its own prefix.
3. The output is a weighted sum of the other sequence's values, so the decoder reads from the encoder at every layer rather than receiving a single summary vector.
4. Because keys and values are fixed once the input is encoded, they are computed once and reused for every generated token.
5. The same mechanism carries any two modalities: swap the encoded side for image patches or audio frames and text positions attend over them unchanged.

Example: What the split buys you

Translating a 40-token sentence into 35 tokens of output: self-attention at the final decoder position scores 35 queries against its own prefix, while cross-attention scores them against all 40 encoder positions. The encoder's keys and values are computed once and reused, so the per-token cost is the 35 × 40 scoring rather than re-encoding the source. The point is not the saving but the access pattern — every output position can look anywhere in the input, which is exactly what a fixed-size summary vector could not offer.

Commonly confused with

Easily merged with self-attention, since the arithmetic is identical and only the sources differ. Self-attention relates a sequence to itself; cross-attention relates one sequence to another, which is why it appears wherever two things must be aligned — source to translation, image to caption, query to retrieved passage. Worth separating from retrieval-augmented generation too: RAG concatenates retrieved text into the prompt and lets ordinary self-attention handle it, whereas cross-attention keeps the retrieved side as a separate encoded sequence. Both let a model condition on external content, and only one changes the architecture.

When to use it

Reach for it when:

• Explaining how encoder-decoder models condition generation on an input sequence.
• Describing how a multimodal model lets text positions read image or audio representations.
• Distinguishing architectural conditioning from prompt-level conditioning.

Reach for something else when:

• Describing a decoder-only model attending over its own context, including retrieved text pasted into the prompt — that is self-attention.
• As a synonym for attention in general; the defining feature is that queries and keys come from different sequences.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Cross-Attention — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path