Bi-Encoder — AI Glossary

Reference details

Related terms

• Dense Retrieval
• Late Interaction (ColBERT)
• Embedding Model

How it works

1. Two towers map queries and documents into one space, and whether they are two copies of the same weights or two separately trained encoders is an architectural choice with consequences.
2. Shared weights force one representation to serve both roles, which works when queries resemble documents and fails when a short question must match a long passage.
3. Separate towers let each side specialise, at the cost of twice the parameters and a strict requirement that the two are always versioned and deployed together.
4. Either way the document tower is run offline over the corpus and the query tower online, so only the query tower's latency is ever on the request path.
5. The similarity function is fixed by training rather than chosen at query time: a model trained with cosine objectives must be queried with cosine, and substituting a dot product on unnormalised vectors silently changes the ranking.
6. Pooling is equally part of the trained model, so taking a mean where the model was trained on a designated position's output produces vectors that are valid-looking and wrong.

Example: Why the work happens before the query arrives

A bi-encoder embeds each document once, offline. A million documents means a million forward passes done ahead of time, and at query time exactly one more — encode the query, then compare vectors, which is arithmetic rather than inference. A cross-encoder cannot do this: it reads query and document together, so scoring a million documents means a million forward passes *per query*. That is the whole reason retrieval is two stages. The bi-encoder is not more accurate; it is the only one of the two that can be precomputed.

Commonly confused with

Bi-encoder and cross-encoder are architectures, not quality tiers. The cross-encoder wins on accuracy because it can attend across the pair; the bi-encoder wins on cost because it never has to see the pair. Reranker is the role a cross-encoder plays in a pipeline, and embedding model is the product form of a bi-encoder.

When to use it

Reach for it when:

• First-stage retrieval over any corpus large enough that per-query scoring is impossible
• Clustering, deduplication and classification, where reusable vectors are the point
• Where documents change rarely and queries are frequent — the precompute amortises

Reach for something else when:

• Final ordering of a small candidate set, where a cross-encoder is affordable and better
• Corpora that churn constantly, since every change means re-encoding and re-indexing
• Comparing vectors from two different bi-encoders — separate models, separate spaces

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Bi-Encoder — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path