MLDR Leaderboard

What this benchmark measures

• long-document retrieval
• multilingual embedding retrieval

Relevant to:

• long-context RAG retriever selection
• multilingual document search

Failure modes it exercises:

• retrieval failure on long documents and across languages

Method and its limits

Multilingual long-document retrieval evaluation introduced with the M3-Embedding model, which supports dense, multi-vector, and sparse retrieval over inputs from short sentences to documents of up to 8,192 tokens.

• The dataset was introduced by the authors of a model evaluated on it, so it is not an independent benchmark.
• The construction of MLDR itself is not described in the primary abstract reviewed here, so corpus provenance and query design are not established.

Dataset

Dataset

Multilingual Long-Document Retrieval

Type

long-document retrieval evaluation set

Freshness

aging

Benchmark release

2024-02-05

The primary paper registers more than 100 working languages and input lengths up to 8,192 tokens for the accompanying model. It does not register MLDR's own size or language count in the abstract reviewed here. The dataset card establishes 13 typologically diverse languages and 493,709 corpus documents, built by sampling lengthy articles from Wikipedia, Wudao and mC4 and generating one question per sampled paragraph with GPT-3.5.

How to read this score

Useful as a long-document retrieval signal, but because the dataset and a leading model share authorship, cross-model comparisons should be treated cautiously.

Similarity to real tasks: High — Established from the primary source: MLDR is the multilingual long-document retrieval evaluation introduced alongside the M3-Embedding model, which supports inputs up to 8,192 tokens across more than 100 working languages. Retrieving from long documents in many languages closely matches production RAG over real corpora. The specific corpus and query construction are not described in the abstract reviewed here.

Data contamination risk: High — Corpus provenance is now established from the dataset card: the documents are sampled from Wikipedia, Wudao and mC4. All three are common pretraining corpora, so overlap with a model's training data should be treated as likely unless a score source documents decontamination.

Benchmark gaming risk: Medium — A dataset published alongside a model by the same authors carries an inherent risk that the model is advantaged by construction, independent of any deliberate optimisation.

What you still need to test yourself

• Re-evaluate on your own long documents with your own chunking strategy.
• Compare against a retriever not authored alongside the benchmark.

This benchmark supports decisions about:

• Screen retrievers for long-document handling beyond typical chunk sizes.

Limitations

• Not an independently constructed benchmark.
• Dataset construction is not documented in the source registered here.

Primary source reviewed 2026-10-11. The M3-Embedding paper was first submitted in February 2024 and has been revised since.

What this evidence does not cover

• Only one model is currently scored on this page, so no comparison is supported.

Sources

• M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation Chen, Xiao, Zhang, Luo, Lian, Liu · accessed 2026-10-11Supports the model's 100-plus working languages, its dense, multi-vector and sparse retrieval functionalities, and its 8,192-token input range; it does not describe MLDR's own construction and does not support any model score shown here.
• MLDR dataset card Xiao et al. (M3-Embedding authors) · accessed 2026-10-25The dataset card published under the M3-Embedding author's account and citing that paper. It establishes MLDR's language count, corpus size and construction method: 13 typologically diverse languages and 493,709 corpus documents, sampled from Wikipedia, Wudao and mC4 with one question generated per sampled paragraph. It does not establish any model's retrieval score.

How this benchmark is scored

Primary source: https://arxiv.org/abs/2402.03216

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to MLDR Leaderboard.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path