MIRACL Leaderboard — AI Model Scores

What this benchmark measures

• multilingual ad hoc retrieval
• monolingual ranking quality

Relevant to:

• multilingual search system selection
• retriever evaluation beyond English

Failure modes it exercises:

• retrieval quality collapse in non-English languages

Method and its limits

Monolingual ad hoc retrieval over Wikipedia in 18 languages, using more than 700k native-speaker relevance judgments across roughly 77k queries.

• The corpus is Wikipedia, so domain-specific retrieval behaviour is not represented.
• Monolingual only: queries and corpora share a language, so cross-lingual retrieval is out of scope.

Dataset

Dataset

MIRACL

Type

native-speaker relevance judgments over Wikipedia

Freshness

aging

18 languages, roughly 77k queries, and over 700k relevance judgments as registered in the primary paper. The date records paper submission, not a corpus snapshot date.

How to read this score

Ranking scores are interpretable per language and directly relevant to retrieval; aggregate figures hide wide per-language variation.

Similarity to real tasks: High — Established from the primary methodology: MIRACL provides over 700k relevance judgments for around 77k queries across 18 languages, all assessed by native speakers, for monolingual ad hoc retrieval over Wikipedia. Human relevance judgments over a real corpus closely resemble how a production retriever is evaluated. The corpus is Wikipedia rather than a domain corpus.

Data contamination risk: Medium — The corpus is public Wikipedia and the judgments are published. Retrieval benchmarks are less exposed to answer memorisation than closed-answer benchmarks, but no run-specific audit is registered.

Benchmark gaming risk: Low — Relevance judgments are human-assessed and the metric is standard retrieval ranking, which limits shortcut optimisation; index and preprocessing choices still affect reported numbers.

What you still need to test yourself

• Re-evaluate on your own domain corpus; Wikipedia results do not transfer automatically.
• Test hybrid lexical plus dense retrieval, which this benchmark does not isolate.

This benchmark supports decisions about:

• Select a retriever for languages you actually serve.
• Identify languages where retrieval quality drops sharply.

Limitations

• Not evidence of domain or enterprise retrieval quality.
• Cross-lingual retrieval is explicitly out of scope.

Primary source reviewed 2026-10-11. MIRACL is treated as an aging 2022 benchmark.

What this evidence does not cover

• Four models are currently scored on this page, and per-language results are not registered.

Sources

• Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languages Zhang, Thakur, Ogundepo, Kamalloo, Alfonso-Hermelo, Li, Liu, Rezagholizadeh, Lin · accessed 2026-10-11Supports the 18-language coverage, the roughly 77k queries and 700k native-speaker judgments, and the monolingual ad hoc retrieval design; it does not support any model score shown here.

How this benchmark is scored

Primary source: https://github.com/project-miracl/miracl

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to MIRACL Leaderboard — AI Model Scores.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path