MGSM Leaderboard

What this benchmark measures

• multilingual mathematical reasoning
• chain-of-thought transfer across languages

Relevant to:

• multilingual reasoning screening
• language-coverage checks for maths-adjacent tasks

Failure modes it exercises:

• reasoning degradation in non-English languages

Method and its limits

250 GSM8K problems manually translated into ten typologically diverse languages, evaluated with chain-of-thought prompting on final-answer accuracy.

• 250 problems per language is a small sample, so single-point differences are not meaningful.
• Translated English problems do not capture culturally specific or language-native reasoning.

Dataset

Dataset

Multilingual Grade School Math

Type

translated grade-school maths word problems

Freshness

aging

Benchmark release

2022-10-06

250 problems in ten languages as registered in the primary paper. The date records paper submission, not a knowledge cutoff.

How to read this score

Accuracy is interpretable per language. Because the set is only 250 items, differences of a few points are within sampling noise.

Similarity to real tasks: Low — Established from the primary methodology: MGSM is 250 grade-school maths problems manually translated from GSM8K into ten typologically diverse languages. It isolates whether multi-step arithmetic reasoning survives translation. It does not reproduce production multilingual work, because the problems are short, self-contained, and carry a single numeric answer.

Data contamination risk: High — MGSM derives from GSM8K, which is public and heavily represented in training and tuning corpora. No run-specific overlap audit is registered for the scores shown here.

Benchmark gaming risk: Medium — Prompt language, chain-of-thought format, and answer extraction all affect the score, and ModelRefs registers no per-score protocol detail.

What you still need to test yourself

• Test the specific languages your users write in, which may not be among the ten covered.
• Evaluate native-language tasks rather than translated English ones.

This benchmark supports decisions about:

• Check whether reasoning quality holds outside English.
• Identify languages where a candidate model degrades sharply.

Limitations

• A 250-item set cannot separate closely matched models.
• Not evidence of multilingual quality outside arithmetic reasoning.

Primary source reviewed 2026-10-11. MGSM is treated as an aging 2022 benchmark.

What this evidence does not cover

• Only two models are currently scored on this page, and per-language breakdowns are not registered.

Sources

• Language Models are Multilingual Chain-of-Thought Reasoners Shi, Suzgun, Freitag, Wang, Srivats, Vosoughi, Chung, Tay, Ruder, Zhou, Das, Wei · accessed 2026-10-11Supports the 250-problem, ten-language construction and its derivation from GSM8K; it does not support any model score shown here.
• MGSM dataset release Google Research · accessed 2026-10-25The official MGSM data release accompanying the paper. It establishes the published problem set and language list; the repository is archived, so it does not establish a maintained benchmark or any per-language model result.

How this benchmark is scored

Primary source: https://arxiv.org/abs/2210.03057

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to MGSM Leaderboard.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path