How We Score

Every benchmark score carries its source

A score is never stored as a bare number. Each record holds the model, the benchmark, the result, the date the run was reported, the name of the source and a link to it. A score that cannot be attributed is not published.

This is why editorial writing across the site never quotes benchmark figures. Numbers live in the governed registry where their provenance travels with them; prose that repeated them would be a second, unsourced copy that no check could keep honest.

Confidence says how the number was obtained

Confidence is a statement about evidence quality, not about how good a model is. Two values are in use, and they distinguish results we can trace to a primary publication from results that passed through an intermediary.

• 0.9 — primary source

  The figure appears in a model card, a paper, or an official benchmark leaderboard. The link in the record points at that publication.

• 0.65 — aggregate or estimated

  The figure comes from a composite index, or is vendor-reported without independent verification. It is still shown, and it is still attributed, but it is marked as the weaker evidence it is.

Verification status is tracked separately from confidence

Confidence describes the source. Status describes what has happened to the record since.

• verified — checked against the cited source
• pending — recorded, not yet checked
• disputed — the source and the figure disagree, or two sources conflict
• deprecated — superseded, kept so the history stays readable

Truth scoring governs what a surface may claim

Above individual scores, each governed surface is scored on twelve categories. Several are hard gates: if one is open, the surface cannot advance, regardless of how high the total is. The score is published with the open gates named rather than as a single figure.

• registry_backed, graph_backed — the content resolves to canonical records and real relationships
• provenance_complete — every claim has a traceable source
• calculation_verified — figures recompute from their inputs
• runtime_verified, ux_verified, search_verified, navigation_verified — the page behaves as described when exercised
• performance_verified, accessibility_verified — measured, not asserted
• methodology_displayed — the reader can see how the number was produced
• evidence_quality — the weighted strength of the sources behind it

States are earned in order

A surface moves through a fixed ladder, and nothing skips a rung. Most of the reference layer sits at provisional, which is an honest description rather than a disclaimer: it means the content is registry-backed and methodology is shown, while independent review has not happened yet.

• provisional — published, sourced, not independently reviewed
• ready_for_review — every hard gate closed, awaiting a reviewer
• independently_verified — a reviewer outside the authoring path has signed off
• verified, locked — checked and frozen against further drift
• released, monitored — in production and re-checked on a schedule

What a score is not

No ranking on this site is an opinion about which model is best. A leaderboard orders measured results on a stated benchmark, under the conditions that benchmark defines, from the sources named in each row. Two models close together on one benchmark are not meaningfully separated by it, and a benchmark with no recorded score for a model produces no row rather than a guess.

This page documents how ModelRefs frames provisional fit signals across model, provider, and workflow scores, including their inputs, evidence requirements, stated limits, and the difference between a score and a guarantee.

Use this page before treating any ModelRefs score as a ranking: it explains what inputs feed a score, what evidence gaps remain open for that entry, and where the score should not be extrapolated beyond its stated scope or protocol.

Scores are decision-support signals bounded by their inputs and evaluation protocol. They do not guarantee real-world performance and should be paired with your own representative evaluation, including failure cases, before any production decision.

• Methodology
• Benchmark methodologies
• Data freshness
• Editorial policy

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to How We Score.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path