ModelRefs / Benchmark Bot — Canonical Workflow
Benchmark Bot — Canonical Workflow
Benchmark Bot: provisional AI workflow implementation reference with candidate models, providers, tools, and architecture.
Overview
Benchmark Bot is a conversational RAG application over the ModelRefs canonical benchmark graph. Users can ask comparison questions across models and benchmarks, request score lookups with primary-source citations, and interrogate methodology differences between evaluation frameworks. The system retrieves structured data from the benchmark registry, generates natural-language answers grounded in canonical scores, and cites the source leaderboard or paper for every number it reports. Unsupported comparisons — where no canonical data exists — are clearly surfaced rather than synthesised.
Implementation profile
| Category | llms |
|---|---|
| Implementation maturity | production |
| Primary use cases | rag, reasoning |
| Deployment options | managed-api, hybrid |
| Architectures | serverless-api, managed-container, self-hosted-cluster |
Candidate models with published references
- BGE-M3
- GPT-5
- GPT-5 Mini
- Claude Sonnet 4.5
- Claude Opus 4
- Gemini 3 Pro
- Llama 4 Scout
- DeepSeek R1
- Mistral Large 2
- Command R+
- o3
- o4 Mini
Coverage means the model is a candidate worth evaluating for this workflow, not a ranking or a recommendation. Models whose reference pages are still in review are omitted.
Benchmarks relevant to this workflow
- τ-bench — Tool-augmented agent benchmark on retail & airline customer-service tasks.
- GAIA — General AI assistant benchmark — multi-tool real-world questions.
These are the benchmarks whose task shape bears on this workflow. Each one only describes its own protocol and date, so confirm the harness matches your workload before treating a score as evidence.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Benchmark Bot — Canonical Workflow.