ModelRefs / Multimodal Assistant — Canonical Workflow
Multimodal Assistant — Canonical Workflow
Canonical Multimodal Assistant workflow: vision/audio/text models, guardrails and deployment.
Overview
A multimodal assistant accepts images, audio and text and produces grounded answers, edits or generations. The canonical stack uses a frontier multimodal model with modality-specific pre/post-processing and content safety guardrails.
Implementation profile
| Category | multimodal-models |
|---|---|
| Implementation maturity | production |
| Primary use cases | agents, extraction, ocr |
| Deployment options | managed-api, hybrid |
| Architectures | serverless-api, managed-container |
Candidate models with published references
- GPT-5
- GPT-5 Mini
- Claude Sonnet 4.5
- Claude Opus 4
- Gemini 3 Pro
- Llama 4 Scout
- o3
- o4 Mini
- Claude Sonnet 4
- Claude 3.5 Sonnet
- Claude 3 Haiku
- Gemini 2.5 Pro
Coverage means the model is a candidate worth evaluating for this workflow, not a ranking or a recommendation. Models whose reference pages are still in review are omitted.
Benchmarks relevant to this workflow
- MMMU — Massive Multi-discipline Multimodal Understanding across 30 subjects.
These are the benchmarks whose task shape bears on this workflow. Each one only describes its own protocol and date, so confirm the harness matches your workload before treating a score as evidence.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Multimodal Assistant — Canonical Workflow.