ModelRefs / LLM Observability — Canonical Workflow
LLM Observability — Canonical Workflow
LLM Observability: provisional AI workflow implementation reference with candidate models, providers, tools, and architecture.
Overview
LLM observability is the end-to-end monitoring stack for a deployed LLM application — traces, evals, cost tracking, drift detection, and incident response so quality or cost regressions surface before users notice them.
Use this page to check what an observability stack needs to cover — tracing, evaluation, cost, drift, and incident workflows — and review the related tool, architecture, and evaluation-pipeline references before implementing one.
This workflow's representative-workload evaluation protocol has been drafted, but ModelRefs does not yet have recorded run evidence confirming detection and attribution across prompt, model, provider, and deployment changes. Treat readiness or completion claims as provisional until validated on your own workload.
Implementation profile
| Category | llms |
|---|---|
| Implementation maturity | enterprise |
| Primary use cases | reasoning |
| Deployment options | managed-api, hybrid |
| Architectures | serverless-api, managed-container, self-hosted-cluster |
What this evidence does not establish
- No target service objectives, task-quality rubric, sampling policy, sensitive-field policy, retention period, or alert threshold is registered.
- No measured trace coverage, telemetry overhead, alert precision, incident detection time, or regression sensitivity is attached.
Candidate models with published references
- BGE-M3
- GPT-5
- GPT-5 Mini
- Claude Sonnet 4.5
- Claude Opus 4
- Gemini 3 Pro
- Llama 4 Scout
- DeepSeek R1
- Mistral Large 2
- Command R+
- o3
- o4 Mini
Coverage means the model is a candidate worth evaluating for this workflow, not a ranking or a recommendation. Models whose reference pages are still in review are omitted.
Benchmarks relevant to this workflow
- τ-bench — Tool-augmented agent benchmark on retail & airline customer-service tasks.
- GAIA — General AI assistant benchmark — multi-tool real-world questions.
These are the benchmarks whose task shape bears on this workflow. Each one only describes its own protocol and date, so confirm the harness matches your workload before treating a score as evidence.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to LLM Observability — Canonical Workflow.