Arize Phoenix — AI Glossary

Reference details

Related terms

• Langfuse
• Distributed Tracing (LLM)
• RAGAS

How it works

1. Instrumentation is emitted as OpenTelemetry spans, so traces travel over a standard protocol rather than a vendor SDK.
2. That makes the collector swappable and lets LLM spans sit beside existing application traces in the same backend.
3. Retrieval spans capture the query, the returned documents and their scores, which is what allows retrieval to be evaluated separately from generation.
4. Built-in evaluators score relevance, groundedness and answer quality over captured traces, so production data becomes the evaluation set.
5. Embeddings can be projected to two dimensions, which surfaces clusters and outliers that per-request inspection does not reveal.

Example: Separating a retrieval failure from a generation failure

An answer is wrong. The trace shows the retrieval span returned three documents, none containing the needed fact, and the generation span faithfully summarised what it was given. The generator did its job; the retriever did not — so tuning the prompt would have been wasted work. Without per-stage capture both hypotheses stay alive and the usual response is to change the prompt, because that is the cheapest thing to change rather than the thing that was broken.

Commonly confused with

Another open-source entrant in the observability and evaluation category, distinguished mainly by its emphasis on retrieval and embedding analysis — visualising embedding spaces and diagnosing RAG failures. Where Langfuse and Braintrust centre on traces and evaluations respectively, this leans towards inspecting the retrieval stage, which matters if RAG is where your failures live.

When to use it

Reach for it when:

• OpenTelemetry is already the tracing standard and LLM spans should join it.
• Retrieval quality needs evaluating separately from generation quality.
• Self-hosting is required for data-handling reasons.

Reach for something else when:

• A fully managed platform is preferred and nobody wants to operate the collector.
• Prompt versioning and staged rollout are the primary need, which is prompt management.
• Volume is low enough that application logs suffice.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Arize Phoenix — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path