Langfuse — AI Glossary

Reference details

Related terms

• LangSmith
• Distributed Tracing (LLM)
• Arize Phoenix

How it works

1. An SDK wraps the call sites and emits a trace per request, with nested spans for retrieval, each model call and each tool invocation.
2. Spans carry inputs, outputs, token counts and latency, so the cost and time of a single request can be attributed to the step that caused it.
3. Prompts are stored and versioned server-side, and the SDK fetches by label, which is what lets a prompt change ship without a code deploy — and be rolled back the same way.
4. Saved production traces become dataset items, so the cases that failed in the wild are exactly what later changes are evaluated against.
5. Evaluations run over a dataset with graders or an LLM judge and are recorded per version, so two prompt versions are comparable on the same inputs.

Example: What span nesting buys when latency is the complaint

A request takes 4.2 seconds and users call it slow. The trace splits it: embedding 0.1s, vector search 0.2s, reranking 1.4s, generation 2.5s. Removing reranking would recover 1.4 seconds, a third of the total, and the generation step is the floor unless the model or output length changes. Without the nesting all that exists is 4.2 seconds and a guess — the breakdown is the whole value, and it is arithmetic rather than insight.

Commonly confused with

One of several LLM observability platforms with heavily overlapping feature lists — tracing, evaluation, prompt management, cost analytics — which makes the comparison a deployment question rather than a feature one. The distinguishing axis here is that it is open-source and self-hostable, so it is usually the answer when trace data cannot leave your infrastructure. Compare with LangSmith, Braintrust and Phoenix on that basis first.

When to use it

Reach for it when:

• Tracing is needed and data residency or self-hosting is a requirement a managed platform cannot meet.
• The stack spans several frameworks, so instrumentation must not be tied to one of them.
• Prompt changes should ship and revert independently of application deploys.

Reach for something else when:

• A managed platform is acceptable and nobody wants to operate the dependency.
• Volume is low enough that application logs answer the questions being asked.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Langfuse — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path