Helicone — AI Glossary

Reference details

Related terms

• LangSmith
• AI Gateway
• Distributed Tracing (LLM)

How it works

1. Requests are routed through a proxy by changing the client's base URL, so adoption requires no SDK instrumentation.
2. The proxy records request and response pairs with token counts and latency, and attributes them via headers for user or feature.
3. Because it sits in the request path it can act as well as observe — caching, rate limiting and retries happen there rather than in application code.
4. That position is also the cost: the proxy is a dependency on the critical path, and its availability becomes yours.
5. It sees the request as sent, so internal steps the application performed between calls are invisible — there is no span nesting to read.

Example: What a proxy sees, and what it cannot

A request takes 4 seconds. The proxy logs one model call of 2.4 seconds, so 1.6 seconds happened somewhere it cannot see — retrieval, reranking, application logic. It can attribute the model call precisely and the rest not at all. An SDK-instrumented tracer with nested spans would break down all 4 seconds. The trade is one line of integration against visibility into everything that is not a model call.

Commonly confused with

A proxy rather than an SDK integration: requests route through it and are logged, so adoption is a base-URL change instead of instrumentation. That is its whole trade — near-zero integration cost, at the price of putting a hop in the request path and a third party in front of every prompt. Tracing-first tools sit beside your code instead of in front of it.

When to use it

Reach for it when:

• Logging, cost attribution and caching are wanted with minimal integration effort.
• Code cannot easily be instrumented, such as third-party or legacy callers.
• Caching and rate limiting belong in the request path rather than in application code.

Reach for something else when:

• Multi-step pipelines need per-stage attribution, which requires span-level instrumentation.
• An extra dependency on the critical path is unacceptable.
• Evaluation and experiment comparison are the main requirement.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Helicone — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path