OpenAI-Compatible API — AI Glossary

Reference details

Related terms

• Chat Completion
• Model Routing
• AI Gateway

How it works

1. The contract is a POST to a chat completions path carrying a JSON body with a model string, a messages array of role-and-content objects, and sampling parameters.
2. The response envelope is equally specified: a choices array holding the message, a finish reason, and a usage object with token counts.
3. Authentication is a bearer token in a standard header and the route sits under a version prefix, so a client needs only a host and a credential as constructor arguments rather than per-provider code.
4. Streaming is server-sent events, with each chunk a data line carrying a partial delta and the stream closed by a sentinel value rather than by the connection ending.
5. Unrecognised request fields are conventionally ignored rather than rejected, which is the detail that makes compatibility deceptive: a request asking for a feature the server does not implement succeeds, and the feature simply does not happen.
6. The contract has no capability-discovery call, so a client cannot ask a server what it supports and has to infer it from whether a request behaved as intended — which is why differences surface as wrong output rather than as an error.

Example: Portable until it is not, and the fraction is countable

Switching provider is base_url and model string. What does not travel is everything at the edges: tool-calling dialects, structured-output support, reasoning parameters, token accounting and error semantics. The core path ports cleanly; the features you actually rely on are where portability quietly stops. The useful move is to count rather than estimate. Audit the call sites: suppose 24 of them exist and 18 send only a model string, a messages array and a temperature, while 6 pass tool definitions or request a schema-constrained response. A provider swap is then a configuration change for 18 and a rewrite for 6 — 6 / 24 = 25% of the surface. That number is worth knowing before the migration rather than during it, and it is the number an abstraction over the model call is actually buying down.

// The 18: portable by construction.
client.chat.completions.create({ model, messages, temperature })

// The 6: provider-specific under a shared envelope.
client.chat.completions.create({ model, messages, tools, tool_choice })
client.chat.completions.create({ model, messages, response_format: { type: "json_schema", json_schema } })

Commonly confused with

Compatible is not equivalent. The label describes the request and response envelope, not behaviour. Two compatible endpoints can differ in tokenisation, default sampling, rate-limit semantics and which optional fields they honour — none of which the shared schema constrains. The consequence is that a provider swap carries the same risk as a model swap and deserves the same evaluation run, because the thing that changed is not the code you can diff.

When to use it

Reach for it when:

• You want to keep provider substitution cheap
• Evaluating several models against the same client code
• Local and hosted inference should share one code path
• Behind a gateway, which is where the dialect translation belongs rather than in application code

Reach for something else when:

• Assuming identical output across providers on the same input
• Relying on an optional field without checking each provider honours it
• You need a provider-specific capability, where the abstraction costs more than it saves
• Without auditing which call sites are actually portable — the 25% above is the figure that decides the migration

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to OpenAI-Compatible API — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path