Stop Sequence — AI Glossary

Reference details

Related terms

• Prompt Template
• Chat Completion
• System Message

How it works

1. The caller supplies one or more strings, and generation halts as soon as any of them is produced. Providers typically accept a small number of them.
2. Matching happens on the decoded text rather than on token ids, so a stop string that straddles a token boundary still triggers — which is why the feature works on arbitrary strings and not only on single tokens.
3. The matched string is excluded from the returned output, so the caller receives the content without the delimiter it chose.
4. The response reports why generation ended, and distinguishing a stop-sequence halt from reaching the token limit is the difference between a complete result and a truncated one.
5. The economics are direct: generation is billed per output token, so halting at the end of the content rather than at the maximum saves everything in between.
6. The hazard is equally direct. A stop string that can legitimately appear inside the content silently truncates it, and the result looks like a short answer rather than an error.

Example: What the token limit costs when nothing stops generation

A classification prompt returns a one-word label — call it 50 tokens including formatting. With `max_tokens` set to 1,000 and no stop sequence, a model that continues past the label with an explanation generates up to 1,000, and you are billed for 1,000: a factor of 1,000 / 50 = 20 on output tokens. At 1,000,000 requests and 15 units per million output tokens, that is the difference between 1,000,000 x 50 = 50,000,000 tokens, or 50 x 15 = 750 units, and 1,000,000 x 1,000 = 1,000,000,000 tokens, or 1,000 x 15 = 15,000 units — 14,250 units a month of output nobody reads. Latency moves with it, since 1,000 tokens at 100 tokens per second is 10 seconds against 0.5. The counterpart risk is worth pricing too: if the stop string can occur inside valid content, say a delimiter that also appears in code the model is quoting, then every response containing it is silently cut short, and because the response simply ends there is nothing in the output that signals the truncation. The finish reason is the only place it shows.

Commonly confused with

A stop sequence is not a token limit, and the response's finish reason is where the difference becomes visible. A stop sequence means the model produced the boundary you asked for, so the content is complete. A token limit means generation was cut off wherever it happened to be, so the content is truncated and may end mid-structure. Both return a shorter response and only one of them is a result. Code that ignores the finish reason treats a truncated JSON object as a complete one, which is the common way this distinction causes a bug rather than a cost.

When to use it

Reach for it when:

• Structured extraction with a closing delimiter, where the boundary is well defined and cannot occur in the payload.
• Classification and short-answer tasks, where the saving against a generous token limit is a large multiple.
• Prompt chaining, where each stage's output must end cleanly for the next to parse it.
• Checked alongside the finish reason, so a stop and a truncation are never confused.

Reach for something else when:

• With a string that can appear in legitimate content, which truncates silently and looks like a short answer.
• As the only bound on generation — a token limit is still needed for the case where the stop string never appears.
• For structure that needs validating rather than delimiting, where a schema-constrained decoder guarantees what a stop string only marks.
• Without reading the finish reason, since that is the only signal distinguishing a complete response from a cut-off one.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Stop Sequence — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path