Requests Per Minute (RPM) — AI Glossary

Reference details

Related terms

• Tokens Per Minute (TPM)
• Rate Limiting
• Retry with Exponential Backoff

How it works

1. The provider counts admitted requests against a per-minute allowance scoped to a key, project or organisation, independent of how large each request is.
2. Enforcement is usually a token bucket or sliding window rather than a counter that resets on the minute, which is why a short burst can be rejected even when the minute's total would have been within budget.
3. RPM is metered in parallel with token-per-minute limits. The binding constraint is whichever exhausts first, so the shape of your traffic — many small calls or few large ones — decides which limit you actually live under.
4. Response headers expose remaining requests and the reset time, making the live headroom readable rather than inferred.
5. Exceeding it returns 429 before inference, so a rejected request costs nothing but occupies a retry slot and adds latency to the caller.
6. The client-side control is a pacer: admit at most quota/60 requests per second with a queue behind it, which converts a rejection problem into a latency problem you can measure.

Example: Which limit binds, and how to tell before you ship

An account has 500 RPM and 200,000 TPM. A chat workload averages 300 tokens per request: at 500 RPM it would consume 500 x 300 = 150,000 TPM, under the 200,000 ceiling, so RPM binds and the effective throughput is 500 requests per minute. A document workload averages 8,000 tokens: 500 requests would need 500 x 8,000 = 4,000,000 TPM, twenty times the ceiling, so TPM binds and the real limit is 200,000 / 8,000 = 25 requests per minute. Same account, same quotas, a 20x difference in request throughput decided entirely by average request size. The crossover is at 200,000 / 500 = 400 tokens per request: below it RPM binds, above it TPM binds. Knowing that one number tells you whether batching requests together will help — above 400 tokens it will not, because you are already paying for tokens, not calls.

Commonly confused with

RPM is not TPM and not concurrency, and all three are commonly collapsed into 'rate limit'. RPM counts calls per minute, TPM counts tokens per minute, and a concurrency limit caps simultaneous in-flight requests regardless of rate. A workload can sit well inside RPM and TPM and still be throttled by concurrency if its requests are long-running, because 10 concurrent streaming requests each taking 60 seconds is 10 requests per minute against a concurrency ceiling of 5. Diagnosing a 429 means reading which counter the headers say is exhausted, not assuming.

When to use it

Reach for it when:

• Capacity planning for chat and short-request workloads, where RPM is usually the binding constraint.
• Sizing a client-side pacer, since quota/60 gives the admission rate directly.
• Deciding whether request batching helps: it does when RPM binds and does nothing when TPM binds.
• Comparing providers for high-volume, low-token traffic, where RPM differences matter more than price per token.

Reach for something else when:

• As the only limit modelled. A pacer tuned to RPM alone will still produce 429s when TPM or concurrency binds first.
• To estimate cost. RPM is an admission constraint; cost follows tokens.
• For long-running streamed requests, where concurrency is the real ceiling and per-minute request counts understate the load.
• Assuming a clean per-minute reset. Sliding-window enforcement rejects bursts that a minute-boundary counter would have admitted.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Requests Per Minute (RPM) — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path