ModelRefs / Cost per Successful Task — AI Glossary

Cost per Successful Task — AI Glossary

Total spend divided by the number of outcomes that actually met the acceptance bar — not by requests or tokens.

Overview

Per-token and per-request pricing hide retries, failed attempts, human rework, and answers that were produced but rejected. Cost per successful task puts the denominator where the value is: define what "successful" means (validated schema, grounded answer, accepted review), then divide total cost — including retries, reranking, and review time — by that count. It matters because a cheaper model that fails more often, or needs three attempts to produce valid output, can be more expensive per completed job than a pricier model that succeeds first time. Common mistakes: comparing candidates on list price alone, excluding retry and review cost, and changing the success definition between candidates.

Reference details

Topic	evaluation
Also known as	cost per successful outcome, effective task cost
Last reviewed	2026-06-24

How it works

Success is defined first, as a checkable predicate — a payload that validates, an answer a reviewer accepts, a patch whose tests pass — and frozen before any candidate is measured.
Every cost that the attempt causes is attributed to it: the first call, every retry, any reranking or verification pass, and the human time spent on failures.
The denominator is the count of outcomes meeting the predicate, not the count of requests, so a request that produced output nobody could use contributes cost and no credit.
The resulting per-outcome figure is therefore a ratio of two different populations — all cost incurred over only the outcomes that qualified — which is what makes it diverge from any per-call price.
Failure handling is priced separately from retrying, because a failure that reaches a human costs a different order of magnitude from one the pipeline catches.
Candidates are then compared on the single resulting figure, which is where list-price ordering frequently reverses.

Example: The cheaper model that costs more

Two candidates. Model A costs $0.60 per attempt and produces acceptable output 70% of the time; Model B costs $1.50 per attempt and succeeds 94% of the time. On list price A is 1.50 / 0.60 = 2.5 times cheaper, and that is the number most comparisons stop at. Now run 100 tasks through each, one attempt apiece, and send the failures to a reviewer. A spends 100 x $0.60 = $60 and produces 70 acceptable outcomes, leaving 30 failures; B spends 100 x $1.50 = $150 and produces 94, leaving 6. If reviewing a failure takes six minutes at a $75 loaded hourly rate, each costs $7.50, so A pays 30 x $7.50 = $225 and B pays 6 x $7.50 = $45. Totals are $285 and $195, and the denominators are 70 and 94 rather than 100 — which is the whole move. Cost per successful task is 285 / 70 ≈ $4.07 for A and 195 / 94 ≈ $2.07 for B, so B is cheaper by a factor of about 1.96 having looked 2.5 times more expensive. The crossover is calculable rather than a matter of judgement. Setting (60 + 30R) / 70 equal to (150 + 6R) / 94 gives 2,400R = 4,860, so the break-even review cost is exactly $2.025 per failure: below that A wins and above it B does. At $7.50 a failure the answer is B by a wide margin, and at $1 a failure it is A.

Commonly confused with

Cost per successful task is not cost per request, and the difference is the denominator rather than the arithmetic. Per-request and per-token figures measure what the provider charges; this measures what the work costs. They diverge exactly in proportion to the failure rate, so the two agree for a system that always succeeds and disagree most for the cheap, unreliable candidate that per-token pricing flatters. It is also not total cost of ownership, which folds in engineering, infrastructure and amortised build cost over a period. This is a unit economic — one number per completed outcome — which is what makes it usable for choosing between candidates rather than for budgeting a programme.

When to use it

Reach for it when:

When comparing models or prompts whose reliability differs, which is where list price and true cost diverge.
Wherever failures reach a human, since review time dominated both candidates' bills above.
With the success predicate frozen before measurement, because moving it between candidates invalidates the comparison.
To find the break-even failure cost, which converts an architecture argument into a threshold.

Reach for something else when:

With the success definition changed between candidates, which makes the denominators incomparable.
Excluding retry and review cost, which is the entire reason the figure differs from list price.
On systems where success is not checkable, since an unverifiable predicate cannot be counted.
As a budget projection, for which total cost of ownership over a period is the right instrument.

Where this appears on ModelRefs

create representative workload evaluations

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Cost per Successful Task — AI Glossary.

Frequently asked questions

What is Cost per Successful Task?

Total spend divided by the number of outcomes that actually met the acceptance bar — not by requests or tokens.

Is Cost per Successful Task the same as cost per successful outcome?

Yes — cost per successful outcome, effective task cost are common aliases for Cost per Successful Task.

What concepts are related to Cost per Successful Task?

Closely related concepts include cost per request, cost per token, inference cost, p99 latency.