Cost per verified task is the total cost of producing an outcome that passed its evaluations, and it counts only the runs that passed.
A cheaper model that fails more often produces fewer passed runs per dollar. The token price falls, and the failed work is done again by a retry, by a larger model, or by a person.
A wrong answer consumes few tokens. The token meter records a saving and the business still pays for the outcome, because the work that answer failed to do gets done somewhere else.
Cost per verified task returns the failed runs to the total. Every token spent on every attempt sits in the numerator, and the denominator is the outcomes that passed their evaluations. A drop in pass rate raises the figure while the token price is falling.
Evaluations already decide whether a response was acceptable to serve. Reading those same evaluations as an accounting signal needs no second pipeline, because the grade and the token count are attached to the same interaction record.
Akka Optimize reads evaluated production traffic to find work a smaller model can take, trains a candidate on that traffic, and shadows live requests with it before it serves any of them. A candidate that does not hold its grade is not promoted, so the cost figure falls only when quality has been held.
Swiggy cut prediction latency from 144ms to 71ms while reducing token consumption 22%. Specialized models tuned from an open-weight base run on up to 80% fewer tokens than foundry models.
Run both against the same evaluations on the same traffic. A candidate shadows live requests without serving them, so both figures come from the same work.
The cost figure inherits the error. A grade is only as good as the criterion behind it, and a vague criterion produces a number that cannot be trusted in either direction.
Cost per verified task prices any change that moves quality and spend together. A prompt change, a routing policy, a compaction setting and a new tool are all measured on the outcomes that passed.
Token tracking and cost reporting across teams and projects are built into the platform. The figures come from the runtime that made the calls, so no ingestion-priced tool is added to collect them.
Routing, training, and the promotion gate a candidate has to pass.
The graded traffic the candidate learns from.
Cost per verified task, and what it prices.
