Routing to a smaller model makes sense for a class of work where the smaller model reaches the graded score the model in service reaches. Production traffic that has been graded is what identifies that class.
Akka Optimize grades every run and compares models class by class. A class moves when the candidate holds the quality threshold and consumes fewer tokens.
A single agent extracts fields from documents, triages incoming tickets, and reasons over contracts. Grading is scoring each response against the criteria set for its class of work, and the three classes carry different criteria.
The graded record makes the comparison concrete. A candidate model runs one class of work. Its responses are scored by the criteria that scored the model in service, and the two scores are read against each other.
A model trained on your own traffic runs faster and consumes up to 80% fewer tokens than a foundry model on the class of work it was trained for. The saving applies to the classes that moved, so the total follows from how much of the traffic those classes represent.
The frontier model keeps the classes that need it, and the routing policy names the cheapest model that meets the threshold for each remaining class. Swiggy cut prediction latency from 144ms to 71ms while reducing token consumption 22%.
The volume follows from how narrow the class is. Coverage decides more than volume, since ten thousand near-identical requests measure one case ten thousand times.
The threshold for the class decides. A class where an error is cheap can accept a candidate scoring slightly lower in exchange for a large reduction in tokens. A class where an error is an incident holds the candidate to the incumbent score.
Routing is per class of work, so one agent can send its extraction work to a specialized model and its contract reasoning to a frontier model on the same request path.
A class with no production history has no graded record to compare against. Synthetic records carrying the same structure extend the corpus, and they are graded by the same criteria.
The criteria come from the evaluations that already decide whether a response is acceptable to serve, and controls in the signed matrix bind the ones that governance requires.
The routing policy names the destination.
Routing traffic to the best open-weight model and training smaller ones on your data.
Cost per verified task, and what it prices.
