Guides  /  Models and routing

What does it take to train a smaller model on your own data?

Answer

Training a smaller model starts from an open-weight base model. Graded records of the work supply what the model learns, and the training runs on capacity inside the environment that already holds the traffic. Reinforcement learning and distillation produce the specialized model from those inputs.

Akka Optimize runs inference, training, grading and scoring on the same runtime as the agents. The trace an agent emitted is the record used to grade, to train, and to serve.

The mechanism

Distillation moves a class of work from a large model to a small one.

Distillation is training a small model on the graded outputs of a larger one until the small model reproduces them on one class of work. Reinforcement learning supplies the score, so the training rewards the responses that passed their criteria and leaves the failures unrewarded.

Schemas from Akka agents feed that training. An agent declares the shape of what it produces, so the records arrive structured and the grading has something exact to score against.

Synthetic records extend the corpus where production is thin. A class of work that appears rarely can be supplemented with generated cases carrying the same structure, graded by the same criteria as the production records.

What the loop requires

Each input is something the running system already holds.

An open-weight baseTraining starts from an open-weight model, which is what makes the resulting weights yours to keep, to move and to serve.
Graded recordsThe evaluations that already decide whether a response was acceptable to serve supply the score the training reads.
Your own environmentTraining runs where the traffic already is, inside your cloud, datacenter or Kubernetes infrastructure.
Serving capacityThe specialized model is served on the runtime that routes to it, and the routing policy decides how much traffic it takes.
A promotion gateA candidate is measured against the model in service on live traffic before it serves any of it.
Where the gain comes from

A model that does one class of work needs less capability to do it.

A foundation model carries capability for every task anyone might ask of it, and every call pays for that breadth. Narrowing a model to one class of work is what removes the cost.

Specialized models trained this way execute faster and consume up to 80% fewer tokens than foundry models, and accuracy on the class they were trained for rises. Swiggy cut prediction latency from 144ms to 71ms while reducing token consumption 22%.

The training reads the same interaction record Akka Verify uses for governance, so cost and compliance are answered from one record.

Questions

Related questions.

Does this require a machine-learning team?

Training, grading and serving are capabilities of the platform, and the inputs are the traffic and evaluations the agents already produce. Choosing the class of work and setting its criteria stay with your team.

Where does the training run?

Inside your own environment, on the same runtime as the agents. Inference, training, grading and scoring share that runtime, so the loop closes without the data moving.

What is the base model?

An open-weight model chosen for the class of work being trained. Starting from an open-weight base is what keeps the tuned weights portable.

Does the model keep improving after it ships?

The loop runs continuously. New production traffic is graded as it arrives. The grades train the next candidate, and the gate decides whether that candidate replaces the model in service.

Is Akka hosting models for other customers?

Akka is not a public model-serving marketplace and does not pretrain foundation models. The work is reinforcement learning, fine-tuning and distillation on your traffic, for your system.

On Akka. Reinforcement learning, distillation and on-runtime serving are capabilities of Akka Optimize. Customer figures. Swiggy results were published by Swiggy engineering.