Online evaluation keeps grading production traffic after the promotion, so a fall in quality appears as a change in the graded score. The routing policy returns the affected class of work to the previous model.
The gate that promoted the model is where the model is removed. Rollback is a change to the routing policy, so moving the traffic back requires no release.
A promotion is decided on a shadow run, which measures a candidate against traffic captured over a period. Traffic changes after that period, through a new document format, a new product, or a phrase that was rare last quarter.
Online evaluation runs against the interaction log while the system serves, so the score for each class of work is current. A fall shows against the score that class already met.
The previous model stays available as a destination the policy can name, so restoring it is a change to the policy. Agents routing through it reach the previous model on the next request.
The grades, the fall and the routing change are written to the interaction record. Reading that record afterwards shows the sequence and where the fall began.
Akka Verify scopes the review to what changed. A prompt change is a half-day cycle. A foundation-model swap runs for weeks and is budgeted before it starts.
Detection follows the evaluation cadence set for the class of work. Event evaluations fire on runtime events without blocking, and periodic evaluations aggregate over the interaction log on a schedule.
The weights stay in your environment. The routing policy stops naming the model as a destination, and the training loop can produce a further candidate from the traffic that exposed the fall.
A promotion does not require retiring the model it replaced. Keeping the previous model available as a destination is what makes the routing change a rollback.
A fall in graded score can come from the model, the prompt, a tool that changed, or the traffic itself. The interaction record holds each of those, so the cause is read from the record.
The kill switch halts in-flight and future tool and LLM calls and moves the system to a defined safe state. Oversight personnel exercise it, and the runtime records that it was exercised.
