A rolling update replaces nodes while runs are part way through multi-step work. Whether that work survives depends on where its progress was recorded when the node went away.
Akka reschedules the run onto a healthy node and resumes it from its last completed step. The deployment does not need a maintenance window, and the run does not need to start again.
Draining a node stops new work reaching it and gives running work a grace period to finish. Agent runs are frequently longer than that grace period, because a run waiting on a model call or a human decision measures in minutes or days.
When the grace period expires the process ends with work still in progress. Nothing about that differs from a crash, so a system that survives a crash survives a deployment, and a system that loses work on a crash loses it on every release.
Teams that lose in-flight work on deployment develop habits to avoid it. A team in that position releases at night, drains slowly, pauses intake ahead of a release, and keeps a runbook for the work that was lost anyway. Each of those is a cost paid on every release.
Norwegian Cruise Line deployed at midnight to avoid disruption before rebuilding on Akka. The company now completes 65 deployments in 12 months with zero rollbacks.
Long enough for short work and never long enough for a run holding a human decision. Setting it longer delays every release, which is why the runtime resumes work instead of waiting for it.
The runtime records whether the call completed. A resumed run reads that record and either skips the call or makes it, and the decision does not depend on a timeout.
A queue holds requests that have not started. The work in this guide has already started, so the queue has nothing to replay. Resuming requires the state of the run, which the queue never held.
Active-active replication carries the state to another region. Recovery time is under a minute and no committed state is lost.
