Guides  /  Cost and scale

What changes between 5 agents and 100?

Answer

Concurrency, partial failure, and retry amplification decide what happens at 100 concurrent runs. A prototype running 5 produces too little of each for the difference to appear during development.

Akka supplies concurrency, supervision, backpressure, and recorded tool execution as properties of the runtime. The behaviour that decides the outcome at 100 runs is already in place at 5.

The mechanism

A shared dependency serves every concurrent run, and its capacity runs out first.

Five agents running at once touch a shared dependency five times. The connection pool has capacity free and the model provider stays inside its rate limit. Every tool call is answered. Nothing queues, so no code that handles queueing runs.

One hundred concurrent runs contend. Connections run out, the provider returns a rate-limit response, and calls that answered in a second answer in twenty. The system now depends on behaviour never reached while it was being built.

A FEW CONCURRENT RUNSorchestratordependencyinside its limitevery run completesAT PRODUCTION CONCURRENCYorchestratordependencyat capacitysome runs completeeach timeout produces a retry
What contention produces

Partial failure becomes the normal case.

ContentionA shared dependency serves one hundred callers at once. Requests queue at the connection pool, at the rate limiter, or at the tool, and the queue is where the latency comes from.
Partial failureSome runs succeed while others fail against the same dependency in the same moment. Handling written for total success and total failure has no branch for that state.
Retry amplificationA timeout produces a retry. The retry adds load to a dependency that is already behind, and the added load produces more timeouts.
Repeated side effectsA retried step calls its tool a second time unless something recorded that the first call executed. At five concurrent runs the duplicate is rare enough to pass testing.
Head-of-line blockingOne slow run holds a slot that other runs need. Throughput falls while every component reports itself healthy.
What this changes

The behaviour has to exist before the load arrives.

A prototype that works at five runs is evidence about five runs. The code that handles contention, partial failure, and duplicate execution gets written after the first production incident. Rework caused by earlier architectural choices consumes 20 to 40% of engineering capacity.

Akka provides that behaviour before the load arrives. Each agent instance processes its own messages one at a time, so its state needs no locking. Backpressure holds a fast producer to the rate its slower consumer can sustain. The runtime records whether a tool call executed, so a retry skips a call that already ran.

Esdiac rebuilt a distributed communications and payments network on Akka after the previous architecture stalled at 5,000 customers. The rebuilt system carries tens of thousands of users at 65% lower infrastructure cost.

Questions

Related questions.

What fails first as concurrency rises?

The shared dependency fails first. Each agent is doing the same work it did at five runs, and the connection pool, the rate limit, or the tool behind them is what runs out.

Does adding instances solve it?

More instances raise the load reaching the shared dependency. Capacity helps when the agent process is the constraint, and it makes contention at the dependency worse.

What happens when the model provider rate-limits the system?

A rate limit is pressure arriving from upstream. The runtime absorbs it by slowing the work that depends on that provider. Code that answers a rate limit with an immediate retry converts the limit into a retry storm.

Does the prototype code survive the move?

The agent logic survives. The scaffolding built around it to compensate for a missing runtime gets replaced.

How far does this go?

Akka scales elastically to 10 million agentic transactions per second with scale-to-zero. Swiggy holds 5,000 predictions a second on a single model.

On Akka. Actor-based concurrency, supervision, and flow control are described in the Akka SDK documentation. Customer figures. Esdiac and Swiggy results are reported by those customers.