Guides  /  Streaming and flow control

What happens when an agent produces work faster than the consumer can take it?

Answer

The excess has to go somewhere. Buffering holds it until the process runs out of memory. Dropping discards it and loses the work. Backpressure holds the producer to the rate the consumer can sustain.

Backpressure is the response that keeps the work. The consumer reports how much it can take, and the producer sends no more than that.

The mechanism

Rate mismatches are everywhere in an agentic system.

A model streams tokens faster than a guardrail can evaluate them. An agent fans out to five sub-agents that each call a tool. A consumer reads an event stream and writes to a database that takes a few thousand writes a second.

Each is a producer running at a different rate from its consumer. In a prototype the difference never shows, because no buffer fills. In production it decides whether the system degrades or stops.

Without it

A system that has no answer picks one by accident.

The first accidental answer is to buffer. Work accumulates in memory, no bound was chosen, and the process runs until the allocator refuses. The kill takes every in-flight task with it and names none of them.

The second is to drop. Dropping is survivable for a metrics sample and unacceptable for a customer request, and the code doing the dropping rarely knows which one it holds.

Retries make both worse. A timeout produces a retry. The retry adds load to a system already behind. The added load produces more timeouts.

WITHOUT BACKPRESSURE producer buffer grows without a bound consumer slower WITH BACKPRESSURE consumer reports its capacity producer bounded buffer consumer
The requirement

Backpressure has to reach every stage.

A demand signalThe consumer states how much it can accept. The producer sends that much and no more, so the rate is set by the slower party.
Bounded buffersEvery queue has a limit that somebody chose. An unbounded queue is a memory leak with a scheduling policy.
End-to-end propagationPressure crosses every component boundary. A single stage that absorbs it silently returns the system to buffering.
Non-blocking waitsA producer held back releases its thread. Blocking a thread to wait converts a rate problem into a thread-pool problem.

Akka propagates demand across component boundaries as a property of the runtime, so a slow consumer slows its producer without either holding a thread. Agents, streams, endpoints, and consumers all participate in it.

Questions

Related questions.

Is this what a queue is for?

A queue moves the buffer somewhere you can see it and operate it, which helps. A queue does not tell the producer to slow down, so a queue that fills faces the same choices a buffer faces. Backpressure is the signal, and a queue is one place to put work while the signal travels.

What about rate limits from the model provider?

A rate limit is backpressure arriving from upstream. The system absorbs it by slowing the work that depends on it, or it converts the limit into a retry storm. Absorbing a rate limit requires the same demand signal, applied in the other direction.

Does backpressure make the system slower?

Backpressure holds the system to the rate of its slowest stage. Throughput is set by that stage either way, and backpressure decides whether you reach the limit deliberately or discover it as an outage.

What breaks first without it?

Memory fails first. The component that dies is the one holding the buffer, which sits several hops upstream from the producer that outran its consumer.

On Akka. Stream processing and flow control are described in the Akka SDK documentation.