Introducing Akka’s new Agent component

Docs

Demo

Examples

• Multi-agent planning and execution
• Chat agent with RAG
• Travel planning agent
• Medical discharge summary tagging
• IoT sensor monitoring
• Release note summarizer
• Customer service chat workflow
• Multi-source health sensor analysis
• Akka Chess
• Shopping cart
• Customer data store
• Wallet transfer
• Wallet transfer with failure compensation
• Choreographed user signup flow

Get Started

Author your first agent

Delegation with effects

A simple, chainable structure for creating instructions that are delegated to the Akka runtime to enact a variety of behaviors.

In Akka, the behavior of your agents is decoupled from their execution, enabling you to focus on defining the behavior of agents while Akka decides how the agent’s implementation will be executed. Delegation removes you from worrying about distributed systems, persistence, elasticity, or networking. With Akka’s hosted services, we use delegation to enable swapping out new, improved runtimes while your services are running without a recompilation or redeployment!

Effects include:

• declare which model will be used
• specify system messages, user messages and additional context (prompts)
• configure session memory
• define available tools whether local or remote MCP
• fail a command by returning an error
• return an error message
• transform responses from a model and reply to incoming commands

effects()  
  .systemMessage("")
  .memory(<<memory_provider>>)
  .tools(<<local_or_remote_MCP>>)
  .userMessage(question)
  .thenReply()

Intrinsic memory

Akka agents each include session memory. Session memory (usually called short-term memory) provides a durable history for agents to maintain context across multiple interactions. Memory can also be shared across multiple agents collaborating on the same goal.

effects()
  .memory(MemoryProvider.limitedWindow().readLast(5))
  .systemMessage("You are a helpful...")
  .userMessage(question)
  .thenReply();

Akka memory is kept in-memory, in-context and durably persisted automatically by the Akka runtime. The Akka runtime is tested against terabytes of data with an embedded data sharding and rebalancing within Akka clusters. Agents deployed into multiple regions have their memory automatically replicated (and persisted) between regions for failover and disaster recovery.

Akka saves user messages, intermediate tool interactions, and AI responses automatically into session memory. Developers do not need to directly invoke or manage memory. Memory can be compressed or managed against a window for managing tokens and performance.

You can also integrate third party memory stores using a MemoryProvider.

Composable

Akka agents are composable with other Akka components, which when combined enable the creation of multi-agent systems and other types of agentic systems including transactional, digital twin, edge, analytical, and autonomous.

Model providers

We have support for every major model type including those from Anthropic, Gemini, Hugging Face, OpenAI, LocalAI, and Ollama. You can add your own models with a custom ModelProvider. You can define a model in an agent from code or configuration.

effects()
  .model(ModelProvider 
    .openAi()
    .withApiKey(System.getenv("OPENAI_API_KEY"))
    .withModelName("gpt-4o")
    .withTemperature(0.6)
    .withMaxTokens(10000))
  .systemMessage("You are a helpful...")
  .userMessage(question)
  .thenReply();

Dynamic prompting

Support for both system and user messages, defined within code, configuration, or 3rd party evaluations.

You can define prompts within a template that are stored in Akka entities (durably persisted, event-driven, and customizable).

componentClient
  .forEventSourcedEntity("activity-agent-prompt") 
  .method(PromptTemplate::update) 
  .invoke(prompt);

Inline evaluation

Execute evaluations of input and output responses to an LLM within the execution context of each call. Create cost- or quality-interrupts that alter agentic workflow.

Agent orchestration

Drive agents from an Endpoint, Consumer (streaming ingest) or Workflow. Leverage Akka Orchestration to guide, control and manage multi-agent systems.

Memory compaction

Streaming events combined with a summarizer agent offer multiple techniques for memory compaction to reduce the size of the context that is provided to an LLM.

Function tools

Define tools that are required by an LLM and handled by an agent.

1. Tools as functions within an agent.
2. Tools as separate objects outside an agent. Akka will use dependency injection to insert the tool at runtime.
3. Tools that are remote MCP servers.

Agent exceptions

Handle exceptions as part of an effect framework with AI-specific error conditions and handling methods.

return effects()
  .systemMessage(SYSTEM_MESSAGE)
  .userMessage(message)
  .onFailure(exception -> {
    return switch (exception) {
      case RateLimitException exc -> 
      case ModelTimeoutException exc ->
      case ToolCallExecutionException exc ->
      default -> 
    };
  })
  .thenReply();

Agents can stream the model response tokens to an endpoint. These tokens can then be pushed to a client using server-sent events (SSE). You can create endpoints that are streaming aware that know how to invoke streaming-aware agents.

// An endpoint that calls an agent
// The agent’s response is streamed over SSE
@Post("/ask")
public HttpResponse ask(Request request) {
  var responseStream = componentClient
    .forAgent()
    .inSession(request.sessionId)
    .tokenStream(agent::query) 
    .source(request.question); 

  return HttpResponses.
    serverSentEvents(responseStream);

Agents can act as MCP clients with the tools available to the agent defined as part of its effects.

return effects()
  .systemMessage(SYSTEM_MESSAGE)
  .mcpTools(
    RemoteMcpTools.fromService("weather"), 
    RemoteMcpTools.fromServer(remote_mcp_server) 
      .addClientHeader(api_token) 
      .withAllowedToolNames(tool_names) 
  )

You can now create MCP servers and host them as an endpoint service. MCP servers can be part of your agentic service or deployed separately. MCP Endpoints support and ACLs, JWTs for authenticated access. They can have suggested client prompts and enforced structured data types for requests to it. MCP Endpoints are composable with other Akka services, so it’s possible to invoke workflows, other HTTP Endpoints, or even other agents from within an MCP Endpoint.

Automatic mapping of structured LLM responses into Java data structures with type and error checking. For example, an LLM that is guided to respond in JSON can have that response mapped to a typed field.

.responseAs(Activity.class) 
.onFailure(//handle-bad-response)

Integrated Testkit for executing agent unit and integration tests. You can assert facts (deterministic responses) and integrate evaluation functions (non-deterministic responses).

var response = client
  .forAgent()
  .inSession(sessionId)
  .method(Agent::ask)
  .invoke()

assertThat(response).contains(...)

In addition to Akka Agents having built-in session memory that is durable, you can use Akka Memory to create long-term memory that is shared by multiple agents, users, or sessions. Akka Memory is automatically replicated across regions when deployed across different regions.

Additionally, you can create read-only Views of aggregated data across long-term and short-term memory. Think of Views as a pre-defined, read-only query against a cross-section of your memory. Views enable the separation of read/write compute, a critical design pattern when scaling memory into millions of Transactions per Second (TPS).

Guide, moderate, and control multi-agent systems even across crashes, delays, or infrastructure failures with sequential, parallel, hierarchical, and human-in-the-loop workflows. Embed a registry for agent, tool, API, and resource governance.

High performance stream processing for ambient, adaptive, and real-time AI. Continuous processing, aggregation and augmentation of live data, metrics, audio and video. Streams ingest from any source and stream between agents, Akka services and the outside world.

• Example: Real-time temperature agent