We Ported 65 OSS Projects With AI

We see a potential for AI to write and maintain entire systems without human involvement. In March, we released Akka Specify, a spec-driven delivery toolkit for creating new systems with the Akka SDK. It takes an opinionated approach to defining and implementing an AI system—inclusive of APIs, orchestration, agents, streaming data, and stateful memory.

We observed a few things after the release of Akka Specify:

1. Our solution architects were completing POCs in hours instead of weeks.
2. Our account executives were shrinking delivery times (by many months) in customer proposals.
3. A customer upgraded a codebase written in Kalix to Akka in a few hours.
4. We redesigned a legacy RAG system with 200K lines of code generating 64% eval accuracy reimplementing it in 48 hours shrinking the LOC to 9K while improving eval accuracy to 95%.

We attributed these gains to the dramatic improvement in intelligence with frontier models, the improved reasoning structure that comes with AI harnesses, and structural constraints baked into the Akka SDK.

The Akka SDK has an opinionated structure that imposes one way, and only one way, to do each thing, such as how one service should reliably invoke another regardless of its location. These restrictions enable us to provide a guarantee that if the AI system compiles and passes tests locally, then it's production-ready without further code modifications. Akka embeds its clustering, elasticity, and resilience runtime on every build, producing a full-stack system while not intermingling resilience and scalability concerns within application code. These SDK opinions have made Akka particularly well-suited to leveraging AI to rapidly generate complex, production-ready backend systems.

In a world where the hyperscalers, Palantir, C3.ai, and the frontier model providers are building sizable Forward Deployed Engineer (FDE) armies, we wondered: why can't AI write and maintain entire systems without human involvement?

So we put it to the test. We rewrote 65 open-source software projects using Sonnet, Opus, and Akka Specify in 99.3 hours total.

The Two-Tranche Approach

We did this in two tranches:

1. Span: A pass across all 65 projects that would create a fully mapped specification and implement a slice of up to 10% of the project's surface area. We did this on OSS projects that were both well-suited to an Akka port and those that were not (Go libraries, Rust modules, systems that didn't have a need for backend services).
2. Deep dives: Complete implementation of all of the project's use cases for those projects where we identified places where Akka could have a significant positive impact: systems with a Python backend, limited persistence, orchestration, scheduling, streaming or distributed fundamentals, feasible scope, and a baseline of a quality that could be adequately measured before/after.

Unless stated otherwise, all tables and graphs are the results from the 65 initial ports done together and do not include the additional data from the 10 fully completed ports. While dramatically increasing the total tokens consumed, there were few places where the trends in the completed 10 differed from the trends in the intial tranche of 65 ports.

Our Experimental Goals

1. Specification structure: Akka Specify takes a specification as its input. We wanted to explore whether specifications should have a structure, schema, and maybe even typed behavior so that systems can more reliably be first-pass generated.
2. Context file gaps: Akka Specify uses the Akka context files for creating a design and guiding an implementation. We wanted to identify (and hopefully close) gaps within those files that cause faulty initial implementations of greenfield systems.
3. Exit Conditions: Delivering a "system" into production is more than code. It's also the harnesses, testing, and organizational requirements that must be satisfied. Akka Specify can enforce Exit Conditions that must be proven by pre-release auditors. We wanted to incorporate best practices from different OSS projects into our list of Exit Conditions and auditors.
4. Runtime performance: We wanted to see if the Akka runtime could improve the performance of various OSS projects, and whether embedded durable execution in Akka could create higher forms of resilience for some projects.
5. Model and effort impact: Whether there was a significant impact in terms of time to complete, accuracy, and token cost when switching between different Anthropic models and levels of effort.

Summary Metrics

The Methodology

We ported 65 OSS projects that were considered popular (large numbers of GitHub stars, strong community interaction online, and had a backend). We picked OSS projects in three different flavors:

1. Popular AI OSS projects—particularly those that focused on context, memory, and orchestration. Many of these projects are anchored in Python without durable execution, and we expected to see significant performance gains. Some frameworks chosen were direct competitors to Akka. We also included some popular AI harnesses like DeepSeek's new plug-in-only framework.
2. Popular networking libraries—particularly those that were built in Go and Typescript. We expected overlap with Akka concepts, and Akka could offer a different structure to such concepts and would likely cause a performance degradation. We wanted to see how well AI would transform a library into a framework where concepts are similar, but the interface would deviate.
3. Popular SAAS OSS projects—with rich, complex Web-based user interfaces. These systems have structured APIs which should be well-suited to doing a complex port.

We licensed the code in every port to be Apache licensing, except where we copied over specific GUI interfaces verbatim, and those were licensed however the original project licensed the code with appropriate attribution.

The Delivery Harness

We created a delivery harness that would run in a self-improvement loop.

1. Setup: We will create a new repository, setup observability, and ensure all discovery and build tooling is installed.
2. Discovery: We execute code analysis, domain model analysis, schema analysis, and runtime interrogation of the existing system in order to generate a series of specifications that define the behavior and conditions of the system itself.
3. Port: We define a series of exit conditions derived from the specification and then leverage the Akka Specify planning, tasking, implementation, building, and testing capabilities to author the system with Akka best practices, and then execute the Akka review to ensure that it matches the design standard expected.
4. Benchmark: We execute both systems with a single runner that measures successful execution of all test suites, counts lines of code and assess end user latency performance. We draft a README and push up to the GitHub repository.
5. Improve: We record what failed in discovery or areas where the implementation had to deviate from the specification, and then speculate on techniques we can use to improve the discovery engine or specification standard for the next iteration.

We would rewrite any API, CLI, and MCP surface adhering to the original's design as closely as possible. For any project with a Web GUI, we brought that GUI over wholesale, but modified the way the GUI accesses endpoints to take advantage of Akka server-side events and other streaming capabilities. We did not port mobile, Slack, WhatsApp, or other types of app interfaces that would require human configuration to properly test for validity.

After completing a detailed discovery phase, AI would generate a detailed specification and implementation plan. Claude was required to use Akka Specify for the implementation leveraging Akka components with a focus on:

1. Minimizing total LOC.
2. Eliminating functional redundancies.
3. Avoid using non-determinism when unnecessary (ie, an agent for the sake of an agent should be avoided).
4. Optimizing for performance (such as using CQRS to separate writes from reads).
5. Leveraging the embedded Akka persistence layer of event-sourced entities and key-value entities, workflow for orchestration, views for read models, and durable state replicated across nodes before attempting to import an external database, graph store, or vector store. We were pleased to find that after all ports had finished, none of the projects needed to import an external persistence engine.

The port was only considered complete and successful if the source system's unit and integration tests would pass when run against the new Akka system. Additionally, Claude was required to run /akka:review after every port which executes a variety of design, performance, and code structure auditors. Claude was required to resolve every identified issue:

The conversion must run human unattended. During the analysis phase of any source system, if there was any ambiguous functionality or behavior AI was instructed to get clarity by running the source system's integration tests. Eventually, in trial runs, Claude invented a form of adversarial testing that it would use to determine whether ambiguous behavior could be deterministically applied, or whether it was indefinitely non-deterministic.

Claude was instructed to vary its choice of models and efforts randomly. After a few of the initial runs consumed a lot of time on Opus xhigh, we removed xhigh from the consideration list for the remaining runs.

Each resulting port was placed into a GitHub repository and given a detailed README with instructions on how to use it, how to re-run the unit and integration tests, and how to rebuild it from scratch using Akka Specify. The full list of projects we converted is in the Appendix.

Answer parity measures how often the port returned the same result as the original on the same input. Ports cluster at 100%, and the ports below it are the ones whose remaining gap is recorded in the port's own benchmark report.

Analysis

Model and Effort Performance

We expected Opus to be quicker as it should get more of the analysis right on the first pass. But what we found was that effort was a better barometer of speed. Since we added guardrails over time, we could ensure that there was adequate output accuracy (ie, the project wasn't done until all unit tests passed) regardless of model and effort.

When taking into account efficiency (an aggregate score of total round trips to the model, total wall clock time to complete, and total tokens consumed), both Opus and Sonnet showed similar efficiency until they hit xhigh effort.

But, while they were largely the same efficiency, you still paid more total tokens for higher efforts than you did for lower efforts.

Complexity-Adjusted Efficiency

The dataset was large enough where we could predict the efficiency that was expected given the total complexity of the original project. Complexity here is measured by the total LOC of the original project, using that as a cheap predictor of total complexity. When conversions were efficient, they were anywhere from 50-90% more efficient than expected, but when they were inefficient, they had some significant outliers where they could be 150-300% less efficient than expected.

LOC and Performance by Project Category

It seems that the type of application that was ported was a predictor of whether Akka could improve its LOC and its performance. Big SAAS applications and large complex frameworks saw significant performance boosts due to Akka event-sourced entities, embedded durable execution, and in-memory durable data.

Guardrails and Efficiency Over Time

Over time, after each port was completed, the engine would self-assess whether there were additional checks that it should do when working on the next port. These would show up as additional auditors and guardrails. And, as you'd expect, as you add more exit conditions, auditors, and guardrails, the ports become less efficient.

Steps vs. Tokens

It wasn't obvious to us before running the experiment, but after looking at the data, it does seem intuitive that younger, smaller models need more round trips ("turns") to go through all of its analysis and work before it signs off on the final outcome. You can see from the data that, generally, sonnet needed a lot more turns than opus, and these extra turns typically led to more tokens being consumed in aggregate. Since the cost per token is cheaper in sonnet vs. opus, this may be a moot point in terms of total cost of a port, but for those who are hosting their own models with spare GPU to execute more inference, the extra turns of a smaller weight model may be worth the cost.

Performance Outcomes

We had hoped to show that the Akka runtime and durable execution engine introduce significant performance gains for each project. That happened in quite a few places, but there were significant degradations in others.

We expected to see Akka abstractions create smaller code bases, but we were surprised to find that the amount of shrinkage could be predicted based upon a project's original LOC size.

Residual is the gap between what a port cost and what its original size predicted it would cost. These ports came in furthest under that prediction, and every one of them ran on sonnet at low effort.

These ports ran furthest over the cost their original size predicted. Four of the five ran on opus, and the two most expensive ran at high effort.

The ports that shed the most code all replaced source that carried its own orchestration, persistence, or transport layer.

The ports that grew all replaced small source slices, where the record and state declarations Akka requires are a larger share of the total.

When you rebuild a program on Akka, two things happen at the same time, pushing the LOC count into different directions.

Why LOC Shrinks With Akka

A lot of what's in a big open-source project isn't the actual feature—it's the plumbing. This includes things like:

• Threads that pass work around so nothing gets stuck.
• Locks that prevent two things from stepping on each other.
• Files-on-disk to remember what happened last time.
• Little safety checks to make sure the data isn't garbage.

Akka comes with all of that built in. So the more plumbing capabilities that were in the original, the better Akka would do.

Why LOC Increases With Akka

Akka asks you to be explicit about a few things that little scripting languages let you fudge. In other words, Akka depends upon Java which is strongly typed (and you can see a long history of writings from different vendors on the benefits and consequences of typing). In a small Python or JavaScript project you write one line that mutates an object and it just works. In Akka you write a SomethingHappened record, a SomethingCommanded record, a SomethingState record, and a bit of code that says "when this thing happens, the state goes from A to B." For big projects that's a rounding error; for tiny projects that structure ends up as a lot of unnecessary overhead.

Performance Gains

The ports that gained the most speed all replaced source that reached a database or an event loop on every request.

The ports that lost the most speed all ship something the source did not: durability, an HTTP surface, or a boundary between components that the original crossed in-process.

We believe that these elements cause Akka to offer a performance improvement:

• Your data is in-memory and durable, persisted transparently without a developer coding database semantics. Most projects hit a database on every request—issue a SQL query, wait for the disk, parse the result, close the connection. In Akka, the "row" you'd query is an in-memory object called an entity. A request that reads or updates one entity is a plain field access. That single change turns a millisecond-per-query database call into a microsecond field read. Most of the biggest speedups in the data—1,000×, 350×, 240×—are this: the source was hitting SQLite or Postgres per request, whereas Akka was reading from memory.
• Writes are durable without being slow. Every change is recorded to an append-only event log so nothing survives a crash gets lost—but the recording happens asynchronously and batched, off the request path. The caller gets its answer while the durability write happens behind the scenes. You get "safe" for the price of "fast."
• No locks or contention. Each entity is single-threaded: only one message at a time, in order, no shared mutable state between requests. Most fast-looking Python and Node code has hidden costs from asyncio.Lock, Mutex, and defensive "check-then-act" guards. Akka single-writer entities make all of those unnecessary.
• Streaming is back-pressured and non-blocking end to end. Data flows through the system without buffering and without threads blocking on each other. Throughput is limited by the slowest stage rather than contention.
• Services are clustered from within. When you scale from one node to a hundred, the same code runs—messages route to whichever node holds the entity, and there's no database in between coordinating anything. Most projects reach for Redis or an external message queue to scale; Akka has that inside the runtime.
• JVM JIT. After a warmup pass the hot paths run at essentially native speed. Interpreted Python and Node can't catch up on tight inner loops.

And there doesn't seem to be any correlation to whether improving LOC would cause a performance increase or not.

About Spec-Driven Delivery

Spec-driven delivery (SDD) is the process of using specifications to define a system and use it as a foundation for its delivery. A side-effect of SDD is the avoidance of technical debt and loss from tribal knowledge migration. When implemented with appropriate guardrails, an SDD system can ensure that it maintains itself against all current best practices, avoiding the introduction of technical debt, and further, no project tribal knowledge is lost when a team member migrates to another project or someone new who is unfamiliar with the project starts to engage. If the specifications are complete and the engine which generates systems from them comprehensive, then knowledge is captured and system maintenance is (mostly) automatic.

There are a number of spec-driven development open source toolkits available. Akka Specify is based upon speckit. We've extended it by:

Structured Projects

We impose an opinionated way that every project must be structured. This structure incorporates everything needed to manage code, tests, evaluations, red teaming, documentation, specifications, and harness configuration.

Environment Validation

A development environment has pre-requisites that need to be installed and properly configured. Some of those are Akka dependencies, and others are defined by an organization. We provide setup and configuration that is AI-aware so that any laptop can be readied for any kind of project.

Build & Deploy

Build, test, and deployment are Akka-aware. This includes an understanding of the build tooling, dependencies analysis, running unit and integration test harnesses, and executing microservices locally within clusters. It automates the setup of any remote dependencies or mocks that may be needed such as calling into 3rd party APIs or other microservices that are running in a staging environment rather than locally. And, if you are not tied into CI/CD, it knows how to authenticate against an Akka platform environment, package a service as a Docker image, deploy it, set up routes, and validate that it's accepting traffic.

Self-Documenting

Akka documents the domain model, interaction model, entity model, schemas, and component hierarchy.

Akka Context

Akka Context is a local, offline mirror of the Akka documentation, core concepts, and principles including theory tied to distributed systems, real-time streaming, brokerless reliable messaging, operations concerns, production readiness, and architecture design considerations.

Exit Conditions & Auditors

Akka Specify can optionally be executed in an enforced mode, which maps every rule or requirement in a specification to an auditor that executes within an AI harness and must successfully pass before returning to the user. Exit conditions can enable a higher level of guarantee on quality, but adding more exit conditions leads to more auditors which can extend build cycles.

Exit conditions have to be satisfied as part of the initial build, before a project is review ready, or before a project is releasable. Enterprises can enforce their own exit conditions, and finding the right balance of conditions that must be satisfied on every build vs. satisfied before the final release is challenging. We have built a large repository of conditions that we use on various projects.

A Specification Standard

Most spec-driven development toolkits are indifferent to the structure of the specification. We've extended speckit to accept an Akka Specification which has a structure that leads to better requirements creation and faster authoring. The more structure we give to a specification, the faster a delivery cycle can complete.

Discovery

Most systems, even though being ideated from scratch, are based upon an existing system. Being able to understand an existing system and derive specifications that AI can understand accelerates delivery. No single technique is sufficient for discovery. It is a collection of techniques that span deep analysis of the existing system, identifying entry points, finding dead code, tracing imports and their usage, comparing ported systems with original systems, chaos engines, adversarial testing, code analysis, schema analysis, file analysis, delivery ordering, evaluation, input analysis, temporal analysis, experimentation, and so on. This effort was, in part, designed to help us advance the intelligence of our discovery tooling.

We studied all of the situations where the porting harness started looping due to mistakes or issues in the specification. Many improvements were automatically added into the harness by the harness itself. But, after discussing it internally, we identified five additional things that we will consider adding to our discovery engine to improve specification quality.

1. Interface enumeration. Extract the source system's complete interface — routes, flags, config keys, env vars, scheduled jobs, event types, migrations — into a manifest before specification begins, and require every entry to resolve to a spec rule or a recorded out-of-scope decision. It identifies interface surface the specification has not accounted for, in either direction.
2. Test-suite ingestion. Parse the source project's own test suite, plus its closed issues and release notes, and treat each assertion as a candidate rule requiring a mapping or an explicit rejection. It identifies behavior the original's authors considered contractual that the specification does not state.
3. Copy-time provenance. Instrument the pipeline's copy path so that taking a file from the clone appends its destination, source, and content hash to the manifest as a side effect. It identifies reused material, and distinguishes files shipped byte-for-byte from files copied and then modified.
4. Differential oracle. Run the source system alongside the rebuild behind a proxy that sends each request to both and compares the responses, active throughout construction rather than only at measurement. It identifies behavioral divergence at the moment it is introduced, and — once fed generated inputs — the cases that weren't initially considered.
5. Adverse-condition observation. Degrade the running original on purpose: withdraw its dependencies one at a time, control its clock, and issue concurrent conflicting writes. It identifies error handling, retry and scheduling semantics, and concurrency behavior, none of which appear during normal operation.

Lessons Learned

We had a few goals when we set out on this experiment. Here is what we found.

1. Specifications Must Be Structured Data

Specifications should have structure, schema, and typed behavior, unambiguously.

Every port that ran through the harness produced a spec before writing any code. What we watched across all iterations is that the structure of the specification determined whether a first-pass implementation was correct. Ports whose specs enumerated their claims, cited them to evidence, and typed their answers (this input produces that output; this state transitions to that state) generated code that matched behavior on the first pass. Ports whose specs were prose—"the system should handle X gracefully"—generated code that was plausible and wrong, and the errors surfaced two to four steps downstream when the port was already committed to a design or a structure.

The rework cost of the second kind is measurable. The most expensive ports in the run—the ones that spent 250+ minutes and burned 400M+ tokens—were disproportionately ports whose early specs left decisions implicit. We added a rule that "any claim about a class needs a list" midway through the conversions and every port after that paid slightly more per iteration but had fewer turns before getting to accuracy and compliance.

2. The Akka Context Files Have Gaps

Across all ports, the model's initial implementation choices were high-quality on the components the Akka context files describe in detail—entities, workflows, endpoints, views—and low-quality on the choices between them. Which effect goes in which component, whether a workflow should orchestrate or a consumer should react, when a KVE is enough and when an event-sourced entity is required—these are the questions where first-pass code was most often rebuilt in the review step.

3. Exit Conditions Capture What "Shipping" Means

Exit Conditions do capture what shipping a system means, and a structured harness can act as the record of what those conditions ought to be.

The delivery harness that we generated that standardized how a port should work—setup, discovery, port, benchmark, improve—is the closest thing we have to a portable definition of "system delivered." Every guardrail we added mid-run (sweep leftover processes, refuse an empty screen, close the rendering gate, warn on a hung tool at four heartbeats) turned into an Exit Condition that needed to be audited on a future project.

This reinforces our thinking around providing an enforced mode for Akka Specify where we allow enterprises to define exit conditions from across a variety of different dimensions, integrate the auditors with internal tooling that knows how to measure these conditions, and to enforce them as part of the project's build and test cycle.

4. The Akka Runtime Delivers on Performance and Resilience

Across the ports where we could time the same slice of work on both sides, the port is dramatically faster whenever the original relied on a database, a language interpreter, defensive locking, or hand-rolled scaffolding—which is most of them. In-memory entities beat SQL queries. Single-threaded actor mailboxes beat locks. Async journal writes give durability without paying for it on the request path. The measured range—from ~1.3× on already-fast Java originals to 1,000×+ on database-bound or event-loop-bound originals—is entirely explained by how much plumbing the original was carrying that the port doesn't need.

Where the port looks slower on the benchmark, the cause is nearly always that the port ships things the source didn't: durability, HTTP surface, or cross-component boundaries. You could argue that this isn't a fair port, that we are porting an apple and turning it into a turnip, and that it is pretty fair, so it's hard to draw strong conclusions from that.

On resilience: yes. Every port ships event-sourced durability by default. Every port survives a mid-request crash. Every port ports cleanly to multi-region without changing the code. These aren't things that many of the originals do out of the box. The runtime's embedded durable execution is a step-function improvement in resilience across every category we tested.

5. Model and Effort: Latency vs. Cost

Sonnet is roughly twice as fast (wall-clock) as Opus per port. Opus uses about 40% fewer tokens per port. Complexity-adjusted efficiency—our composite of steps, time, and tokens normalized to the median port—puts the two models within 8% of each other. Choosing the model is a latency-versus-token-cost tradeoff, not an efficiency-versus-inefficiency tradeoff.

Effort matters more than model. Moving from low to high effort roughly doubles the composite efficiency score inside either model. That's a bigger swing than switching models at any effort level. The most efficient cells in the matrix—opus/low and sonnet/low, effectively tied—do the same-quality work at a fraction of the higher-effort cells' cost. In this run there is no evidence that higher-effort settings produced meaningfully more grounded ports; they just cost more.

Future Experiments

These results give us a baseline for comparison. We want to run a few variations against it for additional insight.

1. Porting with the tools switched off: We port the same project again, with the same model and the same effort setting. This time we take away the tools that read the original code first. The AI gets the original project and Akka, and nothing else. We want to see how much those tools were helping. We think the port will start giving wrong answers before it starts getting bigger, because the tools that help most are the ones that say no: test_census, runtime_errors and mutation_probe. An AI that tries harder cannot do their job for them.
2. Porting onto a different framework: We port the same project onto a framework that forgets its work when something crashes and does not store data for you. Everything Akka hands over for free has to be written by hand instead. All that hand-written code shows up as extra lines. We have seen this happen already: glanceapp/glance came out 44% bigger because it had to rebuild Go's template library itself. Running it tells us how many lines the platform saves.
3. Building from the specification alone: We hand a brand new AI the specification, SPEC-001, and nothing else. That AI never sees the original project and has no copy of it to read. Then we run the same tests against whatever it builds. Running it tells us whether the specification is good enough on its own. Every port we have run so far let the AI read the original code, so none of them answered that question.
4. Breaking the specification on purpose: We take a single rule out of a specification. Then we check that the matching code disappears from both the original and the port, and that nothing else disappears. Until we do this, we have not shown that the measurement can go down at all.

One Overall Finding

If there is a single thing to take from 65 ports, it is that the interesting variable in this system is not the model, not the effort, and not the runtime—it is the discipline of the specification and the auditors. The ports that failed did so because the spec left decisions implicit or the auditors weren't yet strict enough to catch a class of mistake. The ports that succeeded did so because the spec forced enumeration and the auditors refused to sign off on anything less. The platform's biggest lever going forward is upstream of the model: it's in making the spec a data structure and the Exit Conditions a versioned, category-aware library.

We've started to see Akka Specify as intelligence that can execute an AI-assisted delivery lifecycle without human intervention—not just single systems, but the infrastructure and audit layer that makes them production-ready.

Appendix: The Ported Projects

Each port has its own repository, with a README covering how to run it, how to re-run the unit and integration tests, and how to rebuild it from scratch using Akka Specify.