Guides  /  Rollout and delivery

How do you prove a rebuilt system matches the original?

Answer

The original system becomes the answer key. Cases are generated at every boundary and every known conflict. The original's result for each case is recorded, and the rebuild must give the same result for every case on every build.

Akka Specify runs the answer key on every build of the rebuild. A build fails the moment a result differs from the original, long before acceptance testing.

The mechanism

The original system answers every case before the rebuild is written.

An existing system often lives in several forms at once: rate tables or rules in a spreadsheet, the code written around them, and the production service that other systems call. What the system does is known only by running it.

Discovery reads each of those sources and runs the original. Analysis lines the sources up and lists every place they disagree, and an owner signs a decision for each conflict before any code is written. The original then runs as an oracle: cases are generated at every weight break, every boundary between zones or tiers, every combination of rules and every recorded conflict, and the original's answer to each one is locked as the answer key.

The rebuild is correct when it gives the original's answer for every case in the key. Every difference that remains is a decision someone signed, and the record keeps it.

What the check compares

The rebuild is checked case for case on three levels.

ReadThe source of the original is read, and each fact found there is recorded with the file and line it came from.
RunProbes drive a running copy of the original and record its routes, its data, the services it calls, and how it behaves over a sequence of calls, over time and across the range of an input.
DiffThe original and the running rebuild receive the same requests, and their responses are compared request by request.
Where the check runs

The answer key runs on every build, and the original keeps running in production.

Integration tests run the rebuilt components on the real runtime, and the answer key runs beside them on every build. A result that drifts from the original fails that build.

A release that passes its exit conditions goes into pre-production for user acceptance. The original keeps running beside the rebuild until a parallel run on live traffic shows the two agree.

Questions

Related questions.

What if the original system is wrong?

An owner decides which behaviour the rebuild keeps. A known defect becomes a signed decision, and its case is recorded as an expected difference.

What if the sources disagree with each other?

Analysis lists every place the specification, the code and production disagree. Each conflict becomes a decision an owner signs before code is written.

Does the rebuild have to run on the same technology?

The answer key compares only results. The rebuild is designed from Akka components, and the check reads what each system returns for each case.

When is the original retired?

The original is retired after the parallel run on live traffic shows the two systems agree.

On Akka. Discovery, probes, the answer key and exit conditions are capabilities of Akka Specify.