Building a Test Safety Net Retroactively

Building a Test Safety Net Retroactively is the work of buying the right to change a system you did not write. The net does not prove the system is correct. It records what the system does today, precisely enough that tomorrow's change has to declare itself.

Key takeaways

• A characterisation test pins current behaviour rather than asserting correct behaviour. When it fails, the meaning is that something changed, and deciding whether the change was wanted is a separate question with a person attached.
• Build the net around the paths that carry money, data and legal obligation, not around the code that is easiest to test. The perimeter is chosen from the writes you followed in Chapter 3, not from a coverage report.
• Seams decide where the net can attach. Where behaviour can be altered without editing code in place, tests are cheap; where it cannot, the first job is creating one seam, not writing one test.
• Record the actual output as the expectation, including the parts that look wrong. A characterisation test that encodes what you wish the system did is a bug report disguised as a test.
• Coverage percentage is a diagnostic and a terrible target. The net is finished when the paths the business depends on are pinned and the team stops asking permission before changing them.

Read this beside Chapter 3, which produced the seam map and the list of writes, and Chapter 5, which bought the calm this work needs. Chapter 9 is where the net starts paying for itself, because it is what lets repair and delivery run in the same week without either one waiting.

Someone will ask, in week five, what the test coverage is. The answer will be four percent, and the number will be reported upward as a finding about the previous team. It is not a finding. It is a starting position, and it is the wrong thing to be measuring.

The right question is narrower and harder: if I change the pricing rule tomorrow, what tells me the invoice totals still come out the same?

What a characterisation test is, and why it is not a unit test

The distinction is the whole chapter, so it is worth stating carefully.

A unit test encodes intent. Someone decided what the code should do, wrote that down as an expectation, and the test fails when the code disagrees with the intention. That requires knowing the intention, which is exactly what a rescue does not have. Chapter 3 established why: the theory that produced the code left with the people who held it, and Naur's argument is that it does not come back from reading the text.

A characterisation test encodes observation instead. The term is Michael Feathers', from his 2004 work on legacy code, where the same book supplies the definition of legacy code that makes this chapter necessary: code without tests. The technique is mechanical. Call the code with a real input, capture whatever comes out, and assert that it comes out again. You are not claiming the output is right. You are claiming it is current.

That changes what a failure means. A red characterisation test does not say the code is broken. It says the behaviour moved, and now a human has to decide whether the movement was intended. On a rescue that is the most valuable signal available, because the dangerous changes are the ones nobody noticed making.

It also changes how you treat output that looks wrong. If the rounding is off by a cent on invoices over a thousand, you pin the cent. It is behaviour customers have been receiving, possibly for years, and some of them have built reconciliation around it. Record it, add a note beside the test that says this looks wrong and has not been changed, and raise it as a decision rather than fixing it inside a test commit.

Choose the perimeter from the writes, not from the coverage report

The instinct is to start where testing is easy. That produces a net around the utility functions and nothing around the parts that can cost money.

The perimeter comes from the work already done. Chapter 3 located domain logic by following the writes: the schema and its constraints, the scheduled jobs, the queue consumers, and the outbound edges that reach a customer or a third party. Those are the paths to pin, in that order of value, because those are the places where a regression has consequences outside your own system.

Rank them by what a failure costs rather than by how often the code runs. A nightly reconciliation job that touches every ledger row matters more than a page rendering a thousand times an hour, even though the page has more traffic. Money movement, data writes that cannot be reconstructed, anything with a legal or contractual consequence, anything that emails a customer.

Then draw the perimeter as a real boundary and write it down. Inside it, changes require a passing net. Outside it, changes require judgement and nothing else. Publishing that line is what stops the endless argument about whether a given change needs a test, and it makes the untested region a stated risk instead of an oversight.

Expect the perimeter to be small. On most stalled systems, the paths that carry the business are a modest fraction of the code, and a net around them can be built in weeks rather than quarters.

Seams decide where the net can attach

A seam, in Feathers' sense, is a place where behaviour can be altered without editing code in that place. It is the physical precondition for a test: without one, the code under test drags its dependencies into the test with it, and the test becomes an integration you cannot run.

Three kinds show up most often in inherited systems. A constructor seam, where a dependency is passed in and can be substituted. A configuration seam, where a value read from environment or settings decides which implementation runs. A wrapper seam, where a call goes through a thin function you can replace, which is why the first refactor in a rescue is often extracting a direct API call into a one-line method that does nothing else.

Where no seam exists, the first job is creating exactly one, and creating it in the smallest way that works. Not restructuring the module. Extract the untestable call into a method, make that method overridable, and stop. That single change is safe enough to make without a test, which is the property that gets you started when you have nothing.

The tempting alternative is to test everything through the outside: drive the whole system through its HTTP interface and assert on responses. Use it deliberately and sparingly. End-to-end tests over a legacy system are slow, flaky and expensive to diagnose, and a flaky net is worse than no net, because the team learns to rerun it until it passes. That is the same detection loss described in Chapter 5, in a different costume.

Pin the data, not just the code

The part most retroactive test efforts get wrong is fixtures.

A characterisation test is only as good as the input it replays, and synthetic inputs miss the cases that broke production. The best source of realistic input is production itself, which brings an obligation with it. Take a sample of real records, then mask everything identifying before it enters the repository: names, emails, card fragments, addresses, national identifiers. Masking is not a nicety here. A test fixture is a copy of the data with none of the access controls, and it lives in version control forever.

Choose the sample to include the shapes that hurt. The largest invoice ever issued. The order with fourteen line items and a partial refund. The customer with two addresses and one soft-deleted account. The row with the null in the column that supposedly cannot be null, which the schema walk in Chapter 3 will already have found.

Then pin the outputs as whole documents rather than as field assertions. Store the resulting JSON, the generated invoice text, the ledger entries, and diff them on each run. Whole-document comparison catches the change you did not think to assert on, which is precisely the class of regression a rescue produces.

Coverage is a diagnostic, and a target only for people who have run out of judgement

The number will be requested, so give it and frame it.

Coverage measures which lines executed during a test run. It does not measure whether anything was checked, whether the assertions are meaningful, or whether the paths that matter are among the ones covered. A codebase can reach sixty percent by exercising its serialisation layer and still have nothing at all around billing.

Report two numbers instead. Coverage of the perimeter, which should approach complete. Coverage overall, which is context. The first is a commitment and the second is a fact.

There is a related trap in the other direction. Tornhill and Borg's 2022 study across 39 production codebases found low quality code carried far longer maximum issue cycle times, and it is easy to read that as a mandate to improve everything. The net is not the improvement. It is the permission to improve, bought where the improvement is worth the most, and the cheapest place to have no tests is the code you decided in Chapter 4 to archive.

What the net changes about how the week runs

The first observable effect is not fewer bugs. It is a change in how decisions get made.

Before the net, every change to the billing path is escalated, because nobody can tell what it will disturb. Estimates carry a wide margin nobody can explain, and the safest available action is to postpone. After the net, changes inside the perimeter are ordinary work, and the argument shifts to whether the change is wanted.

The second effect is measurable, and it is the one to report. DORA's change fail rate and failed deployment recovery time, baselined in Chapter 5, should both move as the perimeter closes. If they do not move after a month of test writing, the net is around the wrong paths, and the response is to move the perimeter rather than to write more tests.

The third effect belongs to the outgoing team, and it is worth naming. A characterisation net is the only artifact that captures behaviour without requiring anyone to remember why it is that way. It preserves the observable half of a theory that is otherwise leaving the building, which does not replace the interviews in Chapter 3 and does give them a deadline that is less absolute.

Chapter summary

A retroactive test net pins behaviour rather than asserting correctness, which is the only honest thing to do in a system whose intentions left with its authors. Characterisation tests, in Feathers' sense, record the current output as the expectation, including output that looks wrong, so that a failure means the behaviour moved rather than that the code is broken. The perimeter is chosen from the writes located in Chapter 3, ranked by what a failure costs, and drawn as a stated boundary so the untested region becomes a declared risk. Seams decide where the net can attach, and where none exists the correct first move is creating one in the smallest way available, usually by extracting a call into a method that does nothing else. Fixtures come from production, masked before they enter the repository, chosen for the shapes that hurt, and compared as whole documents rather than field by field, because whole-document diffs catch the regressions nobody thought to assert on. Coverage is reported as two numbers, perimeter and overall, with only the first treated as a commitment. And the net's real output is not fewer defects but a change in how the week runs: changes inside the perimeter stop being escalations, the instability metrics from Chapter 5 start to move, and if they do not move within a month the perimeter is in the wrong place.

You can now change this system deliberately. What has not changed is the schedule the business is still holding you to. Chapter 7 is Renegotiating Scope With the Business: how to present a reset that gets accepted, why the conversation fails when it leads with the technical argument, and what evidence has to be in the room before anyone will agree to move a date.