Cognitive Debt and the Return of Boring Engineering

There is a failure that does not look like a failure.

The tests pass. The build is green. The feature works. And no one on the team can tell you why the retry logic uses those particular backoff values, because no one chose them.

Thoughtworks gave this a name in its v34 Technology Radar: cognitive debt. It accumulates, in their words, as AI generates increasingly larger amounts of code and “introduces a wider gap between humans and software systems”. The point is not that the code is bad. It is that the team’s understanding of its own system is now behind the system.

Who This Is For

• Engineers maintaining code their team wrote but nobody remembers deciding
• Tech leads watching velocity rise and confidence fall at the same time
• Anyone on call for a service they could not draw on a whiteboard
• Teams about to adopt agents who would rather install the brakes first

What You Will Need

An existing test suite, however weak, and a tolerance for being told it is weaker than you think. The first tool here exists specifically to deliver that news.

The Pattern

Technical debt is code you would write differently. Cognitive debt is code you cannot explain. They are not the same, and the second is worse, because the usual remedy for the first is a person who understands the system.

flowchart TD
  G[Code generated] --> S[System grows]
  S --> U[Understanding lags]
  U --> D[Cognitive debt]
  D --> R{Incident}
  R -->|Nobody can explain it| L[Long outage]
  D --> F[Feedforward: constrain before]
  D --> B[Feedback: detect after]
  F --> C[Debt stays flat]
  B --> C

Thoughtworks splits the controls into two families. Feedforward controls constrain the work before it happens: spec-driven development, curated agent instructions. Feedback controls detect afterwards. The feedback half is where the older practices are coming back, and mutation testing is the one they name.

Mutation Testing: The Test That Tests Your Tests

Coverage tells you a line executed. It does not tell you anything asserted its behaviour. This distinction was academic when a human wrote both the line and its test, because a person writing a test usually intends it to catch something. It stops being academic when the test was generated alongside the code it tests.

Mutation testing settles it empirically. It changes your source, flipping < to <= or deleting a statement or swapping a boolean, then reruns the suite. If no test fails, that mutation survived, and you have found a line your suite does not actually check.

# src/pricing.py
def apply_discount(total: float, tier: str) -> float:
    if tier == "gold" and total > 100:
        return total * 0.9
    return total

# tests/test_pricing.py : 100% coverage, and close to worthless
def test_gold_discount():
    assert apply_discount(200, "gold") == 180

def test_no_discount():
    assert apply_discount(50, "silver") == 50

Both lines run, so coverage reports 100%. Now mutate:

$ mutmut run --paths-to-mutate src/pricing.py

Survived mutants: 3
  - `total > 100` → `total >= 100`   (no test at the boundary)
  - `total * 0.9` → `total * 1.0`    (no test that 200 gold is not 200)
  - `tier == "gold"` → `tier != "gold"` ... killed

Two survivors, both real. Nothing asserts what happens at exactly 100, and nothing would notice if the discount silently stopped applying to a value the one test does not use. That is the shape of the “almost right” defect the 2026 Stack Overflow survey found 66% of developers naming as their top frustration, and it is invisible to every green dashboard in the pipeline.

Run it on the modules that matter, not the whole repo. It is slow by construction, and a nightly job against your payment code is worth more than a full-repo run nobody waits for.

Characterization Tests, for Code You Inherited From Yourself

The standard technique for taking ownership of unfamiliar code is to write tests that describe what it currently does, correct or not, so that any future change in behaviour becomes visible. Michael Feathers called them characterization tests, in a book about legacy code written long before any of this.

Generated code is legacy code on arrival. Nobody has the context, because nobody built it up.

# Not "what should this do". What does it do now, pinned so a change is loud.
import pytest
from src.invoicing import build_line_items

@pytest.mark.parametrize("case", [
    {"qty": 0,  "expect_lines": 0},
    {"qty": 1,  "expect_lines": 1},
    {"qty": 12, "expect_lines": 12},
    {"qty": -1, "expect_lines": 0},   # current behaviour. Possibly a bug.
])                                    # Pinned, so changing it is a decision.
def test_characterises_line_item_expansion(case):
    lines = build_line_items(qty=case["qty"], sku="ABC")
    assert len(lines) == case["expect_lines"]

Note the third case. You are not asserting that negative quantities should produce no lines. You are recording that they currently do, so that the day someone changes it, the change is a conversation rather than an incident. Writing these is also one of the best tasks to hand to an agent, for the reason covered in what to actually hand a coding agent: the behaviour already exists, so there is a right answer.

Thoughtworks does not name characterization tests in the v34 material. That part is my recommendation, not theirs. What they do name, and what makes the case, is testability as one of the established practices returning under AI pressure.

DORA Metrics, Because Output Stopped Being Informative

Lines of code was always a poor measure. It is now actively misleading: a team producing three times the code with the same review capacity has tripled its queue, not its throughput.

The four DORA measures are deployment frequency, lead time for changes, change failure rate and time to restore service. All four are indifferent to who wrote the code. They measure whether the system reaches users and holds up. Thoughtworks lists them among the well-established techniques the speed of AI is driving teams back toward, alongside zero trust architecture and testability.

Watch change failure rate and time to restore in particular. If cognitive debt is real on your team, that is where it surfaces first: not in shipping less, but in taking longer to fix what you shipped, because the person paged does not understand the code.

Zero Trust, Because an Agent Is Not a Colleague

The security posture matters here for an unglamorous reason. A colleague with repository access has a reputation, a manager and a contract. An agent with repository access has a prompt.

Thoughtworks is blunt about the baseline: sandboxed execution and defence in depth are “non-negotiable table stakes”. Treat agent credentials as untrusted by default, scope them per task, and give an agent no permission it does not need for the brief it was given. That is the subject of the next post in this series, on the harness around the agent.

Before and After

Failure Modes to Avoid

• Treating cognitive debt as technical debt. Refactoring beautifully written code you do not understand does not help. Understanding it does.
• Running mutation testing on everything. It is slow, and the report nobody finishes reading changes nothing. Pick the modules where a defect costs real money.
• Generating the tests and the code in the same pass, unreviewed. The suite then encodes the same misunderstanding twice and reports full coverage.
• Using DORA as a performance review. It is a system diagnostic. Point it at a person and it becomes a target, and stops measuring anything.
• Pairing theatre. Watching an agent produce code is not review, and it feels like it is. Read the finished diff away from the session that made it.
• Assuming documentation fixes it. Documentation written after the fact describes what someone reconstructed. A spec written before describes what someone decided, which is why the feedforward half of the split matters more than the feedback half.

What to Build First

1. Run mutation testing once, on your single most business-critical module. The result is usually enough of a shock to carry the rest of the argument.
2. Kill the top three surviving mutants by adding the assertions they expose. Do not chase a score.
3. Characterize the module nobody wants to touch, before your next change to it.
4. Start recording change failure rate and time to restore. Two numbers, no tooling purchase required.
5. Scope your agent credentials down to the smallest set that lets the current task finish.
6. Write the spec first on the next feature and keep it in the repository as the documentation.

Final Take

Every practice here is at least a decade old. Mutation testing dates to the 1970s, characterization tests to 2004, DORA to a research programme that predates the current wave entirely. None of it is new and none of it is exciting.

That is an awkward finding for an industry that prefers new things, and I think it is the right one. The pressure AI put on software teams did not create a need for novel practice. It removed the slack that let teams skip the practices they already knew about. The one resource that never scaled was the team’s understanding of its own system, and nobody sells you more of that.