What Is Worth Testing, and the Shape That Follows

Testing discussions go wrong when tests are treated as morally good, so that more is better. Tests are a purchase. You spend writing time, run time on every commit forever, and maintenance time whenever the code they touch moves. What you buy is confidence that a specific kind of breakage will be caught before a user meets it.

Once it is a purchase, the useful questions become concrete:

• Which breakages would actually hurt, and how likely are they?
• What does catching this one cost, at each level I could catch it?
• What will this test cost me every time I refactor?

Key idea: the goal is not coverage of the code. It is coverage of the risk. A payment path with no tests and a settings toggle with five is a portfolio nobody chose deliberately, and it is the default outcome of writing tests wherever they are easiest to write.

Everything else in this course is machinery for spending that budget well.

Tests are usually grouped by how much of the system they run. The labels vary between teams, but the trade never does.

The last column is the one that decides architecture. A unit test cannot tell you the two services disagree about a date format, because it never let them talk. An end-to-end test can catch that and everything else, which sounds like a winning argument until you count what it costs to run, to keep stable, and to debug when it fails.

Those costs are not opinions. They are arithmetic, and the arithmetic has a shape.

Put representative durations on the levels and multiply. A unit test that exercises one function in memory runs in about a millisecond. An integration test that touches a real database runs in about 100 milliseconds. A browser-driven end-to-end test runs in about 10 seconds.

Now price three suites that each contain 2,000 tests, differing only in composition.

The pyramid-shaped suite finishes inside a coffee break. The inverted one takes over two hours, which means it cannot run on every commit, which means it stops being a safety net and becomes a nightly report somebody reads on Tuesdays.

Key idea: the pyramid is not an aesthetic preference. It is what falls out of wanting a suite that runs on every change, given that the levels differ in cost by orders of magnitude.

The test pyramid was introduced by Mike Cohn in Succeeding with Agile in 2009: many fast isolated tests at the base, fewer integration tests above, a thin cap of end-to-end tests.

In 2018 Kent C. Dodds proposed the Testing Trophy for front-end work, which deliberately fattens the integration band. The argument is not that Cohn was wrong about cost; it is that in a modern UI, isolated unit tests of components verify implementation details that break on every refactor while missing the failures users actually hit, so the confidence-per-cost ratio at the integration level is better than the pyramid assumes.

Read the two shapes as answers to one question with different inputs. When your units contain real logic, the pyramid's base earns its size. When your units are thin wiring around a framework, testing them in isolation buys little, and the money is better spent one level up.

The generalisable rule: put the weight where your bugs actually live, which is a question about your codebase, not about a diagram.

Level is only half the design. The other half is what the test actually pins down, and this is where most maintenance pain is created.

A test should assert on observable behaviour: what the caller receives, what the system does that someone can see. It should not assert on how the result was reached. The distinction is concrete:

# Brittle: asserts the mechanism
def test_discount_calls_the_rate_table():
    cart = Cart(items=[Item(price=100)])
    with patch("pricing.rate_table.lookup") as lookup:
        lookup.return_value = 0.1
        cart.total()
        lookup.assert_called_once_with("standard")

# Durable: asserts the behaviour
def test_standard_customer_gets_ten_percent_off():
    cart = Cart(items=[Item(price=100)], tier="standard")
    assert cart.total() == 90

Both pass today. Replace the rate table with a pricing service and the first one fails while the software is perfectly correct, which teaches the team that failing tests are noise. That lesson, once learned, is expensive to unlearn.

Gotcha: a test that fails when you refactor without changing behaviour is not protecting you. It is charging you rent on your own design.

Given finite time, the highest-value targets are predictable across almost every codebase:

• Logic with branches and edge cases. Pricing, permissions, date handling, retry policy, anything with the word "unless" in its specification. Cheap to test, dense with mistakes.
• Anything that has already broken. A regression test written the day a bug is fixed is the best-evidenced test you will ever write: you have proof that failure mode is reachable.
• Contracts between components. The serialisation format, the API shape, the database schema assumption. These break silently and at a distance.
• The paths where failure is expensive. Payment, auth, data deletion, anything irreversible. Test these even when they feel obvious.

And the low-value end, where suites accumulate weight without confidence: getters and setters, code that only calls a framework and returns, generated code, and third-party libraries, which are the vendor's job to test and yours to characterise only where you depend on surprising behaviour.

The test that earns its keep tells you something you could not have predicted by reading the code. If you can see the answer at a glance, so can the next reader, and the test is buying very little.

One exercise settles most disagreements about what to test next, and it takes about a minute per candidate.

A test is read far more often than it is written, usually by someone who did not write it, at the worst possible moment: it is red and they do not know why. Two conventions carry most of the readability.

Arrange, act, assert. Set up the world, perform exactly one action, check the outcome. When a test has three actions and seven assertions, a failure tells you the region broke, not the fact.

Name the behaviour, not the function. test_total tells a reader nothing when it fails at 3am. test_expired_coupon_is_rejected_at_checkout names the claim being disproved, which is often the entire debugging session.

In practice: the failure message is the product. A test that fails with assert False has wasted the moment it was built for; one that fails with expected 90, got 100 for tier=standard has already told you where to look. Most modern frameworks give you this for free from a plain comparison, which is one more reason to assert on values rather than on mechanism.

That is the design layer. The next lesson takes the piece that makes unit tests possible at all, and shows how it becomes the most common way for a suite to go wrong: replacing real dependencies with stand-ins.