What should you not test, and how do you decide whether a test is worth having?

22.1.2 · L3 · Interview

Predict first

Which test is the best candidate to delete?

// A: a test that checks a price-discount calculation for edge cases
// B: a test asserting a mock saw four private method calls in a specific order
// C: a test that a bug fix for an expired-token crash stays fixed
  • A
  • B
  • C

Mental model. A smoke alarm is worth having. An alarm that goes off whenever you move the furniture teaches you to ignore alarms, and then the real fire goes unnoticed.

A test is worth having if it fails when behaviour users or other code depend on breaks, and stays quiet when you refactor. By that standard a lot of tests are not worth it. Do not test the framework (that UILabel shows its text, that Codable encodes a plain struct). Do not test trivial code with no decision in it (a stored property, a pass-through getter). Do not test how something works rather than what it does: a test that asserts the private call sequence, or that a mock received exactly these five calls in this order, breaks on every refactor and proves nothing about the outcome. Do not test through the UI what you can test as a function. And avoid tests of pure layout constants and copy that change weekly.

Spend effort where risk is: money and permissions, parsing and mapping of external data, state machines and reducers, date, time-zone and locale logic, migrations, error and edge cases, and every bug you have fixed (write the failing test first, so it stays fixed). Code coverage is a map of where you have no tests, not a score: 90% coverage with weak assertions is worth less than 60% on the logic that matters, and chasing a number produces tests that execute code without checking it. Ask of each test: what bug would this catch, how often does it fail for reasons that are not bugs, and what does it cost to maintain. Delete tests that fail the question; a smaller trusted suite beats a large ignored one.

B tests implementation, not behaviour, and will break on any refactor; A and C protect real behaviour and risk.

Common traps

  • Asserting implementation details (call order, private state). Every refactor breaks the test even though behaviour is unchanged.
  • Treating coverage percentage as the goal. It rewards executing lines, not checking results.

They might follow up with

  • When do you write a test first? For a bug you are fixing and for logic with clear inputs and outputs.
  • What do you do with a test that fails often for no reason? Fix its cause (time, shared state, network) or delete it; do not teach people to ignore red.

Related questions

  • 22.1.1 What is the test pyramid for an iOS app, and what does each layer cost and catch?
  • 22.4.1 What is the difference between dummies, stubs, fakes, spies and mocks, and how do you decide which to use?
  • 22.7.1 Why do tests flake, how do you find and fix them, and what makes a CI test run hermetic?
  • 15.3.2 How do you migrate a large legacy codebase safely: the strangler pattern, adapters and characterization tests?

← What is the test pyramid for an iOS app, and what does each layer…