Acceptance tests make big claims

If we have good acceptance tests, why do we need unit tests?

Acceptance tests tell us a lot. When they pass, they tell us things like:

• A feature works from an end-user perspective.
• All the components required to serve user requests have been wired up correctly. 
• The supporting infrastructure has been installed and configured in the test environment.
• The user interface has not significantly changed since the acceptance tests were first written.
• An intermittant network outage did not occur.

But when they fail, they tell us very little. They make such ambitious claims about the state of the software that failures could indicate a large number of disparate underlying issues.

Unit tests, on the other hand, claim very little. If a unit test fails, it is immediately obvious that there is a problem with a very specific section of code.

In general, the easier it is for a test to fail, the less it tells us when it does. Failures from tests that intermittently break for no reason at all aren't very informative. But if there's a little red dot next to that test that "couldn't possibly fail" then you've just discovered something startlingly new about your system.

Relying on acceptance tests makes sense if the code-base is relatively static, because failures will be relatively rare. But software that must change to remain valuable (i.e. almost all software) also needs fine-grained unit tests to give specific and actionable feedback on failures.

Eliot on projects without Continuous Integration

What are the roots that clutch, what branches grow
Out of this stony rubbish? Son of man,
You cannot say, or guess, for you know only
A heap of broken images, where the sun beats,
And the dead tree gives no shelter, the cricket no relief,
And the dry stone no sound of water.

- T. S. Eliot, The Waste Land

The value of the alternative vote

As someone on the U.K. electoral roll, I want to give an ordered list of preferences when I vote, so that the outcome can reflect my opinion of all the candidates. However, I think the pro-A.V. campaign has made a serious mistake that compromises its chance of success at the May 5 referendum. By describing the proposed system as the "alternative vote" they are emphasising implementation details over the value that voters stand to gain.

The alternative vote is so named because if a voter's highest ranked candidate is unable to win, their vote goes to their highest ranked alternative i.e. their next preference. In other words, the alternative vote is named after a detail of the algorithm used to resolve voters' preferences into an electorial outcome. In Australia, where the system is used to elect our equivalent of Britain's House of Commons, we refer to it as preferential voting. Technically, A.V. is just one possible implementation of a preferential system.

The value of preferential voting is that it gives voters the opportunity to express their views on all the candidates - not just their most favoured. A supporter of the Conservative party living in a seat that is dominated by Labour and the Liberal Democrats should not have to vote tactically to have a say in the outcome. Rather, they should be able to give their first preference to the Conservative candidate and use their subsequent preferences to indicate which of the other two parties they prefer.

The name "preferential voting" satisfies the need of the public to know why they should vote for the new system in the referendum. The name "alternative vote" satisfies the need of electoral reform wonks to discuss the technical details of their area of expertise.

On software projects, we also have the problem that the how can overwhelm the why. Agile software practitioners address this issue by stating requirements as user stories which include the expected value of the proposed feature. Typically, these stories are expressed using the As a... I want... so that... template. The first sentence of this post is written in this format.

By choosing a name that emphasises arcane details over delivered value, the Yes campaign has jeopardised its chances of success on May 5.

Freedom of movement

One of the key disciplines of agile software development is avoiding Big Up Front Design. The conscientious XP developer will resolutely refuse to build anything that isn't required for the current iteration. Speculative code is poison to a project's agility, because it costs effort to build, effort to maintain and obscures the code that is performing useful functions. And often, as Ron Jeffries' adage goes, you ain't gonna need it.

Quality is ensured by a kind of mathematical induction. The original version of the application is small, focused and well-tested, which makes it responsive to change. By writing the minimum of new code every time a new feature is added, developers ensure that the code-base never accrues excess baggage that might impede it in the future.

This approach is counter-intuitive at first, because most people associate planning ahead with saving effort. The idea is that tackling future challenges before they arise can be cheaper. The first catch is that if a developer guesses wrongly they will have spent effort actively contaminating the project with detritus. The second catch is that the first catch happens every time.

However, I do not take this to mean that a developer should never look ahead when making decisions about the present. Simplicity is merely a means to an end, which is the greatest possible flexibility to take the project where it might need to go.

Flexibility must be evaluated relative to the needs of the project in question. If a website is likely to be promoted to a mass audience, then preserving code quality means maintaining the option to introduce caching in a later iteration. It would be a mistake to introduce any caching logic "just in case", because performance optimisations are notoriously hard to second-guess. But it would also be a mistake to take a decision that would make caching hard or impossible to implement.

The minimal solution is not the one with the least files, classes or lines of code. Minimal code makes the least possible number of assumptions about the future. Developers need to interpret what that means for their particular project by maintaining an awareness of upcoming requirements.

Freedom of movement is only meaningful if you know what moves you might need to make.