Acceptance tests make big claims

If we have good acceptance tests, why do we need unit tests?

Acceptance tests tell us a lot. When they pass, they tell us things like:

• A feature works from an end-user perspective.
• All the components required to serve user requests have been wired up correctly. 
• The supporting infrastructure has been installed and configured in the test environment.
• The user interface has not significantly changed since the acceptance tests were first written.
• An intermittant network outage did not occur.

But when they fail, they tell us very little. They make such ambitious claims about the state of the software that failures could indicate a large number of disparate underlying issues.

Unit tests, on the other hand, claim very little. If a unit test fails, it is immediately obvious that there is a problem with a very specific section of code.

In general, the easier it is for a test to fail, the less it tells us when it does. Failures from tests that intermittently break for no reason at all aren't very informative. But if there's a little red dot next to that test that "couldn't possibly fail" then you've just discovered something startlingly new about your system.

Relying on acceptance tests makes sense if the code-base is relatively static, because failures will be relatively rare. But software that must change to remain valuable (i.e. almost all software) also needs fine-grained unit tests to give specific and actionable feedback on failures.

Eliot on projects without Continuous Integration

What are the roots that clutch, what branches grow
Out of this stony rubbish? Son of man,
You cannot say, or guess, for you know only
A heap of broken images, where the sun beats,
And the dead tree gives no shelter, the cricket no relief,
And the dry stone no sound of water.

- T. S. Eliot, The Waste Land

The value of the alternative vote

As someone on the U.K. electoral roll, I want to give an ordered list of preferences when I vote, so that the outcome can reflect my opinion of all the candidates. However, I think the pro-A.V. campaign has made a serious mistake that compromises its chance of success at the May 5 referendum. By describing the proposed system as the "alternative vote" they are emphasising implementation details over the value that voters stand to gain.

The alternative vote is so named because if a voter's highest ranked candidate is unable to win, their vote goes to their highest ranked alternative i.e. their next preference. In other words, the alternative vote is named after a detail of the algorithm used to resolve voters' preferences into an electorial outcome. In Australia, where the system is used to elect our equivalent of Britain's House of Commons, we refer to it as preferential voting. Technically, A.V. is just one possible implementation of a preferential system.

The value of preferential voting is that it gives voters the opportunity to express their views on all the candidates - not just their most favoured. A supporter of the Conservative party living in a seat that is dominated by Labour and the Liberal Democrats should not have to vote tactically to have a say in the outcome. Rather, they should be able to give their first preference to the Conservative candidate and use their subsequent preferences to indicate which of the other two parties they prefer.

The name "preferential voting" satisfies the need of the public to know why they should vote for the new system in the referendum. The name "alternative vote" satisfies the need of electoral reform wonks to discuss the technical details of their area of expertise.

On software projects, we also have the problem that the how can overwhelm the why. Agile software practitioners address this issue by stating requirements as user stories which include the expected value of the proposed feature. Typically, these stories are expressed using the As a... I want... so that... template. The first sentence of this post is written in this format.

By choosing a name that emphasises arcane details over delivered value, the Yes campaign has jeopardised its chances of success on May 5.

Freedom of movement

One of the key disciplines of agile software development is avoiding Big Up Front Design. The conscientious XP developer will resolutely refuse to build anything that isn't required for the current iteration. Speculative code is poison to a project's agility, because it costs effort to build, effort to maintain and obscures the code that is performing useful functions. And often, as Ron Jeffries' adage goes, you ain't gonna need it.

Quality is ensured by a kind of mathematical induction. The original version of the application is small, focused and well-tested, which makes it responsive to change. By writing the minimum of new code every time a new feature is added, developers ensure that the code-base never accrues excess baggage that might impede it in the future.

This approach is counter-intuitive at first, because most people associate planning ahead with saving effort. The idea is that tackling future challenges before they arise can be cheaper. The first catch is that if a developer guesses wrongly they will have spent effort actively contaminating the project with detritus. The second catch is that the first catch happens every time.

However, I do not take this to mean that a developer should never look ahead when making decisions about the present. Simplicity is merely a means to an end, which is the greatest possible flexibility to take the project where it might need to go.

Flexibility must be evaluated relative to the needs of the project in question. If a website is likely to be promoted to a mass audience, then preserving code quality means maintaining the option to introduce caching in a later iteration. It would be a mistake to introduce any caching logic "just in case", because performance optimisations are notoriously hard to second-guess. But it would also be a mistake to take a decision that would make caching hard or impossible to implement.

The minimal solution is not the one with the least files, classes or lines of code. Minimal code makes the least possible number of assumptions about the future. Developers need to interpret what that means for their particular project by maintaining an awareness of upcoming requirements.

Freedom of movement is only meaningful if you know what moves you might need to make.

Clean coders are pragmatists

Every now and again, developers are faced with the choice between "doing things the right way" and "just getting it working". Those who attempt to maintain their professional standards under pressure are occasionally caricatured as idealists, clinging to a quaint notion of purity at the project's expense.

The clean way is the pragmatic way almost every time. To compromise code hygiene for expediency is to believe in a world where that doesn't catch up with you. Dirty coders have an incredibly naive and demonstrably false idea of what poor quality craftsmanship does to an application over time.

I've written dirty code. I've gambled on everything going right just this once. But rarely have I not regretted it.

Coding with ghosts

Software development is one of the most intensely collaborative processes I know. But stereotypically, programming is often seen as an activity for loners.

The majority of collaboration between developers is hidden because it happens across months and years. When I write code, I am working incredibly closely both with the original architect and with the future employee who hasn't graduated yet. I am communicating with people I may never meet, but also to past and future versions of myself.

When I leave work, most of the collaboration that I have participated in that day hasn't even happened yet.

Wikileaks is Napster

Remember Napster, the first widely successful peer-to-peer file-sharing service? In 2001, the Recording Industry Association of America won a lawsuit that killed Napster as a free service.

Lawsuits against Napster did not shut down peer-to-peer file-sharing because, like Wikileaks, Napster was neither the source nor the ultimate destination of the information flowing through it.

Wikileaks is a peer-to-peer file-sharing service. Its current architecture is centralised, which is a weakness that it shares with Napster. But if the central node is taken out, it won't be long before a new service with a decentralised architecture springs up.

The only thing that shutting down Wikileaks could possibly achieve is to make whistle-blowing slightly less convenient. For about a month.