Second order poka-yoke

Poka-yoke is the practice of "mistake-proofing" - designing interfaces in ways that prevent users from making mistakes.

Continuous integration is a form of poka-yoke for software development. Developers are prevented from releasing (obviously) broken software to production by a battery of tests.

However, poka-yoke is about designing to mitigate specific user mistakes (at least as I understand Shingo's definition). In other words, it's about putting up safety barriers between users and anticipated failure modes.

How much better to develop in a software ecosystem that discourages not only the occurrence of mistakes, but encourages the growth of designs that do not admit the possibility of these mistakes.

My colleague Gurpreet Luthra cites the example of making heat-resistant bolts a different size to ordinary ones so that they cannot be accidentally interchanged. However, were the design to be simplified so that the engine was a single piece of metal, bolts would not be required at all.

Sometimes simplification is possible. Other times essential elements of the domain force us to introduce scope for error. Perhaps engines could be constructed differently to simplify assembly. Perhaps not. The only way to tell the difference is to be constantly vigilant for the possibility of eliminating accidental complexity.

Having thorough test coverage is good. Preventing the complexity that disrupts developers' ability to comprehend the system well enough to avoid mistakes is better.

Simple designs can not only make mistakes avoidable, they can make them literally inconceivable. By that construction, you could consider a disciplined pursuit of simplicity second-order poka-yoke.

Continuous receipt - sacrificing versions

Last week I received an email that convinced me that the way we think about versioning has irrevocably
changed. Here is the meat of the mail:

IntelliJ's Perforce plugin now has job support.

While this is a valuable feature for me in my current project, it's hardly a major event for software in general.

What struck me was the author's implicit assumption that IntelliJ is a single continuous entity that changes state. IntelliJ "now" behaves differently to what it did a month ago.

To a user, "IntelliJ" is becoming less a description of an immutable executable than the name of an evolving organism of features. Whereas once we spoke of the 2010 incarnation of a product verses the 2011 product, soon we will speak of how a piece of software behaved in 2012 compared to 2013.

Web applications like Facebook, Twitter and Gmail have been doing this for some time. It's becoming the norm for desktop apps too.

The frantic pace of continuously delivered versions is obliterating the end-user's concept of a version altogether. Where previously we expected to pull new features by explicitly choosing to upgrade, we now accept that our software will frequently be pushed changes - almost in real time.

This large number of minutely differing versions is similar to how animation generates the illusion of motion. The more frequent the changes, the smoother the effect. As video games have increased their frame-rates they have made older games look clunky and awkward by comparison. Publishing games with 1995's industry standard frame-rate would put a studio at a serious competitive disadvantage today.

Developers who are unable to embrace the concepts of continuous delivery will find users perceive their software in much the same way.

Acceptance tests make big claims

If we have good acceptance tests, why do we need unit tests?

Acceptance tests tell us a lot. When they pass, they tell us things like:

• A feature works from an end-user perspective.
• All the components required to serve user requests have been wired up correctly. 
• The supporting infrastructure has been installed and configured in the test environment.
• The user interface has not significantly changed since the acceptance tests were first written.
• An intermittant network outage did not occur.

But when they fail, they tell us very little. They make such ambitious claims about the state of the software that failures could indicate a large number of disparate underlying issues.

Unit tests, on the other hand, claim very little. If a unit test fails, it is immediately obvious that there is a problem with a very specific section of code.

In general, the easier it is for a test to fail, the less it tells us when it does. Failures from tests that intermittently break for no reason at all aren't very informative. But if there's a little red dot next to that test that "couldn't possibly fail" then you've just discovered something startlingly new about your system.

Relying on acceptance tests makes sense if the code-base is relatively static, because failures will be relatively rare. But software that must change to remain valuable (i.e. almost all software) also needs fine-grained unit tests to give specific and actionable feedback on failures.

Eliot on projects without Continuous Integration

What are the roots that clutch, what branches grow
Out of this stony rubbish? Son of man,
You cannot say, or guess, for you know only
A heap of broken images, where the sun beats,
And the dead tree gives no shelter, the cricket no relief,
And the dry stone no sound of water.

- T. S. Eliot, The Waste Land