Architecture is the opposite of surprise

Architects often disagree on technical matters. But there's also a surprising amount of disagreement on what software architecture actually is. Here are a few definitions that I've come across:

• an abstract description of the system
• work done by developers with the title "architect"
• that diagram with clouds and arrows that somebody put on the network drive at the start of the project before we really knew what we were building
• that document signed by the client at the start of the project before we really knew what we were building

The problem with all these definitions is that they imply that systems that don't have architects don't have architectures. Any definition that can't cope with the majority of applications that have emerged without considered architectural analysis isn't very useful.

Martin Fowler's definition is more useful and widely applicable. He suggests that software architecture is the set of “things that people perceive as hard to change.” [PDF] This characterisation is successful because it shifts the focus from the production to the consumption of architecture.

Fowler's definition challenges the intentional fallacy as it applies to software architecture, which is the idea that the meaning of a text belongs to its author. Fowler's architecture can therefore include elements that were never deliberately envisaged by an architect, which in turn lets us consider systems that never had an architect.

A similar idea was advanced in the essay The Death of the Author by the literary critic Roland Barthes:

As soon as a fact is narrated no longer with a view to acting directly on reality but intransitively, that is to say, finally outside of any function other than that of the very practice of the symbol itself, this disconnection occurs, the voice loses its origin, the author enters into his own death, writing begins.

Substitute "architect" for "author" and "development begins" for "writing begins" and Barthes could be talking about what happens when a carefully prepared architecture document is handed over for implementation.

Claude E. Shannon's information theory formally analyses the consumption of texts. He measured the information content of written English by showing test subjects a truncated piece of English text and asking them to guess what letter would come next. They guessed correctly about half the time (which means that English contains roughly 1 bit of information per letter).

The beauty of this experiment is that Shannon didn't need a model of his subjects' knowledge of English. All he had to do was observe was what happened when they applied that knowledge.

Implicit in Shannon's experiment is the idea that English is the sum of all cues that inform speakers as to what could come next. Following his approach, I would define architecture as the sum of all cues that suggest to a developer how a feature should be implemented in a particular system.

These cues can take many forms. Perhaps the arrows and clouds diagram tells a developer in which tier of the system to put a particular piece of logic. But developers are also guided by the language used by stakeholders, organisational structure and the culture of the technology stack.

Under this definition, the more prescriptive a system's architecture, the less information developers need to absorb in order to understand a given feature. In other words, architecture is the opposite of surprise.

Russell on programming language design

A good notation has a subtlety and suggestiveness which make it seem, at times, like a live teacher. 

- Bertrand Russell, in the introduction to Ludwig Wittgenstein's Tractatus Logico-Philosophicus 

Communicative testing

A couple of weeks ago I proposed that tests could be thought of as facts that have to be 'explained' by code. In a comment on that post, p.j. hartlieb pointed out that this paradigm relies high tests dependability. And @hlangeveld suggested that test runs should be seen as analogous to experiments.

p.j. hartlieb and @hlangeveld help drive home the point that the purpose of tests is to provide information. If your tests aren't telling you anything, they're useless.

Normative tests

Test runs tell you whether you've finished new features and if you've broken old ones. I would call that normative information, because it reports on conformance to requirements. That kind of knowledge can answer questions like "Is this change ready to commit?" or "Can we go live on Monday?".

Management love normative information because it helps them make decisions and measure progress. This naturally leads to an over-emphasis on tests' role as a source of normative information.

Informative tests

Good tests are also be informative. They explain the meaning of failures and communicate intent. Tests can serve as alternative requirements documentation. Indeed, systems like Fitnesse unify the two concepts by converting requirements into executable acceptance tests.

The audience for informative tests is almost exclusively the development team. Informative tests provide an intimate perspective on the system's concepts that's necessary to work with the software on a daily basis. This is not information required by management, so the impetus to improve the tests' informative qualities needs to come from the development team themselves.

A Selenium system test that reports failure by dumping a raw exception stacktrace serves its normative function perfectly well. There has been a regression. We are not ready to release. Someone tell management so that they can manage the client's expectations. From Issue 658 in the Selenium bug tracker:

org.openqa.selenium.ElementNotVisibleException: Element is not currently visible and so may not be clicked
System info: os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.6.1', java.version: '1.6.0_15'
Driver info: driver.version: remote
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
 at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:94)
 at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:327)
 at org.openqa.selenium.firefox.FirefoxDriver.execute(FirefoxDriver.java:191)
 at org.openqa.selenium.remote.RemoteWebElement.execute(RemoteWebElement.java:186)
 at org.openqa.selenium.remote.RemoteWebElement.click(RemoteWebElement.java:55)
 at org.openqa.selenium.internal.seleniumemulation.Click.handleSeleneseCommand(Click.java:33)
 at org.openqa.selenium.internal.seleniumemulation.Click.handleSeleneseCommand(Click.java:23)
 at org.openqa.selenium.internal.seleniumemulation.SeleneseCommand.apply(SeleneseCommand.java:30)
 at org.openqa.selenium.WebDriverCommandProcessor$1.call(WebDriverCommandProcessor.java:271)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:637)

If this was all that appeared in your test log, it would be very difficult to interpret the failure. There is no context. It's not apparent what functionality the user has lost, whether the error was handled gracefully or even if the problem is a conflict between user stories.

One way to make the result above more informative would be to catch the exception and log a message like "Error when an administrator attempted to reactivate a blocked account". Product owners don't care about the presence of divs. They care about functionality.

Communicative tests

Testing consumes a lot of effort. The return for that investment is readily available information on the state of the software. The more useful and accessible that information is, the more valuable the tests are.

Donald Knuth's description of literate programming is even more pertinent to testers than other programmers because the only purpose of tests is "explaining to human beings what we want a computer to do."

Blunt quantitative statements are sufficient to communicate normative information to people outside the development team. But to fulfill their potential within the team, test results must be qualitative, explanatory and communicative.