Continuous receipt - sacrificing versions

Last week I received an email that convinced me that the way we think about versioning has irrevocably
changed. Here is the meat of the mail:

IntelliJ's Perforce plugin now has job support.

While this is a valuable feature for me in my current project, it's hardly a major event for software in general.

What struck me was the author's implicit assumption that IntelliJ is a single continuous entity that changes state. IntelliJ "now" behaves differently to what it did a month ago.

To a user, "IntelliJ" is becoming less a description of an immutable executable than the name of an evolving organism of features. Whereas once we spoke of the 2010 incarnation of a product verses the 2011 product, soon we will speak of how a piece of software behaved in 2012 compared to 2013.

Web applications like Facebook, Twitter and Gmail have been doing this for some time. It's becoming the norm for desktop apps too.

The frantic pace of continuously delivered versions is obliterating the end-user's concept of a version altogether. Where previously we expected to pull new features by explicitly choosing to upgrade, we now accept that our software will frequently be pushed changes - almost in real time.

This large number of minutely differing versions is similar to how animation generates the illusion of motion. The more frequent the changes, the smoother the effect. As video games have increased their frame-rates they have made older games look clunky and awkward by comparison. Publishing games with 1995's industry standard frame-rate would put a studio at a serious competitive disadvantage today.

Developers who are unable to embrace the concepts of continuous delivery will find users perceive their software in much the same way.

Designing Haskell

The other day I was struggling through quirk of Java's generics (which are implemented with type erasure). I pondered that it would have been wonderful if generics had been properly baked into Java from the start instead of being superficially retrofitted.

While I'm strengthening Java's type system, I thought, I should wish for type inference to cut out all those redundant type annotations. And if I'm going to do that, I might as well ask for closures!

Of course, some of the worst things about Java are awkward and commonly misused features, like nullability. I occasionally use it myself, but I'd prefer it if there were an alternative. And I'd gladly give up mutability, primitive types and implementation inheritance if only every other Java programmer had to give them up too.

But, I considered, would I just be imposing my personal taste on others? I mean, I like significant whitespace, but I know some other programmers inexplicably prefer curly braces.

If only there were already a magical programming language like this that I could switch to without disturbing the Java developers who are happy with things the way they are.

Eco on over-engineering

This triumph of the mechanism produced a kind of giddiness, whereby the importance attached to what the machine actually produced was less than that attached to the extravagance of the mechanical resources required to produce it, and frequently many of these machines displayed a wholly disproportionate relationship between the simplicity of the effect they produced and the highly sophisticated means required to obtain it.


- Umberto Eco, On Beauty: A History of a Western Idea

Procrustean sizing

Conventional agile estimation involves assigning story points based on the relative size of each piece of work. Most teams refuse to accept stories whose size falls outside a certain limit and send them back for further analysis. Teams using Procrustean sizing take this to the extreme and only accept stories of a single, fixed size.

Procrustes was mythical ancient Greek bandit whose speciality was capturing unwary travellers and making them fit his iron bed. If they were too short, he would stretch them. If they were too tall, he would cut off an appropriate section of their legs.

Procrustean has come to describe the coercion of data into an arbitrary container or structure. The term is often used perjoratively to refer to a simplistic, one-size-fits all approaches.

However, I believe that a Procrustean story sizing is a viable technique for teams using kanban to manage their work. Limiting all stories to a pre-determined size places extra restrictions on story formation but it provides stronger assurances about how work and value will flow through your team.

When all stories are of the same size it becomes easier to reason about the backlog as a whole. Adding up stories of different sizes is dangerous because teams sometimes have a tendency to over or under-estimate differently for larger stories. An eight point story might take more or less than four times as long as a two-pointer when there is not a strictly linear relationship between story points and effort.

This effect is stronger with greater variability. Some teams avoid the temptation to add up stories of different sizes by estimating in animals or some other arbitrary non-numeric scale. However, when there is only one size this complication is avoided and calculations of the total size of the backlog become more reliable.

More importantly, limiting variability helps to maintain a constant flow. Production levelling (as described by Taiichi Ohno in his account of the Toyota Production System) is essential for reducing waste because it ensures that all parts of the pipeline recieve work at a rate they can handle. Uneven flow (mura) results in work building up in some areas while other areas are forced to remain idle and disrupts feedback.

Setting limits on work-in-progress (WIP) is also simpler when there is no distinction between points and stories. A WIP limit of story points can mean that there are too few stories in development to usefully work on in parallel, but a WIP limit of stories can allow too much work to be in progress at once if large stories are in development. When there is too much WIP, it takes longer for problems to show. Also, it becomes difficult to change priorities because expedited stories are blocked by large pieces of work already in progress.

The most obvious disadvantage of the technique is that stories may need to be divided, even when the product owner envisages them as a single unit. This is a problem for any team that does not accept stories of an unlimited magnitude, but it happens more frequently when using Procrustean sizing.

Adopting Procrustean sizing is a tradeoff. If reliable flow is more important than the narrative integrity of stories, then it can assist a team's development process. If preserving stories in the form that product owners originally imagined them is important for communicating with stakeholders, then levelling your stories may not be good option.

Procrustean sizing forces teams to organise plan stories in the form that they can most easily be worked on. It's the software equivalent of Ohno's advice to reduce batch size in manufacturing and, like reducing batch size, takes discipline and effort. Procrustean sizing is a simple constraint, but kanban has already demonstrated the value and power of simple constraints correctly applied.

NB: I have not used this technique in its pure form on a real project. This post is intended as an RFC and exposition of the Procrustean sizing concept. I would be very interested in hearing others' experiences with this or similar ideas.

gets gets my goat

I'm participating in a virtual study group on the book Seven languages in seven weeks. The first week has been on Ruby, which has been a liberating and and frictionless experience - except for one glitch.

One of the exercises in the book involves writing a primitive grep in Ruby. This program reads text line-by-line from stdin and echos the lines that contain a fragment specified as a command-line argument.

My first attempt looked something like this:

#!/usr/bin/ruby 
while line = gets
    puts line if line.match ARGV[0]
end 

But when I tried to run it...

> ./grep.rb foo

./grep.rb:3:in `gets': No such file or directory - foo (Errno::ENOENT)
 from ./grep.rb:3

After a lot of head-scratching, I finally figured out that gets behaves differently if there are command-line arguments. From the Ruby documentation on gets:

Returns (and assigns to $_) the next line from the list of files in ARGV (or $*), or from standard input if no files are present on the command line.

So, I actually needed to do this:

#!/usr/bin/ruby 
while line = STDIN.gets
    puts line if line.match ARGV[0]
end 

After becoming a little frustrated at this seemingly inexplicable overloading of gets, I took a moment to think about why it bothered me so much.

One benefit that Ruby advocates often cite about the language is its adherence to the Principle of Least Surprise. So in the case of this particular unpleasant surprise, I felt cheated.

But then I realised that the strength of my reaction was a backhanded compliment to Ruby. The gets incident stuck in my mind only because it contrasts so strongly with the rest of my experience of Ruby as a consistent and well-designed programming language.

Agile teams need the same skills

There is occasionally anxiety about how project managers, business analysts, QAs and other traditional roles fit into agile projects. A project manager new to agile development might feel that their skills and experience aren't valued because they don't see their old job title explicitly mentioned in a description of an agile team.

Agile schedules still need to be managed. Agile requirements still need to be analysed. Agile codebases still need to be tested. The difference is that agile teams don't assume that these activities have to be done by dedicated team members.

A cross-functional team needs a wide variety of skills to succeed. Sometimes these will be provided by specialists and sometimes by generalists. The balance has to be struck for the specific needs of a project and can't be determined by a crude, one-fits-all rule.

Though agile is a great step forward in software development, it would be arrogant and unfair to think that there is no place in agile teams for software professionals that happen to come from traditionally structured organisations.

Elastic teams absorb shocks

I've come across the idea that because cross-functional teams need a variety of skills, members of the team should aspire to be generalists. I think that is overstating the case somewhat.

Agile teams need to be able to handle workloads requiring varying mixtures of skillsets. In order to cope, teams need elasticity, that is they need to be able to temporarily boost their capacity in a particular kind of work as the situation requires.

This can be better achieved by a mixture of flexibility and focus than by either radical specialisation or radical generalisation.

Specialisation has the obvious drawback that if the specialist in a given area is at full capacity the team cannot take on any more of this kind of work. This leads to the situation where some team members aren't fully utilised but the team cannot commit to taking any more stories to "done done" in the current iteration.

Total generalisation doesn't produce idle team members, but it does prevent a team from reaching its maximum possible potential. Software development is full of difficult and technically-specific problems that may not be understood by someone who has not deeply immersed themselves in that discipline. Furthermore, it's hard for someone who isn't fully comfortable with a domain to coordinate others at the same level.

I prefer teams where there is an expert ready to take charge in the face of any given challenge and who can rely on the rest of the team to fall in behind them. This happens best when team members are specialists in their own discipline and generalists in everything else.

To return to the metaphor in the title of this post, effective agile teams have a definite shape but they have enough elasticity to absorb the shocks of atypical workloads.