Tim on Leadership

Musings on Management and Leadership from Tim Parker

Release Leapfrog

Small companies work on a fairly linear release schedule: work on version 1.0.  When that's out the door, work on version 1.1.  Then move on to version 1.2.  Eventually, start work on version 2.0.  Then 2.1, and so on.  Simple and very linear.  But as soon as your mass of engineers expands to the point where you can work on more than one thing at a time, release management becomes more complicated.  For example, you might have half your team working on bug fixes, and half on new features for the next release.  Or you might have two new sets of features with different development timelines, with a team working on each but independent of each other.  Apart from issues such as integrating codebases and ensuring everything is synchronized, let's look at the higher level picture and look at how releases should be scheduled.  In another article we can look at synchronization issues (which are considerable).

There's two schools of thought about mixing bug fixes and new features into a release. One school says do it, the other says don't.  The logic for keeping new features and bug releases separate is simple: if you've got a product released to the world with bugs in it, focus on fixing those bugs and getting the released software as stable and issue-free as possible.  Mix in some new features and you complicate that task and make it harder for QA to validate the bugs have been fixed.  The logic for mixing is simple, too: why push out a new release (and persuade your customers to download and deploy it) for just a few bug fixes when you can provide some value-add with new capabilities.  The problem with this approach is bug fixes can take a while to get to the customer, while new features are still being developed and testing.  I've worked with both approaches, and over the years have moved more and more to the first school of thought for one simple reason: software quality.

Software being what it is, there will be bugs.  Whether poor coding, lack of adequate testing, misunderstood functionality, GUI mistakes, or whatever, mistakes happen.  Customers will find them, or you'll find them in testing or soaking.  They have to be fixed if possible, simply to ensure the products you put out are the highest quality possible and provide your customers with the sense that you write solid software they can depend on. A proper development and testing cycle will reduce the number of bugs in any release dramatically, but there's still going to be the hard-to-find issues with a particular customer's hardware or software load-out, interactions with other products, unforeseen usage scenarios, and half a dozen other things you couldn't or didn't test for prior to product release.  If a customer finds a bug, it's a bad thing.  They want it fixed and it's up to you to get that fix out there so the product does what it is claimed to.

As mentioned, I've tried both mixed and dedicated bug and new feature releases over the past two decades, and, especially for larger groups, prefer to have dedicated bug fix releases interspersed with new feature releases.  There's even a symmetry to the whole thing.  To wit: version 2.0 adds a major change to the software; version 2.1 is the bug fix for that release; version 2.2 adds new features, followed by version 2.3 which is bug fixes for version 2.2.  And so on with every odd numbered minor release a bug fix and even numbered releases adding new features.  That makes it easy to know what release is doing what, and helps explain the system to the end customer.  If you're lucky and software testing was elaborate and thorough, there will be very few bugs, so the odd-numbered releases will be few and far between.  Of course that scheme screws up as soon as your bug fix release adds more bugs, or more bugs are found that were not addressed in the bug fix.  So then what?  Skip the new release minor increment?  Add a patch number?  In practice, running into this issue means you didn't test the software properly and your testing process needs to be examined in more detail, but in the real world these things happen.  The beauty of this approach is the bug fix releases can be put into production relatively quickly and independent of any new feature that is being worked on that hasn't quite worked out the way it was planned.

What about mixing bug and new feature releases?  That's fairly easy, as you simply work both according to traditional project management methods and follow the same testing processes.  But then again, if the bug fixes are minor or obscure, this may not be an issue.

In larger teams, I like to have two groups working on separate releases, leapfrogging each other.  So, for example, if the current release is 2.1, one team is working on 2.2 and the other on 2.3.  They can be interleaved so that there's a new minor release at intervals and both teams don't complete at the same time, but there's code synchronization issues that can arise that need to be dealt with.  For smaller teams, leapfrogging releases is more difficult but still possible as long as there's tight communications and clear decisions as to what goes where.

As with all software development cycles, it all comes down to managing the product, chasing the goal of bug-free releases, and ensuring you get the most out of your teams.