Tim on Leadership

Musings on Management and Leadership from Tim Parker

What's Wrong With Development?

[Tim's note: This was written a few months after I joined a software company that had tried but failed to implement properly processes for development, testing, and deployment.  I don't want to single out any company for this, but the comments apply to many companies I have been with where an improper amount of blame was placed on the Development team by the executives of the company.]

The Development group has been suffering from some negative comments that have quite a few people in the group concerned that their work is underappreciated and undervalued.  Over the last few months, it does seem as though there have been more negative comments about the Development group than usual, but it is important to keep this in perspective.

First, consider any small software company.  Typically, small companies have a limited resource pool, push to do too much, and often don't have time and resources to perform proper software development and testing processes.  In these cases, software often "breaks" and the development group really is the source of the problem (whether "blame" should lie with them or the overall company approach is another subject entirely).  This company has been no different in its history.  It started with a small group of developers who coded as quickly as they could, as well as they could, but time and resources meant that problems were present in the code.  Inevitably, to someone outside Development, this meant "Dev is to blame".  As the company grew and Development got better organized, better staffed, and better processes, quality increases, but the root cause of problems for those who have been with the company for a while always goes back to "blame Dev".  This is by no means unique to this company: every software company goes through this.  In many ways, there is truth to the "Dev's fault" argument, but it does unfairly weight blame on this group.  The simple fact is, that's the way it is in every company, and we can't expect a non-technical person to differentiate between a development issue, a deployment issue, or an operations issue.

The second aspect of this is that the magnitude of the issues tends to drop considerably as the company and processes grow, but that can simply accentuate the problems that do occur.  For many years, a deployment to the production environment meant a list of problems to be fixed, hotfixes that needed to be pushed out quickly, and work tickets stacking up.  This was again a resourcing and time problem, as we simply could not test everything thoroughly, in the production system environment, or with real data.  That's just the nature of the beast.  However, as we expanded and got better at deployments and testing, the number of major issues dropped dramatically.  In fact, at this company, not a single hour of downtime has been recorded in the last year due to a development mistake.  That's due to the quality being so much higher!  Yet, when a small error does occur (which is inevitable), that error looks big compared to the status quo, even if it would have been simply noise a year ago.

The message to all is to take the larger view.  The quality of our process, our teams, out testing, and our product is better not than at any time in the past.  Everything that comes out of Development is written and tested better than before, and we continue to improve.  But the nature of software cannot be changed.  Mistakes happen, interactions we did not think of occur, and environments sometimes are unpredictable. Our code is massive (millions of lines) and sometimes problems do happen.  That's normal and nothing to be upset about.  What we should be proud of is two things: the mistakes are far fewer and less critical than before, and we quickly get on top of them and get things working properly.

I think it's clear that our products are better now than before, and that our process for development is better too.  We will continue to get better, and our software will continue to be great, but mistakes will happen.  Let's hope they happen less frequently, are less critical, and that we respond properly when they do occur.  As far as blaming Development, well, that will always happen.  Let's not take it personally.  It's ingrained in every software company (and many other companies too!) and isn't personal.