Tim on Leadership

Musings on Management and Leadership from Tim Parker

Day and Night Tango

In several of my posts I've talked about day-night process, with development onshore during the day and testing offshore during our night.  This has become, without doubt, my favorite model for ensuring a better development process and I've employed it at every company I've worked at for the last decade.  Put simply, it hinges on using a code management product that allows our developers onshore (or offshore) to write new code, fix bugs, and so on, checking in their code to the source code control system as they go.  With each check in, several times a day when possible, an automated build is triggered to ensure the changes did not break the compilation.  (If there is a break, the code change gets rejected and must be fixed immediately by the developer; no broken code ever stays in the main development branch if it breaks the compile.)  At the end of our working day here in North America, a final new build is triggered for use by the offshore QA team.

Offshore (in my case, almost always in Vietnam), the QA team rolls into work and finds a fresh build in the code management system as well as a list of all changes and reasons for the changes (for tracking and documentation purposes).  The offshore team takes that build and begins their process of testing each individual change (unit tests should have been done onshore by the developers, but you know how seldom that really gets done!), as well as testing the entire product as a whole.  If time allows, the test suite is followed to test every aspect of the product, but on larger products usually the test cases that apply to the sections of code that were modified are the ones that get examined first.  Even better is if automated testing has been set up for the test suite, which is something I like to have the offshore team do when they have time (or I can afford a few extra bodies for test automation).  A full automated test suite is the ideal situation, but it seldom really happens that way.

By the time my developers roll back in to work in the morning, the Vietnam team has finished their testing and reported the results back to the developers and my onshore QA team and QA director.  After the QA director has signed off on the results, any issues found offshore have to be addressed by the onshore developers as soon as possible.  This almost always means that day, if I have anything to do with it!

The beauty of this "code during day, test at night" process is the code changes the developer made the previous day are still fresh in their minds when they see bug reports.  That way, it's easy to mentally maintain their place in the code, remember what they were doing, and fix the issue quickly.  Let a few days go by and they have to relearn the code again, and the momentum has swung the wrong way.

None of this means there is no place for a QA team onshore.  In fact the exact opposite.  The onshore team is the main driving force and gatekeeper, with the offshore team acting as the night shift and running the routine tests.  The QA process is always handled, ultimately, by my QA director onshore, who has control of both teams and ultimate say in the way the testing goes (and is released).  A good QA director can work with both onshore and offshore teams, managing to get the best out of both.  Communications is the key, of course.

This also doesn't mean you can't develop offshore, either.  I like having offshore development teams working with onshore development teams, as I essentially get two shifts per day!  All this means faster coding, better quality, and a lower overall budget for what we deliver.  But the offshore nightly QA effort has been paramount in my development processes for so long, I wonder if I could work without it any more!