Tim on Leadership

Musings on Management and Leadership from Tim Parker

Offshore Test Automation

If you've read my posts on offshore development and testing, you will realize I am a huge fan of having an offshore test team.  Being able to leave work in North America at the end of the day, then return the next day with the results of a full night's testing of the previous day's code is great from both the speed-of-development and the coding consistency point of view.  No wasted time for the developers here, waiting around for an onshore test team to get back to them with their results. 

So, whenever the politics allows me, I set up an offshore testing team, typically in Vietnam where they excel at testing for a very reasonable cost, and get them working with my onshore QA team to ensure each day's checked-in code meets our quality standards and the overall product quality is high.  As the offshore test team gets more familiar with our product, the codebase, and the test suite we want (often, the offshore test team is the one that writes the test suite under the management of an onshore QA director), they can then start moving the testing to the next (and more important) phase, which is test automation.

Test automation is where a bunch of tests are conducted against the product under the control of software, instead of being managed manually one-at-a-time. As part of the automation process, the obtained result is compared by the test software to the expected result, and if there is any deviation, alerts are generated.  Proper test automation allows for all the test preconditions and environment controls to be manipulated, and the entire process can be done under the control of software so a huge amount of testing can be done without manual intervention.

Test automation is great for my developers, as it lets a huge suite of tests be conducted every night to ensure nothing got "broke" in the previous day's builds.  Routine tests that are passed manually once often are not conducted again unless there's work on that specific chunk of code, but that's a bad approach because interactions can break something far removed from the actual code changes.  So, test automation provides a wide-spectrum health check for the product, quickly and with minimum effort from the local QA team.

The problem with test automation is it takes a huge chunk of time to write the test software, program in the variables required for the test, and automate the whole result comparison and reporting process.  Typically, a test can take a day or more to automate, so when you have a test suite of hundreds or thousands of tests, the time required to build the automated test suite adds up quickly.  So, for many short product development cycles, test automation is often ignored.  But when you're running longer development cycles, or more frequent cycles but with common test bases, then automation is worth the effort.

I've been lucky working with teams in Vietnam that conducted the compete test suite we developed onshore, and they had the initiative to automate and extend the test suite themselves as part of their daily routine.  It takes a bit of time to get the test automation in place, but the early results of test automation led me to devote resources to this process every time I offshore testing.  Typically, I'll allocate a percentage of the team (usually around 10%) to automation of test suites.  That 10% investment pays off huge in both test time savings in the long run, as well as better overall quality.  With the competitive rates in Vietnam, I often end up with an entire test automation suite for less than the cost of one onshore senior QA engineer, which is a great trade-off for the company.  I've automated onshore, as well, but the labor costs make this impractical except where development cycles are long enough to recoup the investment.

So, offshore testing is a great thing in my mind.  Offshore test automation extends that value even more.  It's part of my regular process, and I can't see ending that process in the future, as it pays dividends every time.