Test Environment Management

That is a question which most environment managers will hear all too often. So what is the answer? Yes, would be obvious, but I usually counter with an array of questions such as: What do you understand by “an environment”? What do you want it for? What will be tested on it? Who will manage the testing for the environment? Who will use the environment? When do you need it and how long for? Where do you want it? What should be in the environment? Which interfaces should be used? Do you need stubs? How flexible does it need to be? Who will manage the testdata? How many instances does it make sense to have? Does it need to reflect production? How well does it need to perform? What is the budget? and the list goes on…

Many of the requesters will start to look vague by the third question, and then totally lost by the last. The simple request has become a lot more complex and testing will not likely start within the next hour because the test environment manager wants to understand what is actually required, as opposed to what the tester thought he or she might like.

It is not that I wish to be a pain nor waste time unnecessarily, but getting it right the first time can save money and time in the future. It should be the aim of any environment manager to build stable, manageable and well prepared environments which are fit for purpose and do not contain excess baggage.

As I have previously stated, effective testing relies on reproduction. This can be achieved by keeping changes to a minimum and properly managed.

Knowing exactly what is in the environment and why it is there will allow the manager to better coordinate change. Assessing the impact of environmental changes with the testing team and development whilst understanding the possible effects it might have on the rest of the environment, or other environments for that matter, should reduce the number of unforeseen issues.

Environments have a tendency to get very messy very quickly if inefficiently managed, especially when used in parallel. Timelines, requirements, software, versioning, databuilds, servers, storage, databases and testdata can get out of hand when not well coordinated. People, especially those in development, are quick to question the environments when bugs are found in testing. And not knowing what was where when the software was tested and not being able to reproduce a defect can result in longwinded and expensive analysis phases, giving test environments their bad name.

I have to laugh when requests for test environments to support complex and costly projects are proposed over coffee, without an ounce of thought going into what it is that is actually needed. I also enjoy watching the requester deflate as I question the needs. That is priceless and makes up for a lot of the stress which follows 🙂

Ah yes, test environments; the first point of blame for development and testers alike, the black box, where information goes in and information comes out, the annoying, time consuming and mostly boring part of testing. From my experience people either love them or hate them. But regardless of emotional preferences, they are a pretty bloody vital aspect of testing. In fact without a well-managed test environment, you might as well not bother testing at all. And without testing, who needs testers?

… so, where does this leave us?

With one of the crucial aspects in testing being reproducibility, it is key to properly maintain and manage your test environments.

There is of course a distinct difference between managing an environment to test a simple, standalone app, compared to testing a whole system landscape, however the principles and benefits are the same. A test environment should be cost effective, reliable and purposeful and it needs to be managed so that software defects are reproducible.

Test environments are very much a part of testing, hence the name, however they seem to linger somewhere between developers and testers. Developers see them as an extended playground, testers as a misunderstood necessity and project managers as a budget eating nuisance.

The physical role of a test environment manager is in many projects unpractical and unnecessary, but the responsibilities involved should be addressed and practised; especially with the speed, size and capabilities of current technology. And as we know, things are only going to get faster, bigger and more complex in times to come, challenging testers and environments to keep up the pace.

In the World Quality Report 2013-14, 55% of organizations stated a lack of Test Environment Management skills in-house. http://www.capgemini.com/thought-leadership/world-quality-report-2013-14

As a professional Test Environment Manager, these figures do not surprise me and only reflect what I see in so many of my major clients.