The Importance of Good Testing Environments

Michael Nygard has a good blog post stating that QA Instability Implies Production Instability.

Invariably, when I see a lot of developer effort in production support I also find an unreliable QA environment. It is both unreliable in that it is frequently not available for testing, and unreliable in the sense that the system’s behavior in QA is not a good predictor of its behavior in production.

He describes a lot of the pitfalls in maintaining good enviroments, from test data getting overwritten to anonymisation of production data compromising data integrity. Knowing what needs to be done to build and support good test enviromments is an important tester skill.

From my experience, he’s dead right about relationship between the stability of the test environment and the number of problems that escape into production. This is especially true when it comes to things like interfaces with third-party systems. There is a lot of difference between running an instance of the third party system on one of your own servers and anly having access to a system on a remote server where you can’t change the setup or configuation data.  And the number of bugs did indeed reflect this.

Worse still, when there’s no access to the third-party system at all, and the best you can do is write a crude emulation yourself. I still have nightmares about that one….

This entry was posted in Testing & Software and tagged . Bookmark the permalink.

Comments are closed.