This is a topic that always draws some great responses when discussed where I work. Do you Test on your production systems?

I always come to the same conclusion on this one. Why wouldn’t you want to test in production? I know, I know. Your system is too “special” or “secure” or “regulated” or whatever to be able to test in production. So what are you going to do? Let your customers test it for you? Throw the code over the wall to the people that matter most and hope that it works for them? Take the chance that your customer will just understand when the house of cards comes crashing down in a burning heap of lame?

To those that say it just can’t be done, I say that maybe your system is just lacking testability – you haven’t built it right. To me a testable system is one that has a great handle on control and is inherently observable. If you can’t control and observe the software, you are dead out of the gate. Often, if you solve the control and observation issue, you will find a system that you can test in production – because you engineered it to be easy to do so in any environment.

So take a look at your systems and ask yourself if there are any measures you can take to affect the testability of your system in a way that would lead you to be able to test your system in production. Face it, no matter how you try, your QA systems will never be the same as your production systems. The data, traffic, configurations, scale, timing, etc. will just never match well enough that the tests you run in those environments will catch everything. Change control, make it observable and make sure your system works in production before your customer does it for you!