If you came here looking for hints on how to ‘Test In Quality’, you will be disappointed. I hope to show you that it is ‘impossible’ to do this.
What is the purpose of testing? Go ahead, write it down here: The purpose of testing is to _______________. If you are like most people, you probably put down ‘find bugs’, or something equivalent. Even a lot of testers would say that. And that is understandable. Most testing does find bugs. Testers are usually rated on and sometimes paid by the bugs they find. It can be the most fun part of the job. But reasonable as this viewpoint may appear, it shows a lack of understanding of the true purpose of testing in a product development cycle. And the true value of testing in assuring quality.
Consider, if the true purpose of testing was to find bugs, that would imply that true purpose of development was to generate bugs. Hmmmm? Think about it for a bit. Why would you go to any great lengths to look for things which are not supposed to be there? Wouldn’t it make more sense to claim that the purpose of test is to demonstrate that there are no bugs? Or at least that there are an ‘acceptable number of bugs’?
And as for the relationship of testing to quality, testing is a MEASUREMENT of quality. If a test finds no bugs (assuming the test is a good one), then the quality of the test subject can be said to be high. If a test finds a lot of bugs, then the quality of the test subject can be said to be low.
In other words, once a product is turned over to test, the quality of the product is already determined. The things which have already happened and which determined the quality are: the process chosen to develop the product, the care taken in the intellectual development of the product, the care taken in the coding of the product, and the degree of interaction between these three aspects of the development.
Testing has no effect on the quality at all; it merely evaluates the quality. Now, I’m sure there are many people raising their hand, saying, ‘But the bugs found will be fixed, thus improving the quality’. Sorry, the quality of that original product is exactly the same as it was when it entered test. The instant development was complete, the quality was cast in stone. Of course, often bugs WILL be fixed, and sometimes the new version of the product DOES have better quality. But this improvement is not the ‘result’ of testing. It is the result of a ‘new’, better, product being produced when testing determined that the previous product was inadequate.
Notice that time point ‘development complete’, where the quality is finalized. This is the dividing line between development and test. Test-like activities before this time point CAN affect the quality because the product is still being created. A good process will have ‘reviews’ of each stage to weed out requirements specification, architecture and design flaws. Coding standards, code reviews and coding tools can prevent or remove many coding errors. Path and branch ‘inspection’ (technically a test, but not part of the ‘test’ phase of development) during coding can find some more coding errors and lead to them being fixed before the product is ‘completed’.
In a perfect world, development would result in a product with no bugs, and the test would prove there were no bugs. In the real world, it would be so costly and so time consuming to attempt to have no bugs at all, that few if any products could afford it. And even if it were attempted, the odds of success would be slim. So, realistic tests should expect to find some bugs. Although not proving the product is ‘perfect’, it does show how good (or bad) it is, and provides guidance on how to produce a new product (usually by modifying the old one) which should be better.
Balancing the resources and time required to prevent bugs from being created, and that used to find and fix the bugs which are created, is a difficult and key aspect of designing a product development cycle which efficiently produces high quality products.