Here's my response:
It's important to not lose sight of the primary goal of testing. ie to ensure that the product is released with as few bugs as possible (with the highest quality possible). Automation plays A role in that effort, not THE role.
For functional testing, bugs found via automation tends to be of 2 basic categories. Those found during test case development (ie test coding), and during regression test runs. The first category of bugs is directly correlated to the quality of the test case being automated. The second category of bugs is as a result of code changes and can be caught before it makes it into the build eg. via a pre-checkin system.
Performance and stress testing can only (usually) be performed using automated tests. Performance testing needs to be automated to provide the repeatability. Stress testing needs to be automated to provide the ability to load and\or hammer the system.
For me, the key indicator is where testers are spending their time, and the result of that effort. If testers are spending significant amounts of time on automation, but they are not discovering a whole lot of bugs, then that's a red flag.
Nothing beats an intelligent knowledegeable tester spending time figuring out how to test the feature. It is the result of that intellectual effort that drives the quality of the test effort. It is not the tool, technique, automated vs manual etc that drives the quality of the test effort.
And, as everything else in life, a balance is needed. An overemphasis of one technique of testing will not be beneficial to the ultimate goal of shipping high quality software.
No comments:
Post a Comment