Wednesday, June 3, 2009

Testing


Came across this interesting question on Linked in about code coverage and testing.
Here's my response to the question:

Ultimately, software testing is limited by time, specifically time to ship\release. Software that is 100% bug free is worthless if it's not in the customer's hands. Software that is 100% bug free is also an impossibility. Along the lines of Godel's incompleteness theorem.

100% code coverage has nothing to do about flushing out all bugs. A simple and often quoted example is as thus: Take the following code -

for (int i = 0; i < 10; i++)
{
//do something
}

So you have a test that hits the above code. You get 100% code coverage. But what if the value of the limit in the loop (10) was wrong? That's a bug. But code coverage didn't not expose it.

An oft-repeated phrase in the software engineering field is that testing does not prove the absence of bugs, but rather the presence of bugs.

The most pragmatic goal in testing (IMO) is testing the product to a good enough level of quality for release. The challenge is figuring out what "good enough" implies, and how to achieve it. Nailing this conundrum is what differentiates an effective test effort from one that isn't.

No comments:

Post a Comment