Software testing is…

Over the years many experts have proposed various definitions for software testing, some common definitions of testing including:

… The process of executing a program with the intent of finding errors, (Myers, G. J., the art of software testing, Wiley, 1979)

“Execute a program or system with the intention of finding errors.”-Myers, 1979

Software testing is a process of analyzing or operating software for the purpose of finding bugs. (Chris Brown, Gray Cobb, Robert Culbertson, Introduction to Rapid Software Testing, 2001)

May experts in the field seem to advocate the primary purpose of testing is to find defects; as many as possible as early in the software development lifecycle as possible. Unfortunately there are 2 fundamental problems with this simplistic interpretation of software testing.

Finding as many as defects as we can as early as possible can make testing a numbers game. Does a large bug count imply testing is doing a good job? High numbers of bugs don' t always mean the testing effort is finding the most important critical issues as early as possible. Another potential side effect of simply finding as many as bugs as possible includes artificial explosion of bug count. Defects manifest themselves in many ways; if a tester doesn' t take time to investigate or understand the cause of a bug then they could write several bug reports for a single issue. Unfortunately, novice testers often believe that by finding as many as defects as they can they are improving software quality. YOU CANNOT TEST IN QUALITY.

When the test engineer is focused on finding more bugs they tend to skip over areas or take things for granted. Antidotal evidence suggests that testers often miss simple errors because they are too focused on finding as many as bugs as possible or finding the most heinous bugs. For example, testing boundary conditions some testers may exercise the specific valid boundary values and then try invalid values well above or below the valid boundary values instead of minimum -1 and maximum +1. Also, many testers simply overlook the minimum +1 value and the maximum -1 value assuming if the minimum and maximum value worked so everything in between the range of minimum and max is the valid equivalence class. This oversight can lead to missed errors in arrays or in assignments for ranges of data.