Combinatorial and MC/DC Coverage Levels of Random Testing

Software testing criteria differ in their effectiveness, the numbers of test cases required, and the processes of test generation. Specific criteria often are compared to random testing, and in some cases, random testing shows a surprisingly high level of effectiveness. One reason that this is the case is that any random test set has a specific level of coverage according to any coverage criterion. Numerical evaluation of coverage levels of random testing according to various coverage criteria is an interesting research task and is important in understanding the relationship between different testing approaches. In this paper, we performed an experimental evaluation of the coverage levels of random testing for two criteria: MC/DC and combinatorial t-way testing. Our experiments showed that, when the number of random test cases increased, a high level of coverage was reached rapidly, both for MC/DC and t-way. However, many more random tests are required to reach 100% coverage. An unexpected result was that there were significant differences in the measurement of partial MC/DC coverage by various tools. The results may be used to select optimal methods for practical testing and develop new testing methods based on the integration of existing approaches.