An Empirical Study of the Correlation between Code Coverage
and Reliability Estimation
Mei-Hwa Chen, Michael R. Lyu and W. Eric Wong
ABSTRACT
Existing time-domain models for software reliability often
result in an overestimation of such reliability
because they do not take the nature of testing techniques into account.
Since every testing technique has a limit
to its ability to reveal faults in a given system,
as a technique approaches its saturation region
fewer faults are discovered and reliability growth phenomena
are predicted from the models.
When the software is turned over to field operation,
significant overestimates of reliability are observed.
In this paper, we present a technique to solve this
problem by addressing both time and coverage measures
for the prediction of software failures.
Our technique uses coverage information
collected during testing to extract only effective
data from a given operational profile.
Execution time between test cases
which neither increase coverage nor cause a failure
is reduced by a parameterized factor.
Experiments using this technique were conducted on a program
created in a simulation environment with simulated faults
and on an industrial automatic flight control project which
contained several natural faults.
Results from both experiments indicate that overestimation of
reliability is reduced significantly using our technique.
This new approach not only helps reliability growth models make
more accurate predictions, but also reveals the efficiency of a testing
profile so that more effective testing techniques can be conducted.