Investigating Coverage-Reliability Relationship and Sensitivity of Reliability Estimates to Errors in the Operational Profile

Mei-Hwa Chen, Praerit Garg, Aditya P. Mathur and Vernon J. Rego

Software Engineering Research Center
Department of Computer Sciences
Purdue University
W. Lafayette, In 47907

ABSTRACT

The focus of this work is an investigation into (a)~the correlation between ``true'' reliability of a software system and the white box testing measures such as block coverage, c-uses and p-uses coverage and (b) the sensitivity of reliability estimates to errors in operational profile. We believe that software reliability and testing measures, especially white box testing, are inherently related. Results from experiments are presented to support this belief. We also report a case study to investigate the effect of errors in an operational profile on reliability estimates. Two models for reliability estimation were considered: the Musa-Okumoto model and the Goel-Okumoto model. It was found that the error in reliability estimates from these models grows nonlinearly with errors in operational profile. Results from this case study lend credit to the argument that further research is necessary in development of more robust models for reliability estimation.