ABSTRACT
Receiver Operating Characteristic (ROC) analysis has become the standard statistical methodology for evaluating the diagnostic performance of imaging systems that require a human observer or a computer program to make a diagnostic decision. ROC parameters such as the area under the ROC curve and the index of detectability are useful descriptors of diagnostic performance because they are independent of the bias produced by the variation in the application of decision criteria by the observers. The report describes the basic decision model, the approach to ROC curve-fitting and current experimental and statistical methodology. The multi-reader multi-case (MRMC) approach that utilizes the jackknife technique for resampling cases and the analysis of variance (ANOVA) for the final analysis is stressed. Newer advances that are in progress including analyses of location (LROC) and analysis of cases with multiple lesions (FROC) are included. The report provides some practical guidelines for the design and execution of clinical and laboratory ROC studies. There is a bibliography of historically important and currently relevant publications. Appendices contains web addresses where ROC analysis software can be obtained, an example of an experiment done using ROC analysis, and an example of a statistical power calculation.