Simulation study of a novel method for comparing more than two independent receiver operating characteristic (ROC) curves based on the area under the curves (AUCs).

Abstract

Receiver operating characteristic (ROC) graphs are useful for organising binary classifiers and visualising their performance. In order to compare classifiers it may be needed to reduce the ROC performance to a single scalar value representing expected performance. Such a commonly used summary statistic is the area under the curve (AUC) of the ROC curve. The AUCs can be estimated either parametrically or non-parametrically. The parametric approach assumes that the signal present (positive) and signal absent (negative) groups can be represented as two overlapping Gaussian distributions. If the observations of two or more ROC curves are obtained from the same region of interest, their AUCs are considered to be correlated. A novel asymptotic test for comparing multiple AUCs of several ROC curves was proposed by Meyen and Sooriyarachchi in 2014, and it was of interest to study the behaviour of the test statistic for various sample sizes and varying degrees of overlap between the Gaussian distributions via a simulation study. Hence this study was carried out to test the properties of the test statistic when the AUCs were estimated parametrically by Dorfman and Alf’s method. This simulation was carried out for the case where the AUCs are independent. Inferences were made regarding the distribution of the test statistic for various sample sizes. The test statistic performed better when the spread between the two Gaussian distributions increased, while the test statistic was valid with respect to sample sizes above 100 when 2 ROC curves were being compared simultaneously.