Abstract:
Receiver operating characteristic (ROC) graphs
are useful for organising binary classifiers and visualising
their performance. In order to compare classifiers it may be
needed to reduce the ROC performance to a single scalar value
representing expected performance. Such a commonly used
summary statistic is the area under the curve (AUC) of the
ROC curve. The AUCs can be estimated either parametrically
or non-parametrically. The parametric approach assumes that
the signal present (positive) and signal absent (negative) groups
can be represented as two overlapping Gaussian distributions.
If the observations of two or more ROC curves are obtained
from the same region of interest, their AUCs are considered to
be correlated.
A novel asymptotic test for comparing multiple AUCs of
several ROC curves was proposed by Meyen and Sooriyarachchi
in 2014, and it was of interest to study the behaviour of the
test statistic for various sample sizes and varying degrees of
overlap between the Gaussian distributions via a simulation
study. Hence this study was carried out to test the properties of
the test statistic when the AUCs were estimated parametrically
by Dorfman and Alf’s method. This simulation was carried out
for the case where the AUCs are independent.
Inferences were made regarding the distribution of the test
statistic for various sample sizes. The test statistic performed
better when the spread between the two Gaussian distributions
increased, while the test statistic was valid with respect to
sample sizes above 100 when 2 ROC curves were being
compared simultaneously.
