On the problem of bias in error rate estimation for discriminant analysis

摘要

This paper examines the question of how to make a determination of success for discriminant analysis. Given that the basis for such a judgment lies in estimates of expected error rates, the problem resolves to a comparative study of error rate estimators.We compare several techniques in a setting of high true probabilities of misclassification with varying sample sizes and numbers of groups. Interpretation of the results centers on the problem of estimators which consistently underestimate the true probability of misclassification. Experimental circumstances which tend to amplify this bias are discussed. Estimators which do not suffer from this bias are also presented and discussed.