On the dominance of non-parametric Bayes rule discriminant algorithms in high dimensions

摘要

Considerable attention has been given to the relative performance of the various commonly used discriminant analysis algorithms. This performance has been studied under varying conditions. This author and others have been particularly interested in the behavior of the algorithms as dimension is varied. Here we consider three basic questions: which algorithms perform better in high dimensions, when does it pay to add or delete a dimension, and how discriminant algorithms are best implemented in high dimensions.One of the more interesting results has been the relatively good performance of non-parametric Bayes theorem type algorithms compared to parametric (linear and quadratic) algorithms. Surprisingly this effect occurs even when the underlying distributions are “ideal” for the parametric algorithms, provided, at least, that the true covariance matrices are not too close to singular. Monte Carlo results presented here further confirm this unexpected behavior and add to the existing literature (particularly Van Ness(9) and Van Ness et al.(11) by studying a different class of underlying Gaussian distributions. These and earlier results point out certain procedures, discussed here, which should be used in the selection of the density estimation windows for non-parametric algorithms to improve their performance. Measures of the effect on the various algorithms of adding dimensions are given graphically. A summary of some of the conclusions about several of the more common algorithms is included.