Scaled rotation regularization

摘要

A new regularization method - a scaled rotation - is proposed and compared with the standard linear regularized discriminant analysis. A sense of the method consists in the singular value decomposition S=TDT′ of a sample covariance matrix S and a use of the following representation of an inverse of the covariance matrix S−1=Tα(D+λI)−1Tα′. For certain data structures the scaled rotation helps to reduce the generalization error in small learning-set and high dimensionality cases. Efficacy of the scaled rotation increases if one transforms the data by y=(D+λI)−1/2Tα′x and uses an optimally stopped single layer perceptron classifier afterwards.