Regularized Matrix-Pattern-Oriented Classification Machine with Universum

摘要

Regularization has the ability to effectively improve the generalization performance, which is due to its control for model complexity via priori knowledge. Matrixized learning as one kind of regularization methods can help improving classification accuracy and reducing computational complexity when dealing with matrix data. This success is attributed to the exploitation of structural knowledge of matrix data. This paper generalizes the matrixized learning through taking the advantage of the Universum data which does not belong to any class of interest in classification problems. The generalized method can not only keep the structural knowledge of the matrix data themselves, but also acquire a priori domain knowledge from the whole data distribution. In implementation, we incorporate the previous matrixized work MatMHKS with the Universum strategy, and develop a novel regularized matrix-pattern-oriented classification machine named UMatMHKS. The subsequential experiments have validated the effectiveness of the proposed UMatMHKS. The results has shown that the proposed UMatMHKS achieved an improvement in classification accuracy of 1.52% over the MatMHKS and 3.20% over the USVM on UCI benchmark datasets. The UMatMHKS also has a shorter average running time of 0.41 s over the 0.71 s from the MatMHKS on UCI datasets. Three main characteristics of UMatMHKS lie in: (1) making full use of the domain knowledge of the whole data distribution as well as inheriting the advantages of the matrixized learning; (2) applying Universum learning into the matrixized learning framework; (3) owning a tighter generalization risk bound.