Discriminative least squares regression for multiclass classification based on within-class scatter minimization

摘要

Least square regression has been widely used in pattern classification, due to the compact form and efficient solution. However, two main issues limit its performance for solving the multiclass classification problems. The first one is that employing the hard discrete labels as the regression targets is inappropriate for multiclass classification. The second one is that it focus only on exactly fitting the instances to the target matrix while ignoring the within-class similarity of the instances, resulting in overfitting. To address this issues, we propose a discriminative least squares regression for multiclass classification based on within-class scatter minimization (WCSDLSR). Specifically, a ε-dragging technique is first introduced to relax the hard discrete labels into the slack soft labels, which enlarges the between-class margin for the soft labels as much as possible. The within-class scatter for the soft labels is then constructed as a regularization term to make the transformed instances of the same class closer to each other. These factors ensure WCSDLSR can learn a more compact and discriminative transformation for classification, thus avoiding the overfitting problems. Furthermore, the proposed WCSDLSR can obtain a closed-form solution in each iteration with the lower computational costs. Experimental results on the benchmark datasets demonstrate that the proposed WCSDLSR achieves the better classification performance with the lower computational costs.