Evolving multi-label fuzzy classifier with advanced robustness respecting human uncertainty

摘要

Multi-label classification has attracted much attention in the machine learning community to address the problem of assigning single samples to more than one class at the same time. We propose an evolving multi-label fuzzy classifier (EFC-ML-FWU) which is able to self-adapt and self-evolve its structure and consequent parameters in the form of multiple hyper-planes with new incoming multi-label samples in an incremental, single-pass manner and which especially addresses the intrinsic curse of dimensionality as well as human label uncertainty problems, often apparent in multi-label classification problems, to ensure the advanced robustness of the learned structure and parameters. The former is achieved by integrating feature weights into the learning process, specifically designed for online multi-label classification problems in an incremental manner, measuring the impact of features with respect to their discriminatory power. The features are integrated (i) into the rule evolution criterion, leading to a shrinkage of distances along unimportant dimensions, which reduces the likelihood of unnecessary rule evolution and thus decreases over-fitting due to the curse of dimensionality, (ii) into the first consequent learning part by a variable-regularized RFWLS approach realized through an incremental coordinate descent algorithm, and (iii) into the second consequent learning part employing correlation-based preservation learning by using weight-based thresholds (extending the classical Lipschitz constant-based threshold) within soft shrinkage operations to optimize a feature-based weighted L1-norm on the consequent parameters. Uncertainty in class labels is handled by an integration of sample weights, where lower weights indicate a higher uncertainty in the labels carried by a sample. This leads to (i) a weighted updating of the incremental feature weights, (ii) a weighted update of the rule antecedent space through a weighted incremental clustering process, and (iii) a specific weighted update of the consequent parameters exploring a single-label and a multi-label view of uncertainty. Our approach was evaluated on several data sets from the MULAN repository and showed significantly improved classification accuracy and average precision trend lines compared to alternative (evolving) one-versus-rest or classifier chaining concepts, and especially improved the native EFC-ML method without feature weights and uncertainty handling with performance gains up to 17% in the AUC of the accuracy trends. Furthermore, interesting insights into an improved robustness of the multi-label classifier (i) in the case of wrong labels due to low user experience levels and (ii) in the case of low label certainties but potentially correct labels were obtained.