SMLBoost-adopting a soft-margin like strategy in boosting

摘要

Boosting is well known for its excellence on regular datasets, nevertheless, boosting can also easily overfit the wrong patterns when the training set contains noise. This paper aims at tackling boosting’s noise sensitivity via simulating a soft-margin like procedure. In contrast to the previous studies, which build soft-margin via introducing some regularization terms into their loss functions, this paper aims to build a clean marginal (border) region near the classification hyperplane, thereby, the principle of margin can be rebuilt for boosting in the noisy scenario. The key ingredients of the proposed method consist in a noise detection strategy and a weight updating mechanism, which progressively assigns higher weight to the un-noisy instances that have been pushed into the marginal region defined by an unsupervised margin. We conducted experiments on 20 UCI datasets and 44 imbalanced datasets from KEEL. Experimental results not only demonstrate the superiority of the proposed method over other robust boostings on noisy datasets, they also confirm when combined with oversampling technique that might erroneously generate noisy samples to balance the class distribution, the proposed method can deliver promising imbalanced learning ability.