An evolutionary scheme for decision tree construction

摘要

Classification is a central task in machine learning and data mining. Decision tree (DT) is one of the most popular learning models in data mining. The performance of a DT in a complex decision problem depends on the efficiency of its construction. However, obtaining the optimal DT is not a straightforward process. In this paper, we propose a new evolutionary meta-heuristic optimization based approach for identifying the best settings during the construction of a DT. We designed a genetic algorithm coupled with a multi-task objective function to pull out the optimal DT with the best parameters. This objective function is based on three main factors: (1) Precision over the test samples, (2) Trust in the construction and validation of a DT using the smallest possible training set and the largest possible testing set, and (3) Simplicity in terms of the size of the generated candidate DT, and the used set of attributes. We extensively evaluate our approach on 13 benchmark datasets and a fault diagnosis dataset. The results show that it outperforms classical DT construction methods in terms of accuracy and simplicity. They also show that the proposed approach outperforms Ant-Tree-Miner (an evolutionary DT construction approach), Naive Bayes and Support Vector Machine in terms of accuracy and F-measure.