An iterative model-free feature screening procedure: Forward recursive selection

摘要

Many researchers have studied the combinations of machine learning techniques and traditional statistical strategies, and proposed effective procedures for complicated data sets. Yet, there is still some lack of running time and prediction accuracy. In this paper, we propose an iterative feature screening procedure, named forward recursive selection. We combine the random forest and forward selection to address the model-based limitations and the related requirements. We also use the forward strategy with a limited number of iterations to improve the computational efficiency. To provide the theoretical guarantees of this method, we calculate functions of the permutation importance of this algorithm in different models and data with group structures. Numerical comparisons and empirical analysis support our results, and the proposed procedure works well.