Nested aggregation of experts using inducing points for approximated Gaussian process regression

摘要

Gaussian process regression is a flexible regression scheme but suffers from its high computational complexity regarding the inversion of a matrix with the same size as the training dataset. Aggregation method is one of the approximation techniques for reducing the complexity. In this paper, we propose a novel aggregation method, Nested Aggregation of Experts using Inducing Points (NAE-IP), which is an extension of a conventional method and enables dimensionality reduction by making use of the idea of linear sketching. There are some options for selecting inducing points in the proposed method. The options can introduce test points of interest as inducing points, albeit at the cost of slightly higher computational complexity. The other options exploiting less informative inducing points can yield a significant reduction of the computational complexity. The proposed NAE-IP is theoretically guaranteed to have consistency under certain conditions. Results of our computational experiments using synthetic and real data show that the proposed method achieves lower prediction error and even lower computing time than conventional methods.