Fast Gaussian Process Regression for Big Data

Article Type

Research Article

Publication Title

Big Data Research

Abstract

Gaussian Processes are widely used for regression tasks. A known limitation in the application of Gaussian Processes to regression tasks is that the computation of the solution requires performing a matrix inversion. The solution also requires the storage of a large matrix in memory. These factors restrict the application of Gaussian Process regression to small and moderate size datasets. We present an algorithm that combines estimates from models developed using subsets of the data obtained in a manner similar to the bootstrap. The sample size is a critical parameter for this algorithm. Guidelines for reasonable choices of algorithm parameters, based on a detailed experimental study, are provided. Various techniques have been proposed to scale Gaussian Processes to large-scale regression tasks. The most appropriate choice depends on the problem context. The proposed method is most appropriate for problems where an additive model works well and the response depends on a small number of features. The minimax rate of convergence for such problems is attractive and we can build effective models with a small subset of the data. The Stochastic Variational Gaussian Process and the Sparse Gaussian Process are also appropriate choices for such problems. Results from experiments conducted as part of this study indicate that the algorithm presented in this work can be as effective as these methods. Unlike these methods, the proposed algorithm requires minimal hyper-parameter tuning and is much simpler to implement. The rate of convergence is also attractive.

First Page

12

Last Page

26

DOI

10.1016/j.bdr.2018.06.002

Publication Date

12-1-2018

Comments

All Open Access, Green

This document is currently not available here.

Share

COinS