Random Feature Expansions for Deep Gaussian Processes

Kurt Cutajar, Edwin V. Bonilla, Pietro Michiardi, Maurizio Filippone
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:884-893, 2017.

Abstract

The composition of multiple Gaussian Processes as a Deep Gaussian Process DGP enables a deep probabilistic nonparametric approach to flexibly tackle complex machine learning problems with sound quantification of uncertainty. Existing inference approaches for DGP models have limited scalability and are notoriously cumbersome to construct. In this work we introduce a novel formulation of DGPs based on random feature expansions that we train using stochastic variational inference. This yields a practical learning framework which significantly advances the state-of-the-art in inference for DGPs, and enables accurate quantification of uncertainty. We extensively showcase the scalability and performance of our proposal on several datasets with up to 8 million observations, and various DGP architectures with up to 30 hidden layers.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-cutajar17a, title = {Random Feature Expansions for Deep {G}aussian Processes}, author = {Kurt Cutajar and Edwin V. Bonilla and Pietro Michiardi and Maurizio Filippone}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {884--893}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/cutajar17a/cutajar17a.pdf}, url = {https://proceedings.mlr.press/v70/cutajar17a.html}, abstract = {The composition of multiple Gaussian Processes as a Deep Gaussian Process DGP enables a deep probabilistic nonparametric approach to flexibly tackle complex machine learning problems with sound quantification of uncertainty. Existing inference approaches for DGP models have limited scalability and are notoriously cumbersome to construct. In this work we introduce a novel formulation of DGPs based on random feature expansions that we train using stochastic variational inference. This yields a practical learning framework which significantly advances the state-of-the-art in inference for DGPs, and enables accurate quantification of uncertainty. We extensively showcase the scalability and performance of our proposal on several datasets with up to 8 million observations, and various DGP architectures with up to 30 hidden layers.} }
Endnote
%0 Conference Paper %T Random Feature Expansions for Deep Gaussian Processes %A Kurt Cutajar %A Edwin V. Bonilla %A Pietro Michiardi %A Maurizio Filippone %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-cutajar17a %I PMLR %P 884--893 %U https://proceedings.mlr.press/v70/cutajar17a.html %V 70 %X The composition of multiple Gaussian Processes as a Deep Gaussian Process DGP enables a deep probabilistic nonparametric approach to flexibly tackle complex machine learning problems with sound quantification of uncertainty. Existing inference approaches for DGP models have limited scalability and are notoriously cumbersome to construct. In this work we introduce a novel formulation of DGPs based on random feature expansions that we train using stochastic variational inference. This yields a practical learning framework which significantly advances the state-of-the-art in inference for DGPs, and enables accurate quantification of uncertainty. We extensively showcase the scalability and performance of our proposal on several datasets with up to 8 million observations, and various DGP architectures with up to 30 hidden layers.
APA
Cutajar, K., Bonilla, E.V., Michiardi, P. & Filippone, M.. (2017). Random Feature Expansions for Deep Gaussian Processes. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:884-893 Available from https://proceedings.mlr.press/v70/cutajar17a.html.

Related Material