Contraction rates for sparse variational approximations in Gaussian process regression

Dennis Nieman; Botond Szabo; Harry van Zanten

Contraction rates for sparse variational approximations in Gaussian process regression

Dennis Nieman
, Botond Szabo
, Harry van Zanten

Mathematics

Research output: Contribution to Journal › Article › Academic › peer-review

Abstract

We study the theoretical properties of a variational Bayes method in the Gaussian Process regression model. We consider the inducing variables method introduced by Titsias (2009b) and derive sufficient conditions for obtaining contraction rates for the corresponding variational Bayes (VB) posterior. As examples we show that for three particular covariance kernels (Matérn, squared exponential, random series prior) the VB approach can achieve optimal, minimax contraction rates for a sufficiently large number of appropriately chosen inducing variables. The theoretical findings are demonstrated by numerical experiments.

Original language	English
Article number	205
Pages (from-to)	1-26
Number of pages	26
Journal	Journal of Machine Learning Research
Volume	23
Early online date	22 Jun 2022
Publication status	Published - 2022

Bibliographical note

Funding Information:
This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101041064).

Publisher Copyright:
©2022 Dennis Nieman, Botond Szabo and Harry van Zanten.

Funding

This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101041064).

Keywords

contraction rates
Gaussian Process regression
inducing variables
Variational Bayes

Access to Document

https://www.jmlr.org/papers/volume23/21-1128/21-1128.pdfLicence: CC BY

Persistent URL (handle)

Persistent link

Cite this

@article{9c8aaeb88f2b4fe1b6162d1992875374,

title = "Contraction rates for sparse variational approximations in Gaussian process regression",

abstract = "We study the theoretical properties of a variational Bayes method in the Gaussian Process regression model. We consider the inducing variables method introduced by Titsias (2009b) and derive sufficient conditions for obtaining contraction rates for the corresponding variational Bayes (VB) posterior. As examples we show that for three particular covariance kernels (Mat{\'e}rn, squared exponential, random series prior) the VB approach can achieve optimal, minimax contraction rates for a sufficiently large number of appropriately chosen inducing variables. The theoretical findings are demonstrated by numerical experiments.",

keywords = "contraction rates, Gaussian Process regression, inducing variables, Variational Bayes",

author = "Dennis Nieman and Botond Szabo and \{van Zanten\}, Harry",

note = "Funding Information: This project has received funding from the European Research Council (ERC) under the European Union{\textquoteright}s Horizon 2020 research and innovation programme (grant agreement No. 101041064). Publisher Copyright: {\textcopyright}2022 Dennis Nieman, Botond Szabo and Harry van Zanten.",

year = "2022",

language = "English",

volume = "23",

pages = "1--26",

journal = "Journal of Machine Learning Research",

issn = "1532-4435",

publisher = "Microtome Publishing",

}

TY - JOUR

T1 - Contraction rates for sparse variational approximations in Gaussian process regression

AU - Nieman, Dennis

AU - Szabo, Botond

AU - van Zanten, Harry

N1 - Funding Information: This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101041064). Publisher Copyright: ©2022 Dennis Nieman, Botond Szabo and Harry van Zanten.

PY - 2022

Y1 - 2022

N2 - We study the theoretical properties of a variational Bayes method in the Gaussian Process regression model. We consider the inducing variables method introduced by Titsias (2009b) and derive sufficient conditions for obtaining contraction rates for the corresponding variational Bayes (VB) posterior. As examples we show that for three particular covariance kernels (Matérn, squared exponential, random series prior) the VB approach can achieve optimal, minimax contraction rates for a sufficiently large number of appropriately chosen inducing variables. The theoretical findings are demonstrated by numerical experiments.

AB - We study the theoretical properties of a variational Bayes method in the Gaussian Process regression model. We consider the inducing variables method introduced by Titsias (2009b) and derive sufficient conditions for obtaining contraction rates for the corresponding variational Bayes (VB) posterior. As examples we show that for three particular covariance kernels (Matérn, squared exponential, random series prior) the VB approach can achieve optimal, minimax contraction rates for a sufficiently large number of appropriately chosen inducing variables. The theoretical findings are demonstrated by numerical experiments.

KW - contraction rates

KW - Gaussian Process regression

KW - inducing variables

KW - Variational Bayes

UR - https://www.scopus.com/pages/publications/85140760362

UR - https://www.scopus.com/pages/publications/85140760362#tab=citedBy

UR - https://www.jmlr.org/papers/v23/

M3 - Article

AN - SCOPUS:85140760362

SN - 1532-4435

VL - 23

SP - 1

EP - 26

JO - Journal of Machine Learning Research

JF - Journal of Machine Learning Research

M1 - 205

ER -

Contraction rates for sparse variational approximations in Gaussian process regression

Abstract

Bibliographical note

Funding

Keywords

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this