Article

Heteroscedastic Gaussian process regression

Authors:

Quoc V. Le,

Alex J. Smola,

Stéphane CanuAuthors Info & Claims

ICML '05: Proceedings of the 22nd international conference on Machine learning

Pages 489 - 496

https://doi.org/10.1145/1102351.1102413

Published: 07 August 2005 Publication History

Get Access

Abstract

This paper presents an algorithm to estimate simultaneously both mean and variance of a non parametric regression problem. The key point is that we are able to estimate variance locally unlike standard Gaussian Process regression or SVMs. This means that our estimator adapts to the local noise. The problem is cast in the setting of maximum a posteriori estimation in exponential families. Unlike previous work, we obtain a convex optimization problem which can be solved via Newton's method.

References

[1]

Altun, Y., Hofmann, T., & Smola, A. (2004). Exponential families for conditional random fields. Uncertainty in Artificial Intelligence UAI.

Digital Library

Google Scholar

[2]

Cawley, G., Talbot, N., Foxall, R., Dorling, S., & Mandic, D. (2003). Approximately unbiased estimation of conditional variance in heteroscedastic kernel ridge regression. European Symposium on Artificial Neural Networks (pp. 209--214). d-side.

Google Scholar

[3]

Fine, S., & Scheinberg, K. (2001). Efficient SVM training using low-rank kernel representations. Journal of Machine Learning Research, 2, 243 -- 264. http://www.jmlr.org.

Digital Library

Google Scholar

[4]

Goldberg, P. W., Williams, C. K. I., & Bishop, C. M. (1998). Regression with input-dependent noise: a gaussian process treatment. NIPS 10 (pp. 493--499). Cambridge, MA: MIT Press.

Digital Library

Google Scholar

[5]

Lanckriet, G., Cristianini, N., Bartlett, P., Ghaoui, L. E., & Jordan, M. I. (2004). Learning the kernel matrix with semi-definite programming. Journal of Machine Learning Research, 5, 27 -- 72.

Digital Library

Google Scholar

[6]

Schölkopf, B., Smola, A. J., Williamson, R. C., & Bartlett, P. L. (2000). New support vector algorithms. Neural Computation, 12, 1207 -- 1245.

Digital Library

Google Scholar

[7]

Wahba, G. (1990). Spline models for observational data, vol. 59 of CBMS-NSF Regional Conference Series in Applied Mathematics. Philadelphia: SIAM.

Google Scholar

[8]

Williams, C. K. I. (1998). Prediction with Gaussian processes: From linear regression to linear prediction and beyond. In M. I. Jordan (Ed.), Learning and inference in graphical models, 599 -- 621. Kluwer Academic.

Digital Library

Google Scholar

[9]

Yuan, M., & Wahba, G. (2004). Doubly penalized likelihood estimator in heteroscedastic regression (Technical Report 1084rr). University of Winconsin.

Google Scholar

Cited By

View all

White GSiegel ATovar A(2024)Optimizing Thermoplastic Starch Film with Heteroscedastic Gaussian Processes in Bayesian Experimental Design FrameworkMaterials10.3390/ma1721534517:21(5345)Online publication date: 31-Oct-2024
https://doi.org/10.3390/ma17215345
Udbhav Mallanna DThakur RDwivedi RKurmi VSerra ESpezzano F(2024)Quantifying Uncertainty in Neural Networks through ResidualsProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679983(3714-3718)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679983
Abdulsamad HNickl PKlink PPeters J(2024)Variational Hierarchical Mixtures for Probabilistic Learning of Inverse DynamicsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.331467046:4(1950-1963)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1109/TPAMI.2023.3314670
Show More Cited By

Heteroscedastic Gaussian process regression
1. Mathematics of computing
  1. Probability and statistics
    1. Statistical paradigms
2. Theory of computation
  1. Design and analysis of algorithms

Recommendations

Robust diagnostics for the heteroscedastic regression model

The assumption of equal variance in the normal regression model is not always appropriate. Cook and Weisberg (1983) provide a score test to detect heteroscedasticity, while Patterson and Thompson (1971) propose the residual maximum likelihood (REML) ...
Degeneracy in Heteroscedastic Regression Models

The maximum likelihood estimation in a regression model with heteroscedastic errors is considered. When the design matrices in the model are inappropriately specified, the maximum likelihood estimates of the variances of certain observations are found ...
Maximum likelihood estimation in Gaussian process regression is ill-posed

Gaussian process regression underpins countless academic and industrial applications of machine learning and statistics, with maximum likelihood estimation routinely used to select appropriate parameters for the covariance kernel. However, it remains an ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ICML '05: Proceedings of the 22nd international conference on Machine learning

August 2005

1113 pages

ISBN:1595931805

DOI:10.1145/1102351

General Chair:
Saso Dzeroski
Jozef Stefan Institute, Slovenia
,
Program Chairs:
Luc De Raedt,
Stefan Wrobel

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 August 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

96
Total Citations
View Citations
1,077
Total Downloads

Downloads (Last 12 months)185
Downloads (Last 6 weeks)23

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

White GSiegel ATovar A(2024)Optimizing Thermoplastic Starch Film with Heteroscedastic Gaussian Processes in Bayesian Experimental Design FrameworkMaterials10.3390/ma1721534517:21(5345)Online publication date: 31-Oct-2024
https://doi.org/10.3390/ma17215345
Udbhav Mallanna DThakur RDwivedi RKurmi VSerra ESpezzano F(2024)Quantifying Uncertainty in Neural Networks through ResidualsProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679983(3714-3718)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679983
Abdulsamad HNickl PKlink PPeters J(2024)Variational Hierarchical Mixtures for Probabilistic Learning of Inverse DynamicsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.331467046:4(1950-1963)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1109/TPAMI.2023.3314670
Wang LSun CZhang GZhao KShi HXie Z(2024)Surrogate Assisted Large-Scale Expensive Optimization With Difference-Based Infill Criterion2024 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC60901.2024.10612173(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/CEC60901.2024.10612173
Zhang JSun CZhang GLi JShi HXie Z(2024)Surrogate-Assisted Particle Swarm Optimization with Dual-Subspace Search for Large-Scale Expensive Optimization2024 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC60901.2024.10611757(1-9)Online publication date: 30-Jun-2024
https://doi.org/10.1109/CEC60901.2024.10611757
Deng NAn ZHao JZhang Q(2024)Few shot intelligent assessment method for compaction quality of earthwork considering uncertaintyExpert Systems with Applications10.1016/j.eswa.2024.125000256(125000)Online publication date: Dec-2024
https://doi.org/10.1016/j.eswa.2024.125000
Pankajakshan APal SSnead NAlmeida JBesenhard MAbukhamees SCraig DGavriilidis AMazzei LGalvanin F(2024)MLAPI: A framework for developing machine learning-guided drug particle syntheses in automated continuous flow platformsChemical Engineering Science10.1016/j.ces.2024.120780(120780)Online publication date: Oct-2024
https://doi.org/10.1016/j.ces.2024.120780
Zhou TYe QLuo WRan HShi ZChen J(2024)APPTracker+: Displacement Uncertainty for Occlusion Handling in Low-Frame-Rate Multiple Object TrackingInternational Journal of Computer Vision10.1007/s11263-024-02237-xOnline publication date: 3-Nov-2024
https://doi.org/10.1007/s11263-024-02237-x
Heddar MMehdi BMatougui NTahan SJahazi M(2024)Adaptable multi-objective optimization framework: application to metal additive manufacturingThe International Journal of Advanced Manufacturing Technology10.1007/s00170-024-13489-9132:3-4(1897-1914)Online publication date: 25-Mar-2024
https://doi.org/10.1007/s00170-024-13489-9
Küstner THammernik KRueckert DHepp TGatidis S(2024) Predictive uncertainty in deep learning–based MR image reconstruction using deep ensembles: Evaluation on the fastMRI data set Magnetic Resonance in Medicine10.1002/mrm.3003092:1(289-302)Online publication date: 28-Jan-2024
https://doi.org/10.1002/mrm.30030
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Robust diagnostics for the heteroscedastic regression model

Degeneracy in Heteroscedastic Regression Models

Maximum likelihood estimation in Gaussian process regression is ill-posed