Least Squares Estimation using Sketched Data with Heteroskedastic Errors

Sokbae Lee, Serena Ng
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:12498-12520, 2022.

Abstract

Researchers may perform regressions using a sketch of data of size m instead of the full sample of size n for a variety of reasons. This paper considers the case when the regression errors do not have constant variance and heteroskedasticity robust standard errors would normally be needed for test statistics to provide accurate inference. We show that estimates using data sketched by random projections will behave ’as if’ the errors were homoskedastic. Estimation by random sampling would not have this property. The result arises because the sketched estimates in the case of random projections can be expressed as degenerate U-statistics, and under certain conditions, these statistics are asymptotically normal with homoskedastic variance. We verify that the conditions hold not only in the case of least squares regression when the covariates are exogenous, but also in instrumental variables estimation when the covariates are endogenous. The result implies that inference can be simpler than the full sample case if the sketching scheme is appropriately chosen.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-lee22i, title = {Least Squares Estimation using Sketched Data with Heteroskedastic Errors}, author = {Lee, Sokbae and Ng, Serena}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {12498--12520}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/lee22i/lee22i.pdf}, url = {https://proceedings.mlr.press/v162/lee22i.html}, abstract = {Researchers may perform regressions using a sketch of data of size m instead of the full sample of size n for a variety of reasons. This paper considers the case when the regression errors do not have constant variance and heteroskedasticity robust standard errors would normally be needed for test statistics to provide accurate inference. We show that estimates using data sketched by random projections will behave ’as if’ the errors were homoskedastic. Estimation by random sampling would not have this property. The result arises because the sketched estimates in the case of random projections can be expressed as degenerate U-statistics, and under certain conditions, these statistics are asymptotically normal with homoskedastic variance. We verify that the conditions hold not only in the case of least squares regression when the covariates are exogenous, but also in instrumental variables estimation when the covariates are endogenous. The result implies that inference can be simpler than the full sample case if the sketching scheme is appropriately chosen.} }
Endnote
%0 Conference Paper %T Least Squares Estimation using Sketched Data with Heteroskedastic Errors %A Sokbae Lee %A Serena Ng %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-lee22i %I PMLR %P 12498--12520 %U https://proceedings.mlr.press/v162/lee22i.html %V 162 %X Researchers may perform regressions using a sketch of data of size m instead of the full sample of size n for a variety of reasons. This paper considers the case when the regression errors do not have constant variance and heteroskedasticity robust standard errors would normally be needed for test statistics to provide accurate inference. We show that estimates using data sketched by random projections will behave ’as if’ the errors were homoskedastic. Estimation by random sampling would not have this property. The result arises because the sketched estimates in the case of random projections can be expressed as degenerate U-statistics, and under certain conditions, these statistics are asymptotically normal with homoskedastic variance. We verify that the conditions hold not only in the case of least squares regression when the covariates are exogenous, but also in instrumental variables estimation when the covariates are endogenous. The result implies that inference can be simpler than the full sample case if the sketching scheme is appropriately chosen.
APA
Lee, S. & Ng, S.. (2022). Least Squares Estimation using Sketched Data with Heteroskedastic Errors. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:12498-12520 Available from https://proceedings.mlr.press/v162/lee22i.html.

Related Material