article

Logistic regression diagnostics in ridge regression

Authors:

M. Revan Özkale,

Stanley Lemeshow,

Rodney SturdivantAuthors Info & Claims

Computational Statistics, Volume 33, Issue 2

Pages 563 - 593

https://doi.org/10.1007/s00180-017-0755-x

Published: 01 June 2018 Publication History

Abstract

The adverse effects of multicollinearity and unusual observations are seen in logistic regression and attention had been given in the literature to each of these problems separately. However, multicollinearity and unusual observations can arise simultaneously in logistic regression. The objective of this paper is to propose the statistics for detecting the unusual observations in an ill-conditioned data set under the ridge logistic estimator. A numerical example and two Monte Carlo simulation studies are used to illustrate the methodology. The present investigation shows that ridge logistic estimation copes with unusual observations by downweighting their influence.

References

[1]

Atkinson AC (1985) Plots, transformations and regression. Clarendon Press, Oxford.

[2]

Cook RD (1977) Detection of influential observation in linear regression. Technometrics 19(1):15-18.

[3]

Cook D, Weisberg S (1982) Residuals and influence in regression. Chapman and Hall, New York.

[4]

Duffy DE, Santner TJ (1989) On the small sample properties of norm restricted maximum likelihood estimators for logistic regression models. Commun Stat Theory Methods 18(3):959-980.

[5]

Hoerl AE, Kennard RW, Baldwin KF (1975) Ridge regression: some simulations. Commun Stat Theory Methods 4(2):105-123.

[6]

Hosmer DW, Lemeshow S, Sturdivant RX (2013) Applied logistic regression, 3rd edn. Wiley, Hoboken.

[7]

Jahufer A, Jianbao C (2009) Assessing global influential observations in modified ridge regression. Stat Probab Lett 79(4):513-518.

[8]

Jennings DE (1986) Outliers and residual distributions in logistic regression. J Am Stat Assoc 81(396):987-990.

[9]

Lee AH, Silvapulle MJ (1988) Ridge estimation in logistic regression. Commun Stat Simul 17(4):1231-1257.

[10]

LeCessie S, VanHouwelingen JC (1992) Ridge estimators in logistic regression. Appl Stat 41(1):191-201.

[11]

Lesaffre E, Albert A (1989) Multiple group logistic regression diagnostics. J R Stat Soc Ser C Appl Stat 38:425-440.

[12]

Lesaffre E, Marx BD (1993) Collinearity in generalized linear regression. Commun Stat Theory Methods 22(7):1933-1952.

[13]

McDonald GC, Galarneau DI (1975) A Monte Carlo evaluation of some ridge-type estimators. J Am Stat Assoc 70(350):407-416.

[14]

McCullagh P, Nelder JA (1989) Generalized linear models, 2nd edn. Chapman & Hall, Boca Raton.

[15]

Özkale MR (2013) Influence measures in affine combination type regression. J Appl Stat 40(10):2219-2243.

[16]

Pregibon D (1981) Logistic regression diagnostics. Ann Stat 9(4):705-724.

[17]

Preisser JS, Garcia DI (2005) Alternative computational formulae for generalized linear model diagnostics: identifing influential observations with SAS software. Comput Stat Data Anal 48:755-764.

[18]

Schaefer RL, Roi LD, Wolfe RA (1984) A ridge logistic estimator. Commun Stat Theory Methods 13(1):99-113.

[19]

Schott JR (2005) Matrix analysis for statistics. Wiley, Hoboken.

[20]

Smith EP, Marx BD (1990) Ill-conditioned information matrices, generalized linear models and estimation of the effects of acid rain. Environmetrics 1(1):57-71.

[21]

Wahba G, Golub GH, Heath M (1979) Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21(2):215-223.

[22]

Walker E, Birch JB (1988) Influence measures in ridge regression. Technometrics 30(2):221-227.

[23]

Weissfeld LA, Sereika SM (1991) A multicollinearity diagnostics for generalized linear models. Commun Stat Theory Methods 20(4):1183-1198.

[24]

Williams DA (1987) Generalized linear model diagnostics using the deviance and single case deletions. J R Stat Soc Ser C Appl Stat 36(2):181-191.

Cited By

Yassen MAl-Duais FAlmazah M(2022)Ridge Regression Method and Bayesian Estimators under Composite LINEX Loss Function to Estimate the Shape Parameter in Lomax DistributionComputational Intelligence and Neuroscience10.1155/2022/12006112022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/1200611
Choi MOh SChoi JKim JLee S(2022)A statistical learning framework for predicting left ventricular ejection fraction based on glutathione peroxidase-3 level in ischemic heart diseaseComputers in Biology and Medicine10.1016/j.compbiomed.2022.105929149:COnline publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1016/j.compbiomed.2022.105929
Khan AAmanullah MAmin MAlharbi RMuse AMohamed M(2021)Influence Diagnostic Methods in the Poisson Regression Model with the Liu EstimatorComputational Intelligence and Neuroscience10.1155/2021/44073282021Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1155/2021/4407328
Show More Cited By

Recommendations

On linearized ridge logistic estimator in the presence of multicollinearity
Abstract
Logistic Regression is a very popular method to model the dichotomous data. The maximum likelihood estimator (MLE) of unknown regression parameters of the logistic regression is not too accurate when multicollinearity exists among the covariates. ...
A first-order approximated jackknifed ridge estimator in binary logistic regression

The purpose of this paper is to solve the problem of multicollinearity that affects the estimation of logistic regression model by introducing first-order approximated jackknifed ridge logistic estimator which is more efficient than the first-order ...
Influence measures in ridge regression when the error terms follow an Ar(1) process

Influence concepts have an important place in linear regression models and case deletion is a useful method for assessing the influence of single case. The influence measures in the presence of multicollinearity were discussed under the linear ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computational Statistics

Computational Statistics Volume 33, Issue 2

June 2018

552 pages

ISSN:0943-4062

EISSN:1613-9658

Issue’s Table of Contents

Copyright © Copyright © 2018 Springer-Verlag GmbH Germany, part of Springer Nature.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 June 2018

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yassen MAl-Duais FAlmazah M(2022)Ridge Regression Method and Bayesian Estimators under Composite LINEX Loss Function to Estimate the Shape Parameter in Lomax DistributionComputational Intelligence and Neuroscience10.1155/2022/12006112022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/1200611
Choi MOh SChoi JKim JLee S(2022)A statistical learning framework for predicting left ventricular ejection fraction based on glutathione peroxidase-3 level in ischemic heart diseaseComputers in Biology and Medicine10.1016/j.compbiomed.2022.105929149:COnline publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1016/j.compbiomed.2022.105929
Khan AAmanullah MAmin MAlharbi RMuse AMohamed M(2021)Influence Diagnostic Methods in the Poisson Regression Model with the Liu EstimatorComputational Intelligence and Neuroscience10.1155/2021/44073282021Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1155/2021/4407328
Özkale MArıcan E(2019)A first-order approximated jackknifed ridge estimator in binary logistic regressionComputational Statistics10.1007/s00180-018-0851-634:2(683-712)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s00180-018-0851-6

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents