Nothing Special   »   [go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/wly/hlthec/v24y2015i1p75-85.html
   My bibliography  Save this article

Measuring Overfitting In Nonlinear Models: A New Method And An Application To Health Expenditures

Author

Listed:
  • Marcel Bilger
  • Willard G. Manning
Abstract
When fitting an econometric model, it is well known that we pick up part of the idiosyncratic characteristics of the data along with the systematic relationship between dependent and explanatory variables. This phenomenon is known as overfitting and generally occurs when a model is excessively complex relative to the amount of data available. Overfitting is a major threat to regression analysis in terms of both inference and prediction. We start by showing that the Copas measure becomes confounded by shrinkage or expansion arising from in‐sample bias when applied to the untransformed scale of nonlinear models, which is typically the scale of interest when assessing behaviors or analyzing policies. We then propose a new measure of overfitting that is both expressed on the scale of interest and immune to this problem. We also show how to measure the respective contributions of in‐sample bias and overfitting to the overall predictive bias when applying an estimated model to new data. We finally illustrate the properties of our new measure through both a simulation study and a real‐data illustration based on inpatient healthcare expenditure data, which shows that the distinctions can be important. Copyright © 2013 John Wiley & Sons, Ltd.

Suggested Citation

  • Marcel Bilger & Willard G. Manning, 2015. "Measuring Overfitting In Nonlinear Models: A New Method And An Application To Health Expenditures," Health Economics, John Wiley & Sons, Ltd., vol. 24(1), pages 75-85, January.
    • Handle: RePEc:wly:hlthec:v:24:y:2015:i:1:p:75-85
    DOI: 10.1002/hec.3003
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/hec.3003
    Download Restriction: no
    File URL: https://libkey.io/10.1002/hec.3003?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Mullahy, John, 1998. "Much ado about two: reconsidering retransformation and the two-part model in health econometrics," Journal of Health Economics, Elsevier, vol. 17(3), pages 247-281, June.
    2. John Mullahy, 1998. "Much Ado About Two: Reconsidering Retransformation and the Two-Part Model in Health Economics," NBER Technical Working Papers 0228, National Bureau of Economic Research, Inc.
    3. Manning, Willard G. & Basu, Anirban & Mullahy, John, 2005. "Generalized modeling approaches to risk adjustment of skewed outcomes data," Journal of Health Economics, Elsevier, vol. 24(3), pages 465-488, May.
    4. Manning, Willard G., 1998. "The logged dependent variable, heteroscedasticity, and the retransformation problem," Journal of Health Economics, Elsevier, vol. 17(3), pages 283-295, June.
    5. Steven C. Hill & G. Edward Miller, 2010. "Health expenditure estimation and functional form: applications of the generalized gamma and extended estimating equations models," Health Economics, John Wiley & Sons, Ltd., vol. 19(5), pages 608-627, May.
    6. Anirban Basu & Bhakti V. Arondekar & Paul J. Rathouz, 2006. "Scale of interest versus scale of estimation: comparing alternative estimators for the incremental costs of a comorbidity," Health Economics, John Wiley & Sons, Ltd., vol. 15(10), pages 1091-1107, October.
    7. Blough, David K. & Madden, Carolyn W. & Hornbrook, Mark C., 1999. "Modeling risk using generalized linear models," Journal of Health Economics, Elsevier, vol. 18(2), pages 153-171, April.
    8. Manning, Willard G. & Mullahy, John, 2001. "Estimating log models: to transform or not to transform?," Journal of Health Economics, Elsevier, vol. 20(4), pages 461-494, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Basco, Rodrigo & Hair, Joseph F. & Ringle, Christian M. & Sarstedt, Marko, 2022. "Advancing family business research through modeling nonlinear relationships: Comparing PLS-SEM and multiple regression," Journal of Family Business Strategy, Elsevier, vol. 13(3).
    2. John Yfantopoulos & Athanasios Chantzaras, 2020. "Health-related quality of life and health utilities in insulin-treated type 2 diabetes: the impact of related comorbidities/complications," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 21(5), pages 729-743, July.
    3. Chakrabarty, Himadri Shekhar & Roy, Rudra Prosad, 2021. "Pandemic uncertainties and fiscal procyclicality: A dynamic non-linear approach," International Review of Economics & Finance, Elsevier, vol. 72(C), pages 664-671.
    4. John Mullahy, 2015. "In Memoriam: Willard G. Manning, 1946‐2014," Health Economics, John Wiley & Sons, Ltd., vol. 24(3), pages 253-257, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jones, A.M, 2010. "Models For Health Care," Health, Econometrics and Data Group (HEDG) Working Papers 10/01, HEDG, c/o Department of Economics, University of York.
    2. Borislava Mihaylova & Andrew Briggs & Anthony O'Hagan & Simon G. Thompson, 2011. "Review of statistical methods for analysing healthcare resources and costs," Health Economics, John Wiley & Sons, Ltd., vol. 20(8), pages 897-916, August.
    3. Steven C. Hill & G. Edward Miller, 2010. "Health expenditure estimation and functional form: applications of the generalized gamma and extended estimating equations models," Health Economics, John Wiley & Sons, Ltd., vol. 19(5), pages 608-627, May.
    4. Keane, Michael & Stavrunova, Olena, 2016. "Adverse selection, moral hazard and the demand for Medigap insurance," Journal of Econometrics, Elsevier, vol. 190(1), pages 62-78.
    5. Manning, Willard G. & Basu, Anirban & Mullahy, John, 2005. "Generalized modeling approaches to risk adjustment of skewed outcomes data," Journal of Health Economics, Elsevier, vol. 24(3), pages 465-488, May.
    6. Liu, Lei & Conaway, Mark R. & Knaus, William A. & Bergin, James D., 2008. "A random effects four-part model, with application to correlated medical costs," Computational Statistics & Data Analysis, Elsevier, vol. 52(9), pages 4458-4473, May.
    7. Kathleen Carey & Theodore Stefos, 2011. "Measuring the cost of hospital adverse patient safety events," Health Economics, John Wiley & Sons, Ltd., vol. 20(12), pages 1417-1430, December.
    8. Toni Mora & Joan Gil & Antoni Sicras-Mainar, 2015. "The influence of obesity and overweight on medical costs: a panel data perspective," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 16(2), pages 161-173, March.
    9. Brilleman, Samuel L. & Gravelle, Hugh & Hollinghurst, Sandra & Purdy, Sarah & Salisbury, Chris & Windmeijer, Frank, 2014. "Keep it simple? Predicting primary health care costs with clinical morbidity measures," Journal of Health Economics, Elsevier, vol. 35(C), pages 109-122.
    10. Samuel L Brilleman & Hugh Gravelle & Sandra Hollinghurst & Sarah Purdy & Chris Salisbury & Frank Windmeijer, 2011. "Keep it Simple? Predicting Primary Health Care Costs with Measures of Morbidity and Multimorbidity," Working Papers 072cherp, Centre for Health Economics, University of York.
    11. Keane, Michael & Stavrunova, Olena, 2016. "Adverse selection, moral hazard and the demand for Medigap insurance," Journal of Econometrics, Elsevier, vol. 190(1), pages 62-78.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:hlthec:v:24:y:2015:i:1:p:75-85. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/5749 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.