RETHINKING THE ART AND SCIENCE OF RISK ADJUSTMENT

Risk Adjustment for Measuring Health Care Outcomes attention on estimating and discussing the quality of
(3rd edition), edited by Lisa I. Iezzoni. Health health care. This focus is derived in part from concerns
Administration Press, Chicago, IL, 2003, 508 pp., that the incentives of funding systems, such as pro-
$85.00 (cloth). spective payments, result in cost/quality trade-offs that
have a negative impact on the health of beneciaries
In recent years many academics, health care (Poland, Bollinger, Bedard, & Cohen, 1985; Thomas et
providers, and third-party payers have focused their al., 1986).

Vol. 44, No. 3, 2004 439

Downloaded from https://academic.oup.com/gerontologist/article-abstract/44/3/439/699496
by guest
on 04 December 2017
 Figure 1. The causes of health outcomes.

A high proportion of the research and literature validity and reliability of risk adjustment strategies,
investigating these issues has centered on older pop- general linear and logistic regression, propensity scores,
ulations, either explicitly by analyzing administrative instrumental variables, and hierarchical modeling.
databases such as those of the U.S. Medicare system Perhaps the most signicant change in the third edition,
(Schneeweiss, Wang, Avorn, & Glynn, 2003; Yuan, however, is the expansion in the scope of the discussion
Cooper, Einstadter, Cebul, & Rimm, 2000) or implicitly and examples provided. In addition to in-patient
by focusing on treatments that are more common hospital stays, other health care settings are now
in older patients (Hannan et al., 2003; Luft, 2003; incorporated. Notable among the additions are long-
Rosenberg, 2002). The frequent use of age as a proxy for
term care examples, including discussions of home care
other factors in such analyses seriously biases the
studies conclusions. and nursing homesboth of which are given signicant
A key issue in estimating the quality of health care, of attention within gerontological quality-of-care research.
course, is the quality of the analysisespecially how the Sections have also been added to identify specic issues
researchers attempted to isolate the adverse consequen- in risk adjustment associated with mental health and
ces directly attributable to the quality of care provided. disability.
In medicine, these efforts are often referred to as risk
adjustment (Inouye et al., 2003, Johnson, 2003; Render
et al., 2003); other common terms include outcomes
research (Dimick, Cowan, Upchurch, & Colletti, 2003; Understanding the Concept of Risk Adjustment
Silvet et al., 2003; Weir, Signorini, Dennis, & Murdoch, In an illustration of the basic problem of outcomes
2000) and quality-of-care research (Mor, Angelelli, research, Iezzoni provides a conceptual model of the
Gifford, Morris, & Moore, 2003; Reuben, Shekelle, &
Wenger, 2003; Scott et al., 2003). The ultimate aim of summation of patient factors, treatment effects, and
this body of research is to determine a statistical method random events that produce health outcomes. This
that can clearly distinguish between adverse events that conceptual model is referred to as the Algebra of
are attributable to a specic treatment choice, provider, Effectiveness, although most risk adjustment methods
or delivery venue and those that are due to intrinsic involve much more complicated mathematical and/or
characteristics of the patient. This is a very difcult goal statistical approaches. Figure 1 illustrates the basic
to achieve because of the complexities of human health, problem in risk adjustment and has a similar structure
the impact of random events, and the unavoidable to the conceptual model presented by Iezzoni. In Figure
resource constraints and limitations in current informa- 1, however, four basic sources of health outcomes are
tion systems that often lead to less than optimal data identied: patient characteristics (both clinical and
being utilized. nonclinical), treatment characteristics, organization
The recently published third edition of Risk characteristics (including the facility and providers),
Adjustment for Measuring Health Care Outcomes, and random events. Like the Iezzoni model, Figure 1
edited by Lisa Iezzoni, provides an excellent overview assumes an additive relationship between the causes of
of these and other issues related to risk adjustment and health outcomes, although the true (and causal)
quality-of-care research. The previous versions of this relationships may be more complicated (i.e., involving
book have been widely accepted as the quintessential interactions between the four components and possible
text for those who are interested in outcomes research,
risk adjustment, or quality of care. As is stated in the reverse causationsfor example health outcomes
preface, the aim of the third edition is to introduce the affecting patient and organizational characteristics).
issues underlying risk adjustment and to suggest Instead of including both treatment characteristics
important conceptual and methodological considera- and organization characteristics in the Algebra of
tions in designing and evaluating risk-adjustment Effectiveness model, Iezzoni uses the term treatment
strategies (p. xvii). Although the authors have updated effectiveness. By doing so, she does not explicitly
their focus on methodological techniques in this include quality of care in the model, although
edition, the book still is better described as a justica- assessment of quality of care often drives the de-
tion for why one should perform risk adjustment and velopment of risk adjustment methodology. Hypothet-
some of the processes involved in doing so, rather than ically, one may consider variation in treatment
a manual for how to perform the often quite difcult effectiveness across providers or groups of providers,
statistical techniques involved in risk adjustment. all other model inputs being equal, as a source of
quality-of-care indicators. The distinction should be
New Elements in the Third Edition made that the aim of such assessments are directed at
While much of the text is unchanged, Iezzoni now measuring the effect of organization characteristics on
includes sections on conducting surveys, measuring the health outcomes and not organizational quality, per se.

440 The Gerontologist

Downloaded from https://academic.oup.com/gerontologist/article-abstract/44/3/439/699496
by guest
on 04 December 2017
 The Interaction and Correlation of Variables lines may be developed based on patient age, when
issues of social support, physical functioning, cognitive
The patient factors described in Iezzonis model status, and patient attitudes and preferences are more
include a holistic description of well-being, including efcient predictors of health outcomes. Unfortunately,
physical functional status and psychological, cognitive, however, age is more easily measured and quantied
and psychosocial function. Many of these factors have than these more amorphous and less stable factors.
either strong correlations with each other or may These observations lead to our next consideration,
impact the health of the individual in a multiplicative specically the inuence of patient factors on treatment
fashion (Fang, Liu, Tang, Wang, & Ko, 2004; choice (and compliance), with treatment choice an
Kalantar-Zadeh, Kleiner, Dunne, Lee, & Luft, 1999). important moderator of treatment effectiveness and
The model and methods presented in the book, and health outcomes. Differences in rates of interventions
that are most commonly used, focus on the individual among different age cohorts are typically explained by
patient factors and not the correlations and non-
costbenet arguments due to expected remaining
linearities that are known to exist. This oversight is not
lifetime and/or inability to tolerate aggressive treat-
due to ignorance of these interactions and effects but,
ment due to frailty (Cameron & Williams, 1996;
as Iezzoni observes (pp. 179180), from the trade-offs
made in risk adjustment between content validity for Silverman, McDowell, Musa, Rodriguez, & Martin,
a specic intervention/population and the desire to 1997; van der Steen, Ooms, Ader, Ribbe, & van der
compare results across interventions/populations by Wal, 2002). These arguments may be made by the
using compatible methodologies and to minimize the clinician, the patient, or both when choosing among
resources devoted to the development of new risk intervention options (Baig et al., 2003). An extension of
adjustment strategies. The issue of patient factor our previous argument can then be made. Specically,
interactions is of specic importance in aging popula- that the use of age-based clinical guidelines based on
tions. Almost all risk adjustment methods include age outcomes research that used chronological age as
as a factor to serve as a proxy for the various changes in a proxy for age-related changes in health status, results
health status associated with aging that are now in a further bias against healthier seniors. These
explicitly included in Iezzonis model. healthier individuals may not even receive the oppor-
tunity to respond to a treatment associated with the
greatest probability of positive outcome.
Bias Effects of Using Age as a Proxy
Consider that we are looking at some important
outcome for patient i treated at institution j, denoted Is Age Really Important?
Dij, where a simple risk adjustment equation uses the Perhaps we, as a society and as health researchers,
patients age (Ai) and the volume of the patients treated have focused too closely on chronological age and have
at that institution (Vj) ignored not only the greater picture of current well-
being, but also the actual process or time-dependencies
Dij a bAi dVj eij : of aging. In addition to individual-specic variations of
risk over time, various birth cohorts of individuals live
This simple model demonstrates many of the through different historical periods and through some
potential problems with risk adjustment. First, although of the same periods, but at different ages. The mixture
age is an important variable in most risk adjustment of these factors can inuence their current or future
models, it is also correlated with many other health and health. A 70-year-old individual born during the 1930s
institutional variables that may not be included. Thus, if may not have the same health foundation as a 70-year-
we simply estimate the coefcient for age it will be old individual born during the periods of relative
biased because age will reect the signicance of these afuence in the decades immediately preceding or
other missing variables (i.e., missing variable bias). The following the Depression. In risk adjustment, it is
same can be said for volume. Unless you account for all necessary to consider global temporal changes in health
relevant organizational variables, you will inevitably and medicine and their inuence on cohorts of
have biased parameter estimates. Further, the quality/ individuals, as well as individual uctuations in risk
volume relationship may also suffer from endogeneity. prole.
Specically, if the quality of an institution is correlated The lack of a solid theoretical consideration for the
over time, then patients may be attracted to facilities intricacies and complexities of risk adjustment raises
with better quality. Thus, the presence of quality may much more important questions: Who is ensuring that
in fact drive volume, rather than institutions with the practitioners of risk adjustment are doing it
higher volumes resulting in provision of higher quality correctly? Are we encouraging uniformity of method
health care. over construct validity? While these may sound like
If all relevant factors are not included in a risk technical questions that only academic researchers may
adjustment model, the unexplained variance will be interested in, the quality of risk adjustment is
weight correlated factors that are included. In these a signicant issue with ramications for all participants
circumstances, the use of age as a proxy for age-related in a health care system, from consumers to providers to
changes in health status results in a bias against payers. For example, a poor quality scorecard can
healthier older persons. For instance, treatment guide- cause a physician or hospital to lose clientele or funding

Vol. 44, No. 3, 2004 441

Downloaded from https://academic.oup.com/gerontologist/article-abstract/44/3/439/699496
by guest
on 04 December 2017
 or may even affect the types of care that they are trade-offs that lead to overall lower welfare in the
willing to provide. community (Graff Zivin & Bridges, 2002). In cost
effectiveness analysis, an outcomes-based analysis, the
Extra-Welfarist perspective is employed when only
The Art and Science of Risk Adjustment health outcomes are considered, and the Welfarist
perspective considers both health and nonhealth out-
Given the complexity of risk adjustment, we comes (Birch & Donaldson, 2003). What is missing in
applaud Iezzonis volume for its thoughtful and quality-of-care research is the more holistic perspective
thorough discussion of the intricacies and complexities for patients preferencesfor health, nonhealth and
of risk adjustment and of the increasing importance process consequences of health care. This type of
placed on outcome and quality-of-care research. As approach to measurement is especially relevant to older
a result of this increasing attention, risk adjustment populations who may place a higher value on the
methods are rapidly changing and knowledge is process of successful aging, through maintenance of
increasing exponentially. Perhaps the most important various health and nonhealth attributes of life quality,
aspect of Iezzonis third edition is her continued as compared to indiscriminant increases in longevity
restraint from claiming to provide the pathway to risk (Drewnowski & Evans, 2001). We have argued that we
adjustments holy graila method that perfectly must move beyond age as a proxy for aging in risk
describes the probability of a health outcome from an adjustment. Similarly, we must expand our investiga-
intervention, holding all other factors constant. tions beyond mortality as the sole outcome of interest.
This of course leaves risk adjustment as part art and
part science. There is much skill needed to set up the Darcey D. Terris, MBA, AHRQ Predoctoral Fellow
problem (for which the Iezzoni text is a great resource) and
and much more skill in devising an appropriate John F. P. Bridges, PhD, Assistant Professor
statistical model. It is on this front where one needs Department of Epidemiology and Biostatistics
to look beyond the Iezzoni text. This is not surprising Health Services Research Division
though, given that to perform unbiased and/or efcient School of Medicine
risk adjustment, the most recent advances in statistics, Case Western Reserve University
biostatistics, and econometrics are needed. This is Cleveland, Ohio 44106
because of the complexity of the problem of isolating References
the marginal effects that the four components ex- Baig, N., Myers, R. E., Turner, B. J., Grana, J., Rothermel, T., Schlackman,
pressed in Figure 1 (patient characteristics, treatment N., et al. (2003). Physician-reported reasons for limited follow-up of
characteristics, organization characteristics, and ran- patients with a positive fecal occult blood test screening result. The
dom events) have on the health outcomes of patients. American Journal of Gastroenterology, 98, 20782081.
Birch, S., & Donaldson, C. (2003). Valuing the benets and costs of health
care programmes: Wheres the extra in extra welfarism? Social Science
and Medicine, 56, 11211133.
Cameron, H. J., & Williams, B. O. (1996). Clinical trials in the elderly.
A Final Word Should we do more? Drugs & Aging, 9, 307310.
Dimick, J. B., Cowan, J. A., Jr., Upchurch, G. R., Jr., & Colletti, L. M.
When assessing the impact of organizational char- (2003). Hospital volume and surgical outcomes for elderly patients with
acteristics on health outcomes, even if we could derive colorectal cancer in the United States. The Journal of Surgical Research,
the perfect risk adjustment methodology and isolate the 114, 5056.
Drewnowski, A., & Evans, W. J. (2001). Nutrition, physical activity, and
true quality effects, we may not arrive at the optimal quality of life in older adults: Summary. The Journals of Gerontology:
solution to improving health delivery. First, as stated in Biological and Medical Sciences, 56A(Suppl. 2), 8994.
the beginning of this essay, the rationale behind risk Fang, F.-M., Liu, Y.-T., Tang, Y., Wang, C.-J., & Ko, S.-F. (2004). Quality of
adjustment and quality-of-care research is often life as a survival predictor for patients with advanced head and neck
carcinoma treated with radiotherapy. Cancer, 100, 425432.
estimation of the trade-offs required in balancing Graff Zivin, J., & Bridges, J. (2002). Addressing risk preferences in cost-
quality demands in the face of resource constraints. effectiveness analyses. Applied Health Economics and Health Policy, 1,
As yet, there has been very little literature specically 135139.
Hannan, E. L., Wu, C., Ryan, T. J., Bennett, E., Culliford, A. T., Gold, J. P.,
focused on this relationship, that is, when is quality et al. (2003). Do hospitals and surgeons with higher coronary artery
improvement worth the additional costs. While some bypass graft surgery volumes still have lower risk-adjusted mortality
have looked at the cost-quality relationship in aggre- rates? Circulation, 108, 795801.
Inouye, S. K., Bogardus, S. T., Jr., Vitagliano, G., Desai, M. M., Williams,
gated data, their use of data on average cost and C. S., Grady, J. N., et al. (2003). Burden of illness score for elderly
average quality does not focus on the true marginal persons: Risk adjustment incorporating the cumulative impact of
costs of attempting to achieve better quality. Here we diseases, physiologic abnormalities, and functional impairments. Medical
might be better served to develop a hybrid between Care, 41, 7083.
Johnson, M. L. (2003). Risk assessment and adjustment: Adjusting for sick
quality analysis and cost effectiveness analysis (which is patients or a sick system. Medical Care, 41, 47.
normally used on experimental data) to focus on the Kalantar-Zadeh, K., Kleiner, M., Dunne, E., Lee, G. H., & Luft, F. C. (1999).
true costquality trade-off. A modied quantitative subjective global assessment of nutrition for
dialysis patients. Nephrology Dialysis Transplantation, 14, 17321738.
More importantly, perhaps, by making judgments Luft, H. S. (2003). Variations in patterns of care and outcomes after acute
solely on risk-adjusted health outcomes alone, we myocardial infarction for Medicare beneciaries in fee-for-service and
may be ignoring potentially signicant nonhealth or HMO settings. Health Services Research, 38, 10651079.
nonoutcomes (i.e., the process of care) factors, leading Mor, V., Angelelli, J., Gifford, D., Morris, J., & Moore, T. (2003).
Benchmarking and quality in residential and nursing homes: Lessons
to suboptimal resource allocation decisions. By focus- from the U.S. International Journal of Geriatric Psychiatry, 18, 258266.
ing purely on health outcomes, we may be making Poland, R. L., Bollinger, R. O., Bedard, M. P., & Cohen, S. N. (1985).

442 The Gerontologist

Downloaded from https://academic.oup.com/gerontologist/article-abstract/44/3/439/699496
by guest
on 04 December 2017
 Analysis of the effects of applying federal diagnosis-related grouping Silvet, H., Spencer, F., Yarzebski, J., Lessard, D., Gore, J. M., & Goldberg,
(DRG) guidelines to a population of high-risk newborn infants. R. J. (2003). Community wide trends in the use and outcomes associated
Pediatrics, 76, 104109. with beta-blockers in patients with acute myocardial infarction: The
Render, M. L., Kim, H. M., Welsh, D. E., Timmons, S., Johnston, J., Hui, S., Worchester Heart Attack Study. Archives of Internal Medicine, 163,
et al. (2003). Automated intensive care unit risk adjustment: Results from 21752183.
a national Veterans Affairs study. Critical Care Medicine, 31, 16381646. Thomas, F., Larsen, K., Clemmer, T. P., Burke, J. P., Orme, J. F. Jr., Napoli,
Reuben, D. B., Shekelle, P. G., & Wenger, N. S. (2003). Quality of care for M., et al. (1986). Impact of prospective payments on a tertiary care center
older persons at the dawn of the third millennium. Journal of the receiving large numbers of critically ill patients by aeromedical transport.
American Geriatrics Society, 51, S346S350. Critical Care Medicine, 14, 227230.
Rosenberg, A. L. (2002). Recent innovations in intensive care unit risk- van der Steen, J. T., Ooms, M. E., Ader, H. J., Ribbe, M. W., & van der Wal,
prediction models. Current Opinion in Critical Care, 8, 321330. G. (2002). Withholding antibiotic treatment in pneumonia patients with
Schneeweiss, S., Wang, P. S., Avorn, J., Glynn R. J. (2003). Improved dementia: A quantitative observational study. Archives of Internal
comorbidity adjustment for predicting mortality in Medicare popula- Medicine, 162, 17531760.
tions. Health Services Research, 38, 11031120. Weir, N. U., Signorini, D. F., Dennis, M. S., & Murdoch, P. S. (2000). Trying
Scott, I. A., Denaro, C. P., Flores, J. L., Bennett, C. J., Hickey, A. C., Mudge, to understand routine stroke outcome data: The need for adequate
A. M., et al. (2003). Quality of care of patients hospitalized with casemix adjustment and some practical considerations. Health Bulletin,
congestive heart failure. Internal Medicine Journal, 33, 140151. 58, 301308.
Silverman, M., McDowell, B. J., Musa, D., Rodriguez, E., & Martin, D. Yuan, Z., Cooper, G. S., Einstadter, D., Cebul, R. D. & Rimm, A. A. (2000).
(1997). To treat or not to treat: Issues in decisions not to treat older The association between hospital type and mortality and length of stay: A
persons with cognitive impairment, depression, and incontinence. Journal study of 16.9 million hospitalized Medicare beneciaries. Medical Care,
of the American Geriatrics Society, 45, 10941101. 38, 231245.

Vol. 44, No. 3, 2004 443

Downloaded from https://academic.oup.com/gerontologist/article-abstract/44/3/439/699496
by guest
on 04 December 2017