Nothing Special   »   [go: up one dir, main page]
Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Extensions to Regret-based Decision Curve Analysis: An application to hospice referral for terminal patients

BMC Medical Informatics and Decision Making volume 11, Article number: 77 (2011) Cite this article

Abstract

Background

Despite the well documented advantages of hospice care, most terminally ill patients do not reap the maximum benefit from hospice services, with the majority of them receiving hospice care either prematurely or delayed. Decision systems to improve the hospice referral process are sorely needed.

Methods

We present a novel theoretical framework that is based on well-established methodologies of prognostication and decision analysis to assist with the hospice referral process for terminally ill patients. We linked the SUPPORT statistical model, widely regarded as one of the most accurate models for prognostication of terminally ill patients, with the recently developed regret based decision curve analysis (regret DCA). We extend the regret DCA methodology to consider harms associated with the prognostication test as well as harms and effects of the management strategies. In order to enable patients and physicians in making these complex decisions in real-time, we developed an easily accessible web-based decision support system available at the point of care.

Results

The web-based decision support system facilitates the hospice referral process in three steps. First, the patient or surrogate is interviewed to elicit his/her personal preferences regarding the continuation of life-sustaining treatment vs. palliative care. Then, regret DCA is employed to identify the best strategy for the particular patient in terms of threshold probability at which he/she is indifferent between continuation of treatment and of hospice referral. Finally, if necessary, the probabilities of survival and death for the particular patient are computed based on the SUPPORT prognostication model and contrasted with the patient's threshold probability. The web-based design of the CDSS enables patients, physicians, and family members to participate in the decision process from anywhere internet access is available.

Conclusions

We present a theoretical framework to facilitate the hospice referral process. Further rigorous clinical evaluation including testing in a prospective randomized controlled trial is required and planned.

Peer Review reports

Background

Introduction

Hospice services have been proven to provide better quality of care to dying patients[1–3] by optimizing pain relief [4, 5] and reducing emotional stress [1, 6, 7]. Furthermore, hospice care is associated with greater patient-family satisfaction[8], is shown to be cost effective[9, 10], and most importantly, it has been attributed with increased survival in some patients [11]. Despite these well documented advantages, many terminally ill patients do not reap maximum benefits from hospice care. The fundamental reason for this is related to the less than optimal and frequently poorly timed referral of terminally ill patients to hospice [1, 12]. As a result, many patients die within a few days of referral, or live many years after the referral was made [13].

According to Medicare regulations, a person should be referred to hospice if his/her "life expectancy (LE) is 6 months or less" [1, 14]. Hence, the problem of meaningful referrals relates to the accurate estimation (prognosis) of death within approximately 6 months after evaluation for hospice care. However, statistical models designed to assist physicians in predicting life expectancy (LE), although beneficial [15, 16], so far they failed to improve the quality of care at the end of life [17–21].

One such statistical model is SUPPORT (Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments), designed to calculate the probability of survival over a period of 180 days [22, 23]. Although the SUPPORT model has been well validated [17, 22] for prognostication of LE in terminally ill patients, a controlled trial of SUPPORT failed to demonstrate any impact on the overall quality of care for these patients [17, 20]. We postulate that this lack of impact may be due to the fact that SUPPORT results, were not linked to any decision methodology that would translate the probability of survival to a hospice referral recommendation. Therefore, the full potential of the model's prognostication power remained unexploited.

In this work, we link the SUPPORT prognostication model with the recently developed decision methodology regret DCA[24] to facilitate the hospice referral process. Regret DCA relies on regret theory and decision curve analysis [25] to recommend the optimal management strategy for a patient, accounting for the personal attitudes and values of the particular patient or his/her surrogate.

Furthermore, we extend regret DCA to incorporate harms and effects of treatment as well as harms associated with the prognostication test to the decision model. The presented methodology is integrated into a comprehensive clinical decision support system developed to facilitate the hospice referral process.

Methods

Dataset

In our analysis, we utilized the entire SUPPORT dataset, both development and validation cohorts. The dataset is presented in detail elsewhere [22]. Medical records of 8,329 seriously ill hospitalized adults are included.

Support model

SUPPORT is a multivariable model designed to estimate probability of survival for seriously ill hospitalized patients over a period of the subsequent 180 days. The model variables include the patient's medical condition compatible with one of eight major diagnostic groupings (Acute Respiratory Failure, Multiple Organ System Failure, Chronic Obstructive Pulmonary Disease, Congestive Heart Failure, Hepatic Cirrhosis, Neurological Coma, Lung or Colon Cancer), the patient's current age, number of days in the hospital before study entry, neurologic status, and 11 physiologic measures recorded on day 3 after study entry [22].

The SUPPORT implementation for the estimation of survival probability is detailed in the appendix. Due to the nature of the hospice referral problem we also express the survival probability in terms of mortality. We can convert the estimated survival probability (SP) (equation A2) to probability of death within 180 days (denoted here as p) using the equation:

p = 1 - S P = 1 - P { T ≥ t | disease group = i }
(1)

where SP is the survival probability computed by SUPPORT, i ∈ [1, 8] the patient's disease group, T is the survival time in days, and t is an arbitrary time (typically expressed in days e.g. t ∈ [1,180]).

In terms of accuracy, the SUPPORT model has an area under the receiver-operating characteristics curve (ROC) for prediction of surviving 180 days of 0.79 in the phase I development cohort and 0.78 in the phase II validation cohort [22].

Decision model

Figure 1 depicts the decision tree summarizing the process of hospice referral. The four outcomes and their corresponding utilities (U) shown are:

Figure 1
figure 1

Decision tree for hospice referral. In this figure, p is the probability that a patient's LE is less than or equal to 6 months; 1-p is the probability that a patient's LE is greater than 6 months; Ui: are the utilities associated with each outcome; Rgi is the regret associated with each outcome.

Full size image
  1. 1.

    U 1: Refer the patient to hospice and the patient's LE is less than or equal to 6 months (Hosp|LE ≤ 6).

  2. 2.

    U 2: Refer the patient to hospice and the patient's LE is greater than 6 months (Hosp|LE > 6).

  3. 3.

    U 3: Continue treating the patient and the patient's LE is less than or equal to 6 months (Rx|LE ≤ 6).

  4. 4.

    U 4: Continue treating the patient and the patient's LE is greater than 6 months (Rx|LE > 6).

p is the probability associated with the presence of an event (e.g. patient's LE ≤ 6 months) as predicted by the SUPPORT model, 1 - p is the probability associated with the absence of the same event (e.g. patient's LE > 6 months).

As with any decision, one may come to realize that, in retrospect, an alternative decision would have been preferable. This knowledge may bring a sense of loss or regret [26–32]. In this paper, we use this sense of regret to determine the preferences of the decision maker towards alternative management strategies. Specifically, we employ regret theory to estimate the threshold probability, P t , at which the decision maker (patient, physician, or family member) is indifferent between continuation of treatment vs. hospice referral. Based on the concept of threshold probability, the patient should be referred to hospice if his/her probability of death is greater than or equal to P t (e.g. p ≥ P t ), and he/she should continue receiving curative treatment otherwise (p < P t ).

The threshold probability is derived as [24]:

P t = 1 1 + U 1 - U 3 U 4 - U 2
(2)

In (2) U 1 -U 3 is associated with regret of omission (e.g. the patient was not referred to hospice, instead he/she continued receiving unnecessary treatment) and U 4 -U 2 with regret of commission (e.g. the patient was unnecessary referred to hospice instead of continue receiving life-sustaining treatment) [24].

To elicit the decision maker's regret, and therefore threshold probability, we utilize the DVAS (Dual Visual Analogue Scale) method [24]. One visual analogue scale is used to capture the regret associated with failing to refer the patient to hospice (e.g. continue unnecessary treatment) and the second scale to measure the regret associated with unnecessary hospice referral (e.g. failing to provide life-sustaining treatment) (Figure 2).

Figure 2
figure 2

DVAS (Dual Visual Analogue Scales). The DVAS are used for the elicitation of the decision maker's threshold probability.

Full size image

Elicitation of threshold probability can be achieved through a set of questions such as:

  1. 1.

    On the scale 0 to 100, where 0 indicates no regret and 100 indicates the maximum regret you could feel, how would you weigh the level of your regret if you were not referred to hospice but instead you continued receiving unnecessary treatment? That is, how much would you regret if you did not reap the benefits of hospice care? *Note that this value corresponds to U1 - U3.

  2. 2.

    On the scale 0 to 100, where 0 indicates no regret and 100 indicates the maximum regret you could feel, how would you weigh the level of your regret, if you were referred to hospice instead of continue receiving necessary life-sustaining treatment? That is, how much would you regret if you sustained harms from hospice care? *Note that this value corresponds to U4 - U2.

For example, suppose that the patient - who is aware of his/her terminal condition- answers 50 and 25 to the questions 1 and 2 respectively. This means that the patient considers 50/25 = 2 times worse not to be referred to hospice when necessary than receiving an unnecessary hospice referral. The threshold probability for this patient is (equation 2)

p t = 1 1 + U 1 - U 3 U 4 - U 2 = 1 1 + 5 0 2 5 = 0 . 3 3 or 33%  .

Regret DCA and extensions

The clinical problem we face in the situation of hospice referral is how to use reasonably accurate predictions of death, p, coupled with the patient's preferences (as expressed in terms of threshold probability, P t ) to arrive at the optimal decision for a specific individual. The problem is decomposed into three strategies: (1) act based on the prediction model (SUPPORT) (e.g. refer to hospice if p ≥ P t and continue treating otherwise), (2) refer all patients to hospice, and (3) continue current treatment for all patients (i.e. refer no patients to hospice).

Each of these strategies may inflict physiological and/or psychological damages to the patient. Specifically, a patient may suffer harms due to a treatment strategy (e.g. adverse effects) or harms due to the prognostication test (e.g. a test requiring invasive procedure). We express these harms as loss in utility associated with actions we may undertake. To that end, we define H Rx , H Hosp and H te as the utility losses due to harms of the treatment, hospice, and prognostic test, respectively.

Figure 3 presents the decision tree describing the overall hospice referral problem. p = P(D +) is the probability that the patient's LE is less than or equal to 6 months as estimated by the prediction model (SUPPORT);1 - p = P(D -) is the probability that the patient's LE is greater than 6 months, and U i , i ∈ [1, 4], are the utilities corresponding to each of the decision model outcomes (detailed in the previous section). The variables Hosp and Rx correspond to referring a patient to hospice and continuing current curative treatment, respectively. Rg, is the regret associated with an action, e.g. Rg(Hosp, D-) is the regret one may feel if the patient was referred to hospice when his/her LE was greater than 6 months. Finally, te designates that the patient received a prognostication test.

Figure 3
figure 3

Decision tree describing the overall hospice referral process. In this figure p = P (D +): probability the patient's LE is less than or equal to 6 months; 1 - p = P (D -): probability the patient's LE is greater than 6 months; Ui, i ∈ [1, 4]: the utilities corresponding to each of the decision model outcomes; Hosp: hospice referral; Rx treatment continuation; Rg: regret associated with an action; HRx : utility losses due to harms of treatment; HHosp: utility losses due to harms of hospice; Hte: utility losses due to harms of the prognostic test (SUPPORT).

Full size image

Considering the decision tree in Figure 3 we can compute the expected regret associated with each decision in terms of the utilities of each possible outcome as follows (detailed derivation is presented in the Appendix):

E R g [ H o s p ] = ( 1 - p ) * ( 1 - R R R H o s p ) * P t 1 - P t
(3)
E R g [ R x ] = p * ( 1 - R R R R x )
(4)
E R g [ S U P P O R T ] = 1 - R R R H o s p * ( # T P ∕ n + # F P ∕ n ) - R R R R x * ( # F N ∕ n + # T N ∕ n ) * H t e U 1 - U 3 + H R x - H H o s p + ( 1 - R R R H o s p ) * # F P n * P t 1 - P t + ( 1 - R R R R x ) * # F N n
(5)

In addition to harms, equations 3, 4 and 5 incorporate the effects of treatment and hospice care using measures of Relative Risk Reduction: and RRR Rx RRR Hosp respectively. The values for these measures are treatment specific and can be acquired from the literature. We have incorporated hospice effects because a recent study [11] has shown that early palliative care for patients with metastatic non-small cell lung cancer could increase survival. The variables TP, FP, FN, TN are related to the prognostic capability of the SUPPORT model (see appendix for detailed derivation) [24].

Since the regret of omission and regret of commission have been generalized to include effects and harms related to management strategies and testing, the function of threshold probability (equation 2) becomes:

P t = 1 1 + U 1 - U 3 + H R x - H H o s p U 4 - U 2 - H R x - H H o s p
(6)

Where U 1 - U 3 + H Rx -H Hosp corresponds to the regret associated with not referring the patient to hospice when necessary, and U 4 - U 2 - H Rx - H Hosp corresponds to the regret associated with unnecessary hospice referral.

Choosing the optimal strategy

The optimal strategy is selected as the one which will bring the least amount of regret. The regret DCA algorithm expresses the regret associated with each strategy in terms of threshold probability and is implemented as follows [24]:

  1. 1.

    Select a value for threshold probability.

  2. 2.

    Assuming that patients should be referred to hospice if p ≥ P t and should continue current treatment otherwise, compute #TP and #FP for the prediction model.

  3. 3.

    Calculate the ERg(SUPPORT)using equation 5.

  4. 4.

    Calculate ERg(Rx) using equation 4.

  5. 5.

    Compute the ERg(Hosp)using equation 3.

  6. 6.

    Repeat steps 1 - 6 for a range of threshold probabilities.

  7. 7.

    Graph each expected regret function calculated in steps 3-5 against each threshold probability.

At each threshold probability, the action with the lowest value of expected regret corresponds to the most desired action. For example, in Figure 4, at a threshold probability equal to 10% (e.g. the patient considers 9 times worse not to be referred to hospice when necessary than to receive an unnecessary hospice referral), the optimal strategy is to refer the patient to hospice.

Figure 4
figure 4

Decision curves for hospice referral. In this figure, RRRHosp = 0, RRRRx = 0, HRx = HHosp = Hte = 0. At threshold probability equal to 10%, the optimal decision is refer the patient to hospice; at 40% the optimal decision is to use the SUPPORT model.

Full size image

Figures 5, 6 and 7 depict the regret associated with alternative decision strategies as they relate to different values of hospice effectiveness (Figure 5), treatment effectiveness (Figure 6), and harms due to the prognostication test (Figure 7). As expected, when the harms due to the prognostication test are increased, then the area of threshold probability at which the prognostication model is the optimal decision is reduced (Figure 5). Even though, it is not expected that the SUPPORT model will actually create harms, at least physiological, to the patient, this is not always the case for other diagnostic tests that may be more invasive (e.g. screening for prostate cancer).

Figure 5
figure 5

Decision curves as a function of RRR RX . In this figure, RRRHosp = 0, RRRRx = 2 to 8%, HRx = HHosp = Hte = 0. As the effect of treatment increases, the regret associated with treating all patients and with the SUPPORT model slightly decreases. The strategy of using the SUPPORT model to refer a patient to hospice is the action with the least amount of regret for the wider range of threshold probabilities.

Full size image
Figure 6
figure 6

Decision curves as a function of RRR Hosp . In this figure, RRRHosp = 2% to 8%, RRRRx = 0, HRx = HHosp = Hte = 0. As the effect of hospice care increases, the regret associated with hospice and with the SUPPORT model slightly decreases. As previously, the strategy of using the SUPPORT model to refer a patient to hospice is the action with the least amount of regret for a wide range of threshold probabilities.

Full size image
Figure 7
figure 7

Decision curves as a function of H te . In this figure, RRRHosp = 0, RRRRx = 0, HRx = HHosp = 0 and Hte = 0% and 10%. As the harms associated to the prediction test increase, so does the expected regret of utilizing the SUPPORT model for hospice referral. Increasing the harms due to treatment or due to hospice care does not have an effect on the decision curves.

Full size image

As can be seen from Figures 5, 6 and 7 the optimal decision is derived by the SUPPORT model for a rather wide range of threshold probabilities. Therefore, it appears that the SUPPORT model is the superior strategy for the vast majority of decision makers, regardless the effects of the alternative management strategies. However, since the threshold probability expresses the personal preferences of a particular decision maker, it is not unusual for specific patients to have smaller or greater threshold probability values than the majority of decision makers. This is the power of the proposed methodology, which allows for decision making at the individual level. For example, if the decision maker presents a threshold probability greater than ≈92%, the optimal decision would be to continue life-sustaining treatment even if it is deemed not to be effective(Figure 4). Similarly, for small values of threshold probability, the desired action would be to refer the patient to hospice.

Decision Support System

As our theoretical discussion highlighted, decisions about life and death are complex and difficult at both the emotional and cognitive level. Therefore, it is not surprising that the SUPPORT model originally failed to improve the quality of care for terminally ill patients despite its reasonable accuracy in prediction of probability of survival [17, 20]. Any attempt to focus on a single dimension of the complex hospice referral process is not likely to succeed. An accurate prognostic model is only the first step. Having the apparatus to take into account trade-offs associated with the hospice referral decision while taking into consideration the patients' preferences represent further necessary steps to improve the care of terminally ill patients. In addition, we hypothesize that the SUPPORT intervention failed because it was not available at the point of care in real time. This is because the most desired outcomes are best achieved when decision-making occurs in real-time, at the point of care [33, 34].

To facilitate the decision making process for the hospice referral at bedside, we propose a web-based clinical decision support system (CDSS) that computes the probabilities of survival and death for individual patients using the SUPPORT model, elicits personal preferences from patients and/or physicians, and utilizes regret DCA to suggest the optimal decision for a particular patient.

Features

Access

Our goal is to develop a CDSS that can be accessed by everyone and from anywhere regardless the operating system one uses. At the same time, it is desirable to develop a system that can eventually be integrated with various healthcare providers' electronic medical records (EMR). We concluded that a web-based implementation would fulfil such requirements.

Data storage

The CDSS performs the required computations without retaining or transmitting sensitive and identifiable information.

Results

In this section we present a prototype of the CDSS, developed to demonstrate the applicability of our theoretical framework for hospice referral. Each subsection describes the results of the methods shown in the previous section in conjunction with the description of the corresponding module. Figure 8 depicts the logical diagram that outlines the operation of the CDSS. Briefly, the operation begins by interviewing the patient or surrogate to elicit his/her threshold probability. Based on the value of threshold probability equation, the optimal strategy for the particular patient is derived (e.g. refer to hospice, continue treatment, or use of prediction model). If the optimal strategy is to follow the prediction model (SUPPORT), then using the equations A1, A2 and 1, the probabilities of survival and death are computed for the particular patient. The probability of death is then contrasted with the patient's threshold probability and the optimal decision is derived (refer to hospice, or continue treatment). At each step described, the patient selects the level of information he/she wishes to be exposed to. For example, the patient may not wish to know his/her threshold probability or probability of death. Instead he/she wishes to know only the optimal decision regarding his/her condition.

Figure 8
figure 8

Block diagram outlining the operation of the DSS.

Full size image

General implementation details

The proposed CDSS is a web-based application residing on the USF Health servers. The web address is http://health.usf.edu/research/ebm/decisionaids.htm. It has been developed based on the Adobe® ColdFusion® application technology and the interface has been designed using html and JavaScript programming languages. The hardware and software requirements from the user's point of view are modest. The system runs on any contemporary computer with net browsing capabilities. However, at this stage the CDSS is not optimized for use with handheld devices. The CDSS consists of 3 different modules as described below.

Elicitation of threshold probability module

The threshold elicitation module consists of the dual visual analogue scales, used to weigh the patient's regret in the case of wrong decisions. Each scale has 100 points where 0 corresponds to no regret and 100 to maximum regret. Depending on the role of the decision maker (e.g. patient/surrogate or physician) two different sets of questions are displayed. These questions are designed to capture the regret of omission and the regret of commission. For the remainder of this paper, we assume that the decision maker is the patient. As in pain scales [35], each visual analogue scale uses facial expressions to graphically represent variations in regret (Figure 9). A summary of the decision maker's preferences is presented for final verification. The threshold probability for the particular patient is derived using equation 2, however is not displayed until the decision maker requests it.

Figure 9
figure 9

Elicitation of threshold probability. The user (patient/surrogate/physician) weighs the two alternative management strategies in terms of regret.

Full size image

Decision module

The decision module utilizes the decision maker's threshold probability and the regret DCA methodology to derive the optimal decision. For example, the preferences of the patient depicted in Figure 9, correspond to a threshold probability equal to 29%. From Figure 4 the strategy that will bring the least amount of regret is to use the prognostication model (SUPPORT) for the hospice referral recommendation. In this case, the decision module initiates the SUPPORT module.

SUPPORT module

If the optimal strategy derived by the decision module is to utilize the prognostication model, the SUPPORT module is enabled. This module (Figure 10) is used to compute the probability of death for the particular patient based on the SUPPORT prognostication model. Currently, the user inserts all required information to the CDSS. In the future, this information will be captured automatically from the health care provider's electronic medical records system. Data validation restrictions have been imposed to protect the integrity of the collected data.

Figure 10
figure 10

SUPPORT user interface. The user enters all information regarding the particular patient to compute the probability of death and survival within the next 6 months. LE results are presented to the patient through the decision justification module after the patient's request.

Full size image

Once the values of all available variables have been inserted in the corresponding cells, the patient's life expectancy and probabilities of survival and death are computed. The decision module is employed again to display the optimal recommendation.

Decision justification module

The decision justification module explains in detail and at the user's request the reasons that led to a particular recommendation (Figure 11). It contains information regarding the decision maker's threshold probability, the optimal strategy associated with the threshold probability and the patient's probability of death (if applicable). Since people often misinterpret probabilities [36], we complement the results presented in terms of probabilities using frequency format (Figure 11). The latter format is currently considered the best way to represent favorable and unfavorable facts regarding medical interventions [37]. The justification module is highly technical and should only be reviewed by decision makers who wish to know more about their or their patient's condition.

Figure 11
figure 11

Justification of the hospice referral recommendation. The particular patient depicted has 29% threshold probability at which the optimal strategy is derived by the SUPPORT model. The patient has 85% probability of death in the next 180 days. Therefore, the optimal decision is to be referred to hopsice.

Full size image

Case Study

Figure 12 summarizes the decision process for a patient whose information is simulated in Figures 10, 11 and 12. The probability of death and the threshold probability of this patient have been computed as 85% and 29% respectively. At a 29% threshold probability, the optimal strategy is to use the prediction model for hospice referral (Figure 4). Therefore, since p > P t the patient should be referred to hospice. For completeness, all possible decision routes are depicted in Figure 12. The route corresponding to the specific simulated patient is shown using bold arrows.

Figure 12
figure 12

Block diagram summarizing the decision process. The bold route corresponds to the patient simulated in figures 10, 11 and 12.

Full size image

Discussion

In this article we describe both the theory and application behind a hospice referral clinical decision support system. To the best of our knowledge, this is the first CDSS that integrates two well established methodologies, one for prognostication (SUPPORT) and the other for decision making (regret DCA), to assist with the hospice referral decision-making process.

The recently developed regret DCA incorporates the decision maker's preferences towards alternative management strategies from the perspective of regret theory in terms of threshold probability. Such an approach promotes personalized patient care. We anticipate that the regret-based approach is more appropriate for the hospice referral process than other preference elicitation techniques, due to the nature of the problem where there are really no optimal options available- the optimal decision can be only considered as the one with the least regret.

Modern cognitive theories increasingly focus on the so called dual-processing theory in which both intuition (system 1) and analytical, deliberative process (system 2) are important for balancing risks and benefits in the decision-making process [38]. We believe that rational decision-making should take into account both formal principles of rationality and human intuition about good decisions[24, 39, 40]. One way to accomplish this is to use regret, a cognitive emotion, to serve as the link between systems 1 and 2 [24]. By taking into account the consequences of our actions as well as the circumstances under which we can live with our mistakes we anticipate that the goal of reconciling the formal principles of rationality and human intuitions about good decisions can be met [24, 29, 39, 40]. This is particularly true in the situation of terminally ill patients.

Our web-based CDSS reflects modern cognitive theories to facilitate integration of the decision-making ingredients necessary for hospice referral decisions. The CDSS encapsulates all required information for the hospice referral process into a flexible software that can be used at bedside. Obviously, hospice referral decisions are complex and must be exercised with full compassion and deliberation. We advise against the use of our system as an automatic decision making tool that by-passes important personal interactions between the patient and his/her physician. It is important to stress that the elicitation of the threshold probability as described herein reflects the belief (also captured in recent legislation [41]) that patients and their families want to be told the "truth" about the patients' terminal sickness [22, 41, 42] and that physicians have ethical obligations to share this information with patients and their families [41, 42]. Our system should be understood as an aid to facilitate decisions in terminal phases of patient lives.

Our approach has limitations as well. The main limitation of the proposed system remains the complexity of the SUPPORT model. Currently, the system still requires manual entry of data. In addition, failure to enter all data can jeopardize the accuracy of prediction and therefore, the decision process. To cope with this limitation, we plan to integrate our system into various health providers' EMRs. Based on each EMR, specifically designed queries will be used to retrieve lab values and patient demographics to be fed automatically into our system; a process that will reduce the amount of missing values and input errors.

The second limitation of the proposed system is that empirical data are not available to assess how the system actually works in practice. While we plan to undertake empirical testing of the system described here, we believe that a strong theoretical underpinning will enable better hospice referral decisions even in the current form. This is because our system will essentially operationalize the decision-making process, which is supposed to occur in every day practice. Nevertheless, we need to firstly, identify the system's feasibility in real life settings and ultimately, if it appears to be usable and assessed favourably by all those involved in the hospice-referral decision-making process, to test it in randomized controlled trials against traditional care.

Our future plans include both empirical testing and implementation of multiple additional prognostication models which will be used in parallel to assess optimal decisions regarding hospice referral and take advantage of the regret DCA methodology. We anticipate that for a different range of threshold probabilities these models may perform better than the SUPPORT model. Furthermore, our intent is to develop a separate version of our CDSS optimized for mobile devices.

Conclusions

In this work we have presented the theoretical framework, accompanied by the associated CDSS, to facilitate end of life care decisions. Our work combines the prognostication power of the SUPPORT model, the simplicity of the DVAS methodology in eliciting people's preferences and the effectiveness of regret DCA at evaluating alternative management strategies to resolve the dilemma of choosing traditional vs. palliative care for patients at terminal stages. A clinical evaluation of the CDSS is planned.

Appendix

Support implementation

SUPPORT is implemented in two steps. First, the SUPPORT physiology score is computed based on equation A1 [22].

S P S = 259 . 9{ A R F / M O S F } + 263 . 4{ C O P D / C H F } + 241 . 4{ C i r r h o s i s / C o m a } + 281 . 5{ L u n g / C o l o n C a n c e r } − 0.0 6174min ( P a O 2 / F i O 2 , 225 ) − 0. 6316min( M e a n B P , 6 0 ) + 1 .0 2 0 5 W B C − 0. 3676( W B C − 8) + − 0. 5631( W B C − 11) + + 0. 2691min( A l b , 4 . 6) + 0. 2312 A r e s p − 2 . 362 T e m p + 1 . 326( T e m p − 36 . 6) + + 2 . 473( T e m p − 38 . 3) + − 1 . 579 × 1 0 – 1 H R + 9 . 77 0 × 1 0 – 5 ( H R − 55) + 3 − 2 . 189 × 1 0 − 4 ( H R − 8 0 ) + 3 + 1 . 518 × 1 0 - 4 ( H R − 11 0 ) + 3 − 3 .0 62 × 1 0 − 5 ( H R − 149 ) + 3 + 0. 9763 B i l − 0. 7481( B i l − 7) + − 6 . 8761 C r + 11 . 6 0 58( C r − 0. 6 00 ) + 3 − 21 . 8413( C r − 1 .000 ) + 3 + 1 0. 3574 ( C r − 1 . 5 00 ) + 3 − 0. 1219 ( C r − 5 . 399 ) + 3 − 0. 6167 0 96 N a + 0.00 21118( N a − 128) + 3 − 0.00 3673 0 ( N a − 135 ) + 3 + 0.000 6126( N a − 139) + 3 + 0.000 9486 ( N a − 148 ) + 3 − 6 . 278{ C O P D / C H F } × min( A l b , 4 . 6) − 11 . 45{ L u n g / C o l o n C a n c e r } × min( A l b , 4 . 6) + { A R F / M O S F } [ − 2 . 3549 W B C + 2 . 7494( W B C − 8) + − 0. 4638( W B C − 11) + ]
(A1)

where:Alb: albumin; Aresp: APACHE III respiration score; Bil: bilirubin; Cr: Creatinine; Na: sodium; PaO 2 : partial pressure oxygen in arterial blood; MeanBP: mean arterial blood pressure; WBC: white blood cell count in thousands; Temp: temperature in Celsius; HR: heart rate per minute; ARF: Acute respiratory failure; MOSF: Multiple organ failure; Cirrhosis: Cirrhosis; Coma: Coma; Lung: Lung cancer; ColonCancer: Colon cancer; COPD: Chronic obstructive pulmonary disease; CHF: Congestive heart failure. Also:

{ d i s e a s e g r o u p } = 1 , if patient in the disease group 0 , otherwise
( x ) + = x , if x > 0 0 , otherwise

WBC = 9, if WBC < 9 and {disease group } ≠ ARF /MOSF

WBC = 40, if WBC > 40

Cr = 15, if Cr >15

The second step in implementing the SUPPORT model is to calculate the probability of survival for the individual patient based on equation A2 [22].

P { T ≥ t | disease group = i } = S i ( t ) e X b ^
(A2)

where T: survival time in days; t: arbitrary time; S described in Table 1[22] and

Table 1 Values of Survival (S) as described in equation A2 for different disease types and varying survival times
Full size table
X b ^ = − 3 . 652 + 0. 8356{ C H F } + 0. 9257{ C i r r h o s i s } + 0. 6287{ L u n g C a n c e r } ± 1 . 18 0 3{ M O S F w / M a l i g } + 0.0 1434 S c o m a ± 0.0 1935 A g e + 0. 2413 C a n c e r − 1 . 863 [ H d a y + 3 . 4 ] − 1 + 0.0 8121 S P S + A g e [ 0.0 15261{ C O P D / C H F / C i r r h o s i s } + 0.00 9 0 47{ C o m a } − 0.00 8294{ C a n c e r } ] + A g e [ − 0.0 12498{ C H F } − 0.00 4578{ C i r r h o s i s } − 0.00 1435{ L u n g C a n c e r } − 0.0 13891{ M O S F w / M a l i g }

where Scoma: SUPPORT coma score (0-100); MOSFw/Malig: Multiple organ failure with malignancies; Hday: day in hospital when qualified for study; Cancer: Cancer by comorbidity or primary disease category (0 = no; 1 = present; 2 = metastatic) [22].

Derivation of the Expected Regret functions

As outlined in the Introduction, seriously and terminally ill patients may reap a number of benefits by the hospice program. Nevertheless, after enrollment into hospice, the patient (or the family, or the physician) may feel that this was a wrong decision, and subsequently may regret it. Similarly, the patient may feel regret for the treatment that he/she continues to receive because it is unnecessary, inappropriate, and/or harmful. Figure 3 represents our hospice decision tree in terms of regret from which we can compute the expected values of regret associated with each strategy as follows:

E R g [ H o s p ] = ( 1 - p ) * ( U 4 - U 2 - H R x - H H o s p )
(A3)
E R g [ R x ] = p * ( U 1 - U 3 + H R x - H H o s p )
(A4)
E R g [ S U P P O R T ] = p * T P * H t e + ( 1 - p ) * F P * ( U 4 - U 2 - ( H R x - H H o s p ) + H t e ) + p * F N * ( U 1 - U 3 + ( H R x - H H o s p ) + H t e ) + ( 1 - p ) * T N * H t e
(A5)

The variables TP, FP, TN, FN are related to the probabilities P (p ≥ P t ∩ D +), P (p ≥ P t ∩ D -), P (p < P t ∩ D -) and P (p < P t ∩ D +) respectively, and are estimated as follows:

  • P (p ≥ P t ∩ D +) ≈ the number of patients who will die within 6 months and for whom the prognostic probability is greater than or equal to P t (with #TP = number of patients with true positive results, P ( p ≥ P t ∩ D + ) ≈ # T P n , where n is the total number of patients in the study).

  • P (p ≥ P t ∩ D -) ≈the number of patients who will survive for longer than 6 months and for whom the prognostic probability is greater than or equal to P t (with #FP = number of patients with false positive results, P ( p ≥ P t ∩ D − ) ≈ # F P n ) .

  • P (p < P t ∩ D +) ≈the number of patients who will die within 6 months and for whom the prognostic probability is less than P t (with #FN = number of patients with false negative results, P ( p < P t ∩ D + ) ≈ # F N n ) .

  • P (p < P t ∩ D -) ≈the number of patients who will survive for longer than 6 months and for whom the prognostic probability is less than P t (with #TN = number of patients with true negative results, P ( p < P t ∩ D − ) ≈ # T N n ) .

To incorporate the effects of alternative treatments (e.g. treatment and hospice care) in equations A3-A5 we use the Relative Risk Reduction reported in literature for each strategy as follows:

E R g [ H o s p ] = ( 1 - p ) * ( 1 - R R R H o s p ) * ( U 4 - U 2 - H R x - H H o s p )
(A6)
E R g [ R x ] = p * ( 1 - R R R R x ) * ( U 1 - U 3 + H R x - H H o s p )
(A7)
E R g [ S U P P O R T ] = p * ( 1 - R R R H o s p ) * T P * H t e + ( 1 - p ) * ( 1 - R R R H o s p ) * F P * ( U 4 - U 2 - ( H R x - H H o s p ) + H t e ) + p * ( 1 - R R R R x ) * F N * ( U 1 - U 3 + ( H R x - H H o s p e ) + H t e ) + ( 1 - p ) * ( 1 - R R R R x ) * T N * H t e
(A8)

Since TP + FN = 1 and FP + TN = 1, we have:

p * T P + ( 1 - p ) * F P + p * F N + ( 1 - p ) * T N = p + ( 1 - p ) = 1

Therefore, equation A8 becomes:

E R g [ S U P P O R T ] = ( 1 − p ∗ R R R H o s p ∗ T P − ( 1 − p ) ∗ R R R H o s p ∗ F P − p ∗ R R R R x ∗ F N − ( 1 − p ) ∗ R R R R x ∗ T N ) ∗ H t e + ( 1 − p ) ∗ ( 1 − R R R H o s p ) ∗ F P ∗ ( U 4 − U 2 − ( H R x − H H o s p ) ) + p ∗ ( 1 − R R R R x ) ∗ F N ∗ ( U 1 − U 3 + ( H R x − H H o s p ) )
(A9)

Scaling the equations A3, A4 and A9 with the quantity (U 1 - U 3 + H Rx - H Hosp ) and replacing the expression U 4 - U 2 - ( H R x - H H o s p ) U 1 - U 3 + H R x - H H o s p with P t 1 - P t , we derive the final equations for the expected regret (equations 3, 4, and 5).

References

  1. Christakis NA: Death foretold. Prophecy and prognosis in medical care. 2001, The University of Chicago Press

    Google Scholar 

  2. Bulkin W, Lukashok H: Rx for dying: the case for hospice. N Engl J Med. 1988, 318 (6): 376-378. 10.1056/NEJM198802113180609.

    Article  CAS  PubMed  Google Scholar 

  3. Christakis NA, Iwashyna TJ: Spousal illness burden is associated with delayed use of hospice care in terminally ill patients. J Palliat Med. 1998, 1 (1): 3-10. 10.1089/jpm.1998.1.3.

    Article  CAS  PubMed  Google Scholar 

  4. Goldberg RJ, Mor V, Wiemann M, Greer DS, Hiris J: Analgesic use in terminal cancer patients: report from the National Hospice Study. J Chronic Dis. 1986, 39 (1): 37-45. 10.1016/0021-9681(86)90105-0.

    Article  CAS  PubMed  Google Scholar 

  5. Greer DS, Mor V: An overview of National Hospice Study findings. J Chronic Dis. 1986, 39 (1): 5-7. 10.1016/0021-9681(86)90102-5.

    Article  CAS  PubMed  Google Scholar 

  6. Moinpour CM, Polissar L: Factors affecting place of death of hospice and non-hospice cancer patients. Am J Public Health. 1989, 79 (11): 1549-1551. 10.2105/AJPH.79.11.1549.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Kane RL, Klein SJ, Bernstein L, Rothernberg R, Wales J: Hospice role in alleviating the emotional stress of terminal patients and their families. Med Care. 1985, 23 (3): 189-197. 10.1097/00005650-198503000-00001.

    Article  CAS  PubMed  Google Scholar 

  8. Dawson NJ: Need satisfaction in terminal care settings. Soc Sci Med. 1991, 32 (1): 83-87. 10.1016/0277-9536(91)90131-U.

    Article  CAS  PubMed  Google Scholar 

  9. Kidder D: The effects of hospice coverage on Medicare expenditures. Health Serv Res. 1992, 27 (2): 195-217.

    CAS  PubMed  PubMed Central  Google Scholar 

  10. Mor V, Kidder D: Cost savings in hospice: final results of the National Hospice Study. Health Serv Res. 1985, 20 (4): 407-422.

    CAS  PubMed  PubMed Central  Google Scholar 

  11. Temel JS, Greer JA, Muzikansky A, Gallagher ER, Admane S, Jackson VA, Dahlin CM, Blinderman CD, Jacobsen J, Pirl WF: Early palliative care for patients with metastatic non-small-cell lung cancer. N Engl J Med. 2010, 363 (8): 733-742. 10.1056/NEJMoa1000678.

    Article  CAS  PubMed  Google Scholar 

  12. Christakis NA: Timing of referral of terminally ill patients to an outpatient hospice. J Gen Intern Med. 1994, 9 (6): 314-320. 10.1007/BF02599178.

    Article  CAS  PubMed  Google Scholar 

  13. Christakis NA, Escarce JJ: Survival of Medicare patients after enrolment in hospice programs. The New England Journal of Medicine. 1996, 335 (3): 172-178. 10.1056/NEJM199607183350306.

    Article  CAS  PubMed  Google Scholar 

  14. [http://www.ssa.gov/OP_Home/ssact/title18/1861.htm]

  15. Dawes RM, Faust D, Meehl PE: Clinical versus actuarial judgment. Science. 1989, 243 (4899): 1668-1674. 10.1126/science.2648573.

    Article  CAS  PubMed  Google Scholar 

  16. Hastie R, Dawes RM: Rational choice in an uncertain world. 2001, London: Sage Publications Inc

    Google Scholar 

  17. The Support Investigators: A Controlled Trial to Improve Care for Seriously III Hospitalized Patients: The Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments (SUPPORT). JAMA. 1995, 247 (20): 1591-1598.

    Google Scholar 

  18. Cook DJ, Guyatt GH, Jaeschke R, Reeve J, Spanier A, King D, Molloy DW, Willan A, Streiner DL, Canadian Critical Care, Trials G: Determinants in Canadian Health Care Workers of the Decision to Withdraw Life Support From the Critically Ill. JAMA. 1995, 273 (9): 703-708. 10.1001/jama.1995.03520330033033.

    Article  CAS  PubMed  Google Scholar 

  19. Hamel MB, Goldman L, Teno J, Lynn J, Davis RB, Harrell FE, Connors AF, Califf R, Kussin P, Bellamy P: Identification of Comatose Patients at High Risk for Death or Severe Disability. JAMA. 1995, 273 (23): 1842-1848. 10.1001/jama.1995.03520470050031.

    Article  CAS  PubMed  Google Scholar 

  20. Teno J, Lynn J, Connors A, Wenger N, Phillips RS, Alzola C, Murphy DP, Desbiens N, WA K: The illusion of end-of-life resource savings with advance directives. SUPPORT Investigators. Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatment. J Am Geriatr Soc. 1997, 45 (4): 519-520.

    Article  Google Scholar 

  21. Teno JM, Lynn J, Phillips RS, Murphy D, Youngner SJ, Bellamy P, Connors AFJ, Desbiens NA, Fulkerson W, Knaus WA: Do formal advance directives affect resuscitation decisions and the use of resources for seriously ill patients? SUPPORT Investigators. Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments. J Clin Ethics. 1994, 5 (1): 23-30.

    CAS  PubMed  Google Scholar 

  22. Knaus WA, Harrell FE, Lynn J, Goldman L, Phillips RS, Connors AF, Dawson NV, Fulkerson WJ, Califf RM, Desbiens N: The SUPPORT Prognostic Model: Objective Estimates of Survival for Seriously Ill Hospitalized Adults. Annals of Internal Medicine. 1995, 122 (3): 191-203.

    Article  CAS  PubMed  Google Scholar 

  23. Lynn J, Knaus WA: Background for SUPPORT. J Clin Epidimiol. 1990, 43 (Suppl): 1S-4S.

    Article  Google Scholar 

  24. Tsalatsanis A, Hozo I, Vickers A, Djulbegovic B: A regret theory approach to decision curve analysis: a novel method for eliciting decision makers' preferences and decision-making. BMC Med Inform Decis Mak. 2010, 10: 51-10.1186/1472-6947-10-51.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Vickers A, Elkin E: Decision curve analysis: a novel method for evaluating prediction models. Med Dec Making. 2006, 26 (6): 565-574. 10.1177/0272989X06295361.

    Article  Google Scholar 

  26. Djulbegovic B, Hozo I: When Should Potentially False Research Findings Be Considered Acceptable?. PLoS Med. 2007, 4 (2): e26-10.1371/journal.pmed.0040026.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Djulbegovic B, Hozo I, Schwartz A, McMasters KM: Acceptable regret in medical decision making. Med Hypotheses. 1999, 53 (3): 253-259. 10.1054/mehy.1998.0020.

    Article  CAS  PubMed  Google Scholar 

  28. Hozo I, Djulbegovic B: When is diagnostic testing inappropriate or irrational? Acceptable regret approach. Med Dec Making. 2008, 28 (4): 540-553. 10.1177/0272989X08315249.

    Article  Google Scholar 

  29. Hozo I, Djulbegovic B: Will insistence on practicing medicine according to expected utility theory lead to an increase in diagnostic testing?. Med Dec Making. 2009, 29: 320-322. 10.1177/0272989X09334370.

    Article  Google Scholar 

  30. Bell DE: Regret in Decision Making under Uncertainty. Operations Research. 1982, 30: 961-981. 10.1287/opre.30.5.961.

    Article  Google Scholar 

  31. Loomes G, Sugden R: Regret theory: an alternative theory of rational choice. Economic J. 1982, 92: 805-824. 10.2307/2232669.

    Article  Google Scholar 

  32. Zeelenberg M, Pieters R: A theory of regret regulation 1.1. J Consumer Psychol. 2007, 17: 29-35. 10.1207/s15327663jcp1701_6.

    Article  Google Scholar 

  33. Olsen LA, Aisner D, McGinnis MJ: The learning healthcare system: Workshop summary (IOM roundtable on evidence based medicine). 2008, Washington: The National Academic Press

    Google Scholar 

  34. Eden J, Wheatley B, McNeil B, Sox H: Knowing what works in health care: A roadmap for the Nation. 2008, Washington: The National Academies Press

    Google Scholar 

  35. McCaffery M, Beebe A: Pain: Clinical manual for nursing practice. 1993, Baltimore: VV Mosby Company

    Google Scholar 

  36. Gigerenzer G, Todd PM, ABC-Research-Group: Simple heuristics that make us smart. 1999, New York: Oxford University Press

    Google Scholar 

  37. Schwartz LM, Woloshin S, Welch HG: Using a Drug Facts Box to Communicate Drug Benefits and Harms. Annals of Internal Medicine. 2009, 150 (8): 516-527.

    Article  PubMed  Google Scholar 

  38. Kahneman D: Maps of bounded rationality: psychology for behavioral economics. American Economic Review. 2003, 93: 1449-1475. 10.1257/000282803322655392.

    Article  Google Scholar 

  39. Krantz DH, Kunreuther HC: Goals and plans in decision making. Judgement and decision making. 2007, 52 (3): 137-168.

    Google Scholar 

  40. Rawls J: A theory of justice. Revised edition. 1999, Cambridge: Harvard University Press

    Google Scholar 

  41. Astrow AB, Popp B: The Palliative Care Information Act in real life. N Engl J Med. 2011, 364 (20): 1885-1887. 10.1056/NEJMp1102392.

    Article  CAS  PubMed  Google Scholar 

  42. Mack JW, Weeks JC, Wright AA, Block SD, Prigerson HG: End-of-life discussions, goal attainment, and distress at the end of life: predictors and outcomes of receipt of care consistent with preferences. J Clin Oncol. 2010, 28 (7): 1203-1208. 10.1200/JCO.2009.25.4672.

    Article  PubMed  PubMed Central  Google Scholar 

Pre-publication history

Download references

Acknowledgements

This work is supported by the Department of Army grant #W81 XWH 09-2-0175 (PI Djulbegovic).

Author information

Authors and Affiliations

  1. Center for Evidence-based Medicine and Health Outcomes Research, University of South Florida, Tampa, FL, USA

    Athanasios Tsalatsanis, Laura E Barnes & Benjamin Djulbegovic

  2. Department of Mathematics, Indiana University Northwest, Gary, IN, USA

    Iztok Hozo

  3. H. Lee Moffitt Cancer Center & Research Institute, Tampa, FL, USA

    Benjamin Djulbegovic

Authors
  1. Athanasios Tsalatsanis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laura E Barnes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Iztok Hozo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Benjamin Djulbegovic
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Athanasios Tsalatsanis.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

All authors contributed equally to this work. All authors have read and approved the final manuscript.

Authors’ original submitted files for images

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Tsalatsanis, A., Barnes, L.E., Hozo, I. et al. Extensions to Regret-based Decision Curve Analysis: An application to hospice referral for terminal patients. BMC Med Inform Decis Mak 11, 77 (2011). https://doi.org/10.1186/1472-6947-11-77

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1472-6947-11-77

Keywords

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

Contact us