Introduction

Determining when hospitalized patients will be discharged is a complex process that depends on a variety of factors and a mutual understanding of patients’ discharge readiness [1, 2]. Confidence in an accurate expected discharge date (EDD) early during hospitalization allows the interprofessional team to establish safe, timely, and comprehensive discharge plans, particularly for complex patients with multiple chronic conditions [1, 3,4,5]. Furthermore, it has important implications for macro- and unit-level operations [6, 7]. An accurate EDD enables operational and clinical leaders to forecast bed capacity, manage emergency room and surgical volume, optimize patient throughput, adjust staff allocation, and facilitate early day discharges [6, 8,9,10]. For nurses, nurse care coordinators (CCRNs), advanced practice providers (APPs), and physicians, having confidence in the EDD estimate prompts earlier identification of medical and non-medical discharge barriers, facilitates timely delivery of transitional care interventions such as discharge medication reconciliation and patient counseling by pharmacists, and helps to establish clear expectations for discharge with patients and their caregivers [4, 9, 11, 12]. From the patient’s perspective, transparent communication about the EDD encourages preparation for discharge earlier during the hospital course, thereby mitigating readmission risk while improving safety and satisfaction during transitions [13,14,15,16,17].

While estimating the EDD is a hospital priority, it is neither reliable nor efficient, and it is not well understood [4, 18,19,20]. The discharge planning process is often chaotic due to rapidly evolving patient conditions and plans of care, last minute medication changes, pending insurance authorizations, arrangement of home services, post-acute care bed availability, and other factors [3, 5]. Moreover, roles and responsibilities for entering the EDD in the electronic health record (EHR) and revising it throughout the hospitalization remain unclear. These inefficient workflows and unstructured data entries exacerbate informational siloes and increase the risk of miscommunication among the care team, unit leaders, and hospital administrators attempting to ascertain the “source of truth” for the EDD of individual patients [21,22,23].

While it is not surprising that EDD entries are increasingly accurate as the actual discharge date approaches [24], little is known about the types of factors that are associated with an accurate EDD entered early in the hospital encounter [14, 15, 25]. Knowledge of the types of patient, encounter, unit, and provider factors commonly found in the EHR that are associated with accurate early EDD entries, including factors external to the EHR, can spur quality improvement initiatives to improve confidence about its accuracy. The COVID-19 pandemic further highlighted the importance of having accurate EDDs as hospitals were required to provide sufficient bed capacity, streamline processes for post-acute care decision-making, adhere to infection control guidelines and physical distancing mandates, and ensure capacity to address recurrent surges while optimizing management of volume for elective and urgent procedures and surgeries [26].

In this study we sought to develop multivariable models using a variety of EHR and non-EHR based factors to discriminate an accurate EDD entered early during hospitalization. Understanding the types of factors that are associated with an accurate EDD entered early during hospitalization is the first step for understanding how best to use EHR and non-EHR data to build more sophisticated machine learning models to confidently forecast the discharge date.

Methods

Study Design, Setting, Participants

This manuscript was developed in adherence to the Strengthening the Reporting of Observational Studies in Epidemiology guidelines [27]. We conducted a retrospective cohort study, approved by the Mass-General Brigham (MGB) Human Research Committee, at Brigham and Women’s Hospital (BWH), a 793-bed academic medical center affiliated with MGB in Boston, MA. All MGB hospitals use a common commercial EHR (Epic Systems, Inc.). All patients discharged from six general medicine units between January 2017 and June 2018 were considered eligible if they were either admitted or transferred into these units. Five units were staffed by housestaff teams (two “intensive” teaching units, where residents were given reduced patient-loads and more time for educational activities [28], and three general teaching units), and the sixth unit was staffed by an APP team. Each general medicine team—composed of an attending and two APPs (non-teaching), an attending, resident, and two interns (general teaching), or two attendings, two residents, and three interns (“intensive” teaching)—was geographically localized, and thus, cared for the majority of patients admitted to the unit to which that team was assigned [29]. A minority of patients admitted to the study units were cared for by oncology, surgery, or other non-general medicine teams. We excluded encounters of patients who expired in the hospital, those who were discharged against medical advice, and those who did not have any EDD entries within the first 24 h of their hospitalization.

Interdisciplinary team huddles (ITHs) occurred on weekdays on each unit, at 8:30 AM for the APP team and at 11:30 AM for housestaff teams. During these huddles, physicians and APPs reviewed the discharge plans with the care coordinator, charge nurse, and unit clerk. The unit clerk was primarily responsible for entering and/or updating the EDD in the EHR during and after ITHs, and this workflow did not change during the study period. However, any team member (e.g., clinician, care coordinator) could enter and update the EDD using the “Huddle Note” report (Fig. 1) at any time during the hospital encounter.

Fig. 1
figure 1

Clinician’s workflow for entering an EDD from the Patient Summary view in the EHR. Image is used with permission, Copyright ©2021 Epic Systems Corporation.

Full size image

Data Sources & Collection

Using our Enterprise Data Warehouse (EDW), which captures all data from our EHR, we retrieved all available EDD entries for eligible patients discharged from study units during the study period. We retrieved relevant patient-level administrative data associated with the hospital encounter, such as demographic information, admission-discharge-transfer (ADT) timestamps, insurance status, comorbidity indices, length of stay (LOS), discharge destination (home, skilled nursing facility, hospice facility, etc.), and the All Patients Refined Diagnosis-related Group (DRG) assigned to the encounter in claims data as well as its associated weight. Other EHR data included role of individual entering the EDD and daily census of the study units. Lastly, we retrieved non-EHR data, including daily snowfall in Boston, MA, from publicly available sources (National Centers for Environmental Information, National Oceanic and Atmospheric Administration) [30]. Data from our EDW and external sources were combined into a single database, linking individual EDD entries at the level of the hospital encounter.

Definition of Outcome & Independent Variables

We defined early EDD accuracy (dependent outcome) as instances in which any EDD entered during the first 24 h of hospitalization was equal to the patient’s actual discharge date (ADD). Our independent variables were derived from patient, encounter, unit, provider, and external data collected for eligible hospitalizations. Patient variables included age, sex, race, ethnicity, primary language, socioeconomic status (median income by zip code), insurance status, Elixhauser Comorbidity Index [31, 32], and the number of prior hospital admissions over the previous 12 months. Encounter variables included transfer from an outside hospital, DRG weight, LOS, admission on a weekend, discharge on a weekend, discharge destination, and discharging service (general medicine vs. non-general medicine). Our unit, provider, and external variables included the unit census at the beginning of the day of hospitalization, unit type (“intensive” teaching, general teaching, or non-teaching), role of the individual entering the EDD, and snowfall during the hospital encounter (primary model) or on day of admission (secondary model). In general, we identified explanatory variables within each category (patient, encounter, unit, etc.) that corresponded to structured data elements (i.e., that could be read or written to the EHR via an application programming interface or API) addressed known disparities or biases (e.g., racial inequities), had internal and external validity (e.g., discharge day on weekday or weekend), or correlated with the intended purpose of the clinical unit (e.g., teaching, non-teaching). Of note, for potential predictor variables that were likely co-linear with other variables (e.g., admission to observation status often associated with short LOS), we did not include both.

Multivariable Analyses

We conducted all analyses using SAS version 9.4 (SAS Institute, Cary, NC). We used descriptive statistics to compare demographic and administrative measures at the level of the hospital encounter in patients with vs. without accurate early EDDs as numbers and percentages, or means with standard deviations, as appropriate. In our primary multivariable logistic regression analysis, we employed generalized estimating equations (GEE) to model early EDD accuracy as the dependent outcome while adjusting for all other covariates and clustering by patient. For our secondary multivariable logistic regression analysis, we used GEE to model early EDD accuracy using only covariates present upon admission (e.g., demographic characteristics, prior hospitalizations, etc.), while also clustering by patient. We calculated unadjusted and adjusted odds ratios with 95% confidence intervals and p-values for each covariate. To assess the multivariable models’ abilities to accurately predict the EDD entered early during hospitalization, we calculated the c-statistic for each model, a measure of discrimination.

Results

Of the 8628 patient hospital encounters, 3917 (45.4%) were eligible for inclusion (Fig. 2). Of these 3917 encounters, 890 (22.7%) had at least one accurate EDD entered within the first 24 h of the patient’s admission to the hospital. Of the 890 accurate early EDD entries, 605 (68.0%) had only one EDD entry within the first 24 h. Encounters with an accurate early EDD entry tended to have a lower Elixhauser Comorbidity Index, were more likely to be admitted to observation or on a weekday, and were more likely to have an elective admission (Table 1).

Fig. 2
figure 2

Study Flow Diagram

Full size image
Table 1 Baseline characteristics of eligible patient hospital encounters, n = 3917
Full size table

In the primary analysis, the covariates significantly positively associated with early EDD accuracy (Table 2) included admit day during weekdays (OR 1.37, 95% CI 1.14–1.64, p < 0.001); discharge day during weekdays (OR 1.32, 95% CI 1.10–1.59, p = 0.003); general teaching units one, two, and three (OR 1.51, 95% CI 1.12–2.04, p = 0.007; OR 1.40, 95% CI 1.05–1.85, p = 0.02; OR 1.39, 95% CI 1.02–1.88, p = 0.04, respectively); and clinician-entered EDD (OR 1.81, 95% CI 1.26–2.62, p = 0.002). The covariates which were significantly negatively associated with early EDD accuracy (Table 2) included Elixhauser Comorbidity Index ≥ 11 (OR 0.82, 95% CI 0.68–0.99, p = 0.04); and LOS between 2 and 7 days (OR 0.39, 95% CI 0.32–0.48, p < 0.001) and 7 or more days (OR 0.01, 95% CI 0.005–0.02, p < 0.001). The c-statistic for the primary multivariable model was 0.75.

Of the 200 early EDDs assigned by clinicians, 94 (47.0%) were assigned by an APP (93 by a physician assistant, one by a nurse practitioner), 50 (25.0%) were assigned by an attending physician, 43 (21.5%) were assigned by a registered nurse (RN), and 13 (6.5%) were assigned by a physician trainee (nine by a fellow, four by a resident). Regarding rates of accurate early EDD entries by clinician type, attending physicians had 22 correct of 50 total entries (44.0%), APPs had 28 correct of 94 total entries (29.8%), RNs had 10 correct of 43 total entries (23.3%), and physician trainees had three correct of 13 total entries (23.1%).

Table 2 Analysis of Early EDD Accuracy, n = 3917
Full size table

In the secondary multivariable logistic regression model (Table 3), the covariate significantly positively associated with early EDD accuracy was admit day during weekdays (OR 1.22, 95% CI 1.03–1.45, p = 0.02). The covariates significantly negatively associated with early EDD accuracy included Elixhauser Comorbidity Index ≥ 11 (OR 0.59, 95% CI 0.50–0.70, p < 0.001), any prior admissions within the previous 12 months (OR 0.78, 95% CI 0.66–0.91, p = 0.002), and transfer from an outside hospital (OR 0.73, 95% CI 0.59–0.90, p = 0.003). The c-statistic for the secondary multivariable model was 0.60.

Table 3 Analysis of Early EDD Accuracy Using Factors Present on Admission, n = 3917
Full size table

Discussion

We conducted a retrospective study in which we used multivariable logistic regression using a priori identified patient, encounter, unit, provider, and external factors to model an accurate early EDD, defined as instances in which any EDD entered during the first 24 h of hospitalization was equal to the ADD. Just 22.7% of encounters had an accurate EDD entered in the EHR within the first 24 h of hospitalization. In our primary multivariable model, admit day during weekdays, discharge day during weekdays, general teaching units, and clinician-entered EDD were positively associated with an accurate early EDD entry during the hospital encounter, whereas a higher Elixhauser Comorbidity Index and moderate or long LOS were inversely associated. Discrimination was acceptable for the primary model (c-statistic 0.75) but was poor (c-statistic 0.60) for the secondary model that used factors present only upon admission.

Our observations have several explanations. First, patients admitted or discharged during the weekdays had a higher likelihood of an accurate early EDD entry. This was likely because ITHs routinely occurred from Monday through Friday. Immediately after ITHs, unit clerks would routinely update the EDD. Second, planned weekend discharges were likely deferred for a variety of reasons: transient weekend staff opting to defer a discharge until the weekday; limited care coordinator availability; and lack of bed availability or appropriate weekend staffing at receiving facilities. These factors likely contributed to the accuracy by which the ultimate discharge date—which would be on a weekday—would differ from the EDD originally entered in the EHR. Third, while a small number of EDDs were entered by clinicians, when an EDD was entered by a clinician early during hospitalization, such entries were more likely to be accurate – 31.5% of clinician-entered early EDDs were accurate compared to 22.2% of non-clinician early EDD entries (the majority of which were by unit clerks). This was likely because of clinicians’ direct involvement in patient care at the bedside (nurse or responding clinician), or in the decision regarding discharge (attending). Fourth, on teaching units, ITHs took place at the conclusion of formal clinical rounds as opposed to before rounds on non-teaching units. This likely allowed clinicians to make more accurate forecasts of the EDD based on the updated plan. Finally, it is noteworthy that “intensive” teaching units were not associated with accurate early EDD entries and may be attributed to staffing models, often consisting of two subspecialist attendings, and prioritization of educational activities. General teaching units, on the other hand, were staffed by hospitalist attendings who typically prioritize hospital efficiency and operational throughput [33, 34].

Conversely, Elixhauser Comorbidity Index and LOS and were inversely associated with early EDD accuracy. These factors should not be surprising: sicker patients with longer LOS typically have more complex clinical courses, therapeutic management options, and dispositions, which make it increasingly difficult for both clinicians and non-clinicians to forecast an accurate EDD early during hospitalization [35, 36]. The presence of snowfall during hospitalization, a factor external to the EHR and chosen a priori to understand the effects of weather on early EDD accuracy, takes into consideration patients’ preference (clinical staff agreeing to delay discharge until caregivers could safely drive the patient home) or delays at receiving facilities (skilled nursing facilities unable to receive a patient transfer until current patients were safely discharged) that cannot always be anticipated. While not statistically significant in the primary multivariable model (OR 0.84, 95% CI 0.68–1.03, p = 0.09), we observed a trend towards a negative association with early EDD accuracy.

Similar to other studies, we found that estimating the discharge date during hospitalization is challenging [20, 37,38,39]. For our analysis, we defined patient, encounter, unit, and provider variables a priori based on data that were readily available in most EHRs. Of the many variables typically retrievable from most EHRs (and thus, could be used as generalizable predictors), only Elixhauser Comorbidity Index scores and LOS were significantly associated with lower early EDD accuracy after adjusting for variables associated with markers of care complexity (prior admissions, transfer from outside hospital, DRG weight, discharge destination). Moreover, in our secondary multivariable model, in which we analyzed only factors present upon admission, markers of care complexity including a higher Elixhauser Comorbidity Index, any prior hospital admissions, and transfer from an outside hospital were all significantly associated with lower early EDD accuracy. These findings suggest that initial LOS projections and EDD estimates at the time of admission are unlikely to be useful as predictors for patients with more complex care needs and explains why accuracy of EDD entries improve closer to the actual discharge data as observed by Henry et al. [24].

Our study also offers early insights into the challenges of using generalizable variables common to most EHRs to predict an accurate EDD early during hospitalization [20, 39]. While our primary multivariable model had acceptable discrimination of the outcome, few variables would be useful for prospective prediction, as suggested by the poor discriminative ability of our secondary model. Additionally, while external data sources (bed availability at skilled nursing facilities, weather data, etc.) and even patient-reported data (status of completion of outcome questionnaires and discharge checklists, etc.) [14, 25] may improve predictive utility, this would need to be further validated, ideally using machine learning techniques [38, 40]. Nonetheless, the rapid adoption of APIs provides an opportunity to use external data sources to improve predictions beyond EHR data alone [38,39,40,41]. Finally, our study highlights potential actionable targets for quality improvement initiatives which can be prioritized by hospital operational and clinical leaders. For example, given that more than half (51.5%) of all encounters analyzed did not have a single early EDD entry, hospital-level strategies could consider incentivizing EDDs entered within the first 24 h of hospitalization, potentially targeted towards attendings at the same time of attestations for level of care (inpatient vs. observation), as required by Medicare [42].

Limitations

Our study has several limitations. First, it was conducted on patients admitted to general medicine units at a single academic medical center; different services, clinical units, and hospitals may vary in their EDD documentation practices. Second, we defined an accurate early EDD as any EDD entry within the first 24 h of admission; while it is possible that inaccurate entries may have occurred before or after the accurate EDD entry, most encounters in our dataset had just one EDD entry. Third, while other types of EHR data (conditional discharge orders, provisional discharge status, social determinants of health, etc.) could have predictive value, many were not routinely entered or updated by clinical staff at the time of this analysis. Fourth, we lacked access to other external sources of data (bed availability at receiving facilities); however, this could be true for most hospitals. Fifth, our study did not address why EDD entries were incorrect – a variety of complex workflow factors could be implicated, and our analysis did not take consider the extent to which entries were inaccurate (e.g., one day early vs. ten days late). Anecdotally, the pattern of EDD updates (e.g., rolling forward EDD by 3 days) by non-clinical staff (unit clerks) on certain days of the week (Fridays) are likely arbitrary based upon knowledge that the patient would not be discharged over the weekend. We note that while the responsibility for updating the EDD at our institution lies primarily with unit clerks, the delineation of responsibility is not always clear and likely varies by institution.

Conclusions & Implications

In summary, we conducted multivariable analyses to understand the types of variables that have potential to discriminate an accurate EDD entered early during hospitalization. Our analyses provide new insights about the types of factors that could be used to develop clinically and operationally useful prediction models of early EDD accuracy. Such models will likely require use of data from EHRs and external sources that have potential to influence the ADD. Future efforts can be directed towards using, developing, and validating predictive models of EDD accuracy, which can instill confidence in patients, clinicians, and operational leaders.