Physicians’ gender and their use of electronic health records: findings from a mixed-methods usability study

Study design

We conducted a cross-sectional, mixed-methods assessment of the physician-EHR relationship among medical intensivists and the EHR (Epic; Epic Systems, Verona, WI) at one institution. We investigated this relationship by age, gender, professional role, and experience with the EHR.

We have previously reported the methodology for the current study in detail.²⁴ In brief, physicians participated 1-by-1 in a 3-part study that included: (1) completion of 4 simulation patient cases in the EHR, (2) completion of 4 surveys, and (3) participation in a face-to-face, semistructured interview. For each participant, all 3 parts of the study took place during a single encounter. We obtained written informed consent from all participants. Participants were compensated with $100 gift cards. This study was reviewed and approved by the Institutional Review Board.

For the study’s simulation component, 4 cases representative of typical medical ICU (MICU) patients were developed by a board-certified pulmonary and critical care attending and an internal medicine resident (T.B. and C.C.), neither of whom participated in the study. Description of test cases are summarized subsequently, while accompanying clinical tasks are discussed in greater detail in previously published work.²⁴ Two research assistants and an Epic consultant built the patient cases into the institutional EHR training environment, which closely mimics the live Epic environment.

Cases:

• A 44-year-old woman with multisystem organ failure. Participants review clinical documentation, manage medications, and respond to consultations.

• A 60-year-old woman with acute hypoxic respiratory failure. Participants review clinical documentation and flowsheets, evaluate changes and mechanical ventilation, and analyze microbiology data.

• A 25-year-old man with severe infection (sepsis). Participants assess flowsheets, laboratory data, antibiotics, and fluid management.

• A 56-year-old male trauma patient with postoperative heart failure and volume overload. Participants identify weight trends during previous visits and manage intravenous fluids and medications.

We provided standardized instructions to each participant to complete each patient scenario. Participants logged into their usual EHR view and reviewed the clinical cases sequentially, completing the assigned clinical tasks by providing verbal responses to questions posed by the experimenter or by performing actions in the EHR as indicated. Case questions were created by a domain expert (T.B.) and included questions about consultations received by the patient, the ordered labs, explanations about modifications in orders, justifications for current and changes in ventilator settings, lists of intravenous fluids and their changes, and the latest patient vital signs.

We recorded users’ times, mouse clicks, and keystrokes. We also documented participants’ age, gender, professional role, number of years of Epic experience (self-reported), and estimated number of hours using Epic per week (self-reported).

Following the simulations, the primary author (S.K.) conducted 1-on-1 interviews, accompanied by a note-taker who recorded interviewee responses. Participants were asked to share their perceptions and elaborate on: 1) the accessibility of information in the EHR and 2) the association between EHR use and physician burnout. Interviews were audio recorded and transcribed verbatim by 2 trained, research assistants. Interview transcripts were subsequently coded and analyzed by a qualitative research expert using standard software (Dedoose; University of California, Los Angeles, Los Angeles, CA).

Setting and participants

The study was conducted at a tertiary academic medical center in the Southeast with a 30-bed MICU. All testing took place in a standardized EHR usability laboratory on site, equipped with computer workstation, away from live clinical environments. We recruited participants through flyers and departmental emails. Eligible participants (1) were physicians in the MICU (ie, faculty or trainee) and (2) had MICU critical care experience using Epic.

Outcomes

Primary outcomes were task performance scores, efficiency of use (clicks, time, screens), and perception of workload, usability, and satisfaction. Secondary outcomes were interview responses addressing EHR usability and association with burnout (Table 1).

Statistical analysis

Descriptive statistics were used for subgroup comparisons. Statistical analyses were conducted using independent sample t tests and 2-way analyses of variance. First, independent sample t tests were conducted on all 3 of the surveys (NASA-Task Load Index, Questionnaire for User Interaction Satisfaction, and System Usability Scale) and on performance data to examine differences in responses between genders. A 2-way analysis of variance was conducted for gender and age to determine if there are significant interactions between these 2 variables on both the survey responses and performance. Participants were categorized by age (25-29, 30-34, and 35+ years), which generally correspond to level of clinical training. A 2-way analysis of variance was then conducted with gender and age category as fixed factors. All statistical testing included adjustment to control for potential confounding by age or clinical role. We used IBM’s SPSS version 22.0 (IBM Corp, Armonk, NY).

Performance

As shown in Table 3, the mean performance score was 19.1 of a possible 21 points, equaling an accuracy rate of 91.2% ± 7.1%. Overall task performance scores were similar for men (90% ± 9.3%) and women (92% ± 4.4%), with no statistically significant differences (difference = 2.0%; 95% confidence interval [CI], –8.6% to 3.3%; P = .374). Scores for each individual patient case were likewise similar, with men and women performing equally well.

Efficiency

Though the differences were not statistically significant, women consistently had higher levels of efficiency than men on several indicators, including task completion time and total mouse clicks. As per Table 3, participants completed all 4 cases in a mean time of 34.7 ± 8.5 minutes, but did not spend an equivalent amount of time on each case. In general, female physicians spent less total time completing all 4 cases, with a mean total time of 31.6 ± 8.7 minutes while men spent a mean total of 38.7 ± 10.8 minutes (difference = 7.1; 95% CI, –3.23 to 16.13; P = .207). No interactions were observed between either age group or clinical role and task completion time.

Clicks: Participants recorded a total mean number of 327 ± 87 mouse clicks in completing all 4 test cases (Table 3). The distribution of mouse clicks corresponded to the time spent on each case. Women recorded fewer total mouse clicks compared with men, with a mean number of 301 ± 67 mouse clicks and 355 ± 101 mouse clicks, respectively (difference = 54; 95% CI, –13.411 to 121.211; P = .13). Female physicians visited 3 more EHR screens in total (n = 86) as compared with male physicians (n = 83) (difference = 3; 95% CI, –14 to 20; P = .71).

Gender differences in perceived workload, satisfaction, and usability

Physicians’ reported workload, satisfaction, and usability scores were calculated from the NASA-Task Load Index, Questionnaire for User Interaction Satisfaction, and System Usability Scale (SUS), respectively. Across the board, we found significant gender differences for each of the 3 surveys (Figure 1).

Workload: Overall, male physicians reported a significantly higher perceived EHR workload compared with female physicians, with a mean workload of 54.3 ± 5.95 and 36.8 ± 10.57, respectively (difference = 17.5; 95% CI, 41.8 to 48.6; P < .001). Men perceived workload percentages were 61.6%, which were almost a third higher than women. Using industry standard thresholds for overwork (total Task Load Index >55), male participants exhibited overwork during the EHR simulation whereas women did not (Supplementary Appendices A).³³

Effort and Frustration: Male physicians reported significantly higher levels of frustration with the EHR compared with women, with mean male frustration scores more than twice as high as the women: 61.25 ± 18.8 vs 28.1 ± 21.6 (difference = 33.15; 95% CI, 19.0 to 47.3; P < .001). As shown in Figure 1, Male physicians also reported significantly more effort in completing tasks in the EHR compared with women, with men having a mean score of 66.3 ± 10.7 and women having a mean score of 41.1 ± 20.8 (difference = 22.2; 95% CI, 11.2-38.9; P < .001). Similarly, men reported that completing tasks in the EHR was significantly more mentally demanding. Mean score for men was 65.0 ± 18.8 compared with 45.9 ± 21.6 for women (difference = 19.1; 95% CI, 2.2 to 35.9; P = .02). For performance, female physicians had a mean score of 50.7 ± 14.3 while male physicians had a mean score of 44.1 ± 22.2, which was not statistically significant (P = .38). As for physical demand, male physicians reported a mean score of 15 ± 8.2, while female physicians had a mean score of 11.1 ± 10.2, which also was not statistically significant (P = .31). Scores >55 (of 100) signify overwork. There were no interactions found between either age group or clinical role and perceived EHR workload.

EHR Satisfaction: Female physicians reported significantly higher satisfaction with the ease of use of the EHR interface than men, as women had a mean score of 4.08 ± 0.49 vs 3.42 ± 0.99 for men (difference = 0.66; 95% CI, 0.02-1.32; P = .03). Female physicians also found the EHR system significantly easier to learn how to operate, with a mean score of 5.77 ± 1.74 vs 4.17 ± 1.47 for men (difference = 1.6; 95% CI, 0.256-2.94; P = .002). Women reported marginally significantly higher satisfaction with information accessibility (difference = 1.08; 95% CI, –1.05 to 2.81; P = .06) and ability to learn new features through trial and error than men (difference = 1.31; 95% CI, –0.178 to 2.97; P = .07). There were no interactions found between either age group or clinical role and satisfaction (Figure 2 and Supplementary Appendix B).

EHR Usability: Women’s perceived overall usability of the EHR was only marginally higher than that of the men, with a mean total of 66.35 ± 11.75 of 100 for women vs 56.04 ± 14.04 for men (difference = 10.31; 95% CI, 10.30-15.25; P = .06). Both of these scores fall in the “marginal usability” range by traditional interpretation of the SUS tool. Similarly, female physicians reported significantly lower scores than male physicians for both EHR complexity ratings (difference = 0.77; 95% CI, –1.125 to –0.070; P = .03) and EHR cumbersomeness (difference = 0.69; 95% CI, –1.414 to –0.137; P = .01). There were not differences in reported confidence levels using the EHR (difference = 0.38; 95% CI, –0.370 to 1.139; P = .30) (Figures 3, 4 and Supplementary Appendix C ).

Semistructured interview findings

During 1-on-1, semistructured interviews, 80% (n = 20) of participants reported frustration when trying to find information in the EHR. Consistent with the survey data, men expressed more frustration than did women: 11 of 12 male physicians vs 9 of 13 female physicians reported experiencing difficulty locating information. When asked about relationships between EHRs and burnout, 68% believe the EHR contributes to burnout (75% of male vs 57% of female physicians), Table 4.

For both men and women, the major complaint focused on time spent looking through patient information, alert fatigue, and documentation (Table 5). Participants repeatedly said they became doctors to work with people, not computers.

Different for New Physicians: One-third of participants who did not see a relationship between EHR and burnout (n = 8, 32%) noted that they only knew care with EHRs and therefore, they can’t compare work to pre-EHR eras. However, participants who practiced in the pre-EHR era were grateful for not chasing paper charts or reading illegible handwriting.

EHR features: favorites and frustrations

We asked participants to name the top 3 favorite and most frustrating features of Epic (Table 6). For favorite EHR features, male and female physicians both reported the summary features in the flowsheet screen that organizes and presents patient information; and the Results Review screen and active medication screens—including medication interactions and allergy warning, plus longitudinal trends. The third favorite feature differed between genders. Men preferred the “Care Everywhere” function (allows viewing records from different institutions using the same EHR system). In contrast, the majority of women favored the general search function, analogous to the “Google search bar” within a patient’s chart, and the filter option in chart review as these features expedite information seeking and retrieval.

EHR Usability: Poorly designed interfaces and functionality problems were among the top frustrating features noted by both women and men. Participants noted poor interface designs and numerous errors messages associated with orders (leading to “alert fatigue”). They cited problems from variability between inpatient and outpatient formats, and inconsistent display of lab results. Physicians also reported difficulty navigating through blood product administration, difficulty determining the total volume and timing of intravenous fluid administration, and the fact that the microbiology tab only show data from current hospitalization without old microbiology data juxtaposed.

Male physicians report high frustration with information overload, explicitly complaining of too much information in the EHR (eg, the need to visit multiple screens because of poor design, too many pathways to access the same information, non-objective organization of information, jumbled and dense formatting of the medication administration record). Women were frustrated by redundancy (specifically, in the Results Review and Active Medication tabs) and discordance in data presentation (eg, outdated and conflicting information in the Vital Signs tab compared with the flowsheet). As with the men, women also report poor medication administration record formats for reviewing active and current medications.

Strengths and limitations and future direction

Strengths of this study include a mixed-methods approach, high fidelity of the simulation EHR cases to actual patient cases, and diverse physician roles. Another strength is the study of medical intensivists’ EHR experience that combines data-intense patient records coupled with complex coordination of care plans, which allows for a holistic assessment of EHR usability.

Limitations include a single setting and single-EHR system, which reduce generalizability. However, we note that Epic is the most commonly used EHR among acute care hospitals in the United States.⁴² The breadth of our mixed-methods testing limited the overall sample size, and the relatively small sample size could have led to a chance finding or sampling bias. The slight imbalance in gender representation in the attending physician group may have introduced bias. However, we note that the literature indicates that 85%-97% of usability issues emerge with a sample size of 5-15 participants^31–33; our sample (n = 25) exceeds the conventional usability study standards, which adds credibility to our findings.

The focus on the MICU, while possibly comparable to other ICUs and very complex inpatient settings, may limit the generalizability of the findings to other medical areas and specialties. Additional research in other settings will be needed to explore wider generalizability. Also, although consistent gender differences arose, it is probable that EHR satisfaction rates may have been influenced by the nature of the medical specialization explored in this study.

Future work should investigate the potential ramifications of such gender-based usability patterns with respect to patient outcomes. This study detected different EHR usability patterns based on gender suggesting distinguishable satisfaction and burnout levels. Although there were slight differences in performance, more research is needed to investigate the effect of these usability gender-differences on patient safety and the long-term well-being of physicians. Furthermore, more studies are needed to assess the impact of EHR use on other professional roles such as nursing, pharmacy, and other health professions.