Antibiotic information application offers nurses quick support

ARTICLE IN PRESS American Journal of Infection Control ■■ (2016) ■■-■■ Contents lists available at ScienceDirect American Journal of Infection Control American Journal of Infection Control j o u r n a l h o m e p a g e : w w w. a j i c j o u r n a l . o r g Major articles Antibiotic information application offers nurses quick support Jobke Wentzel PhD a,*, Regine van Drie-Pierik PharmD b, Lars Nijdam PharmD b, Jos Geesing PhD c, Robbert Sanderman PhD a,d, Julia E.W.C. van Gemert-Pijnen PhD a,e a Department of Psychology, Health, and Technology, Faculty of Behavioural, Management and Social Sciences, Universitity of Twente, Enschede, The Netherlands b Department of Pharmacy, Medisch Spectrum Twente, Enschede, The Netherlands c Department of Pulmonary Medicine, Medisch Spectrum, Enschede, The Netherlands d Health Psychology Section, Department of Health Sciences, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands e Department of Clinical Microbiology, University Medical Center Groningen, Groningen, The Netherlands Key Words: eHealth antibiotic stewardship nurse task support Background: Nurses can be crucial contributors to antibiotic stewardship programs (ASPs), interventions aimed at improving antibiotic use, but nurse empowerment in ASPs adds to their job complexity. Nurses work in complex settings with high cognitive loads, which ask for easily accessible information. An information application (app) was developed to support nurses in ASPs. The efficiency, effectiveness, and user satisfaction regarding this antibiotic app were tested in a pilot study. Methods: The app was introduced into 2 lung wards of a local teaching hospital. During the 8-month pilot study, the 62 nurses of the wards had access to the app. Changes in user satisfaction regarding information support, safety attitudes, and ASP behavior were assessed with a questionnaire. At baseline, 28 nurses completed the (e-mail) questionnaire; after the study, 18 nurses participated. Scenario-based tests were done to assess app efficiency and effectiveness at baseline (n = 16) and in a randomized control (without the app, n = 17) and intervention condition (with the app, n = 17). Results: Significant improvements were found regarding task support (P = .041), reliability (P = .004), unobtrusiveness (P = .000), relevance (P = .002), user friendliness (P = .000), speed, and hyperlinks (P = .001). An improvement in communication was observed regarding nurse-physician understanding (P = .034). With the app, nurses solved the scenarios faster than without it. Conclusions: The human-centered design approach and persuasive strategy of task support were effective in reducing time needed to find information. Stewardship-related behaviors need active education strategies. © 2015 Association for Professionals in Infection Control and Epidemiology, Inc. Published by Elsevier Inc. All rights reserved. Nurses often work in complex medical settings and are required to check and act on various information flows and actual patient progress.1,2 This results in a high cognitive load, with risks of errors and suboptimal care.3 As more and more information and communication systems are in place to provide up-to-date information,4 integrated displays facilitate the timely processing and interpretation of information in clinical settings.5 With such systems, it is especially important that the design and content are adjusted * Address correspondence to Jobke Wentzel, PhD, University of Twente, PO Box 219, 7500AE Enschede, The Netherlands. E-mail address: m.j.wentzel@utwente.nl (J. Wentzel). Funding/Support: Interreg IVa-funded project EurSafety Heath-net (III-1-02 = 73), part of a Dutch-German cross-border network supported by the European Commission, the German Federal States of Nordrhein-Westfalen and Niedersachsen, and the Dutch provinces of Overijssel, Gelderland, and Limburg. Conflicts of Interest: None to report. to the target group with respect to their cognitive tasks and processes and the specific (clinical) setting in which they are used.6 Technology can therefore be supportive of health care processes in clinical settings, partially because it has the ability to support or reinforce behavior.7 It may be especially assistive in overcoming the challenges with adherence to guidelines and clinical protocols, for example by applying persuasive technology.8 The main goal of persuasive technology is to motivate users to reach goals via technology through persuasion, and not coercion or deception.8 The persuasive systems design (PSD) model, proposed by OinasKukkonen and Harjumaa, describes various strategies that can make technology persuasive.8 These strategies include supporting primary tasks (facilitating core tasks that need to be performed via the technology), offering dialogue support (easy interaction with—and via—the system), offering system credibility support (via cues of trustworthiness and reliability of the system and its contents), and offering social support (creating the motivation to use the system via social cues, such as comparisons or observations).8 The PSD model 0196-6553/© 2015 Association for Professionals in Infection Control and Epidemiology, Inc. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.ajic.2015.12.038 ARTICLE IN PRESS 2 J. Wentzel et al. / American Journal of Infection Control ■■ (2016) ■■-■■ has been applied to facilitate health behaviors of patients, but it has also shown to retain its value in enabling health care workers to perform tasks more efficiently and with greater enjoyment.9 Technology, especially when it is intended to be persuasive, has a strong potential for supporting nurses during their daily tasks. In this study, we focus on the information situation regarding antibiotic use, as part of the implementation of an antibiotic stewardship program (ASP). These programs aim to improve antibiotic use in clinical settings by offering a bundle of interventions aimed at the key stakeholders (prescribers, pharmacists, or clinical microbiologists).10 These improvements in the use of antibiotics are needed to control the increase in antimicrobial resistance. This has become a serious threat to patient safety because pathogens that have become resistant to antibiotics can cause infections that are difficult or impossible to treat.11 This is especially true in clinical settings, where patients’ health is already weak and bacteria can be transferred easily among patients.12,13 Besides a need for new antibiotics (the development of which takes considerable time and resources), experts state the importance of using antibiotics only when it is really necessary, using only the appropriate type, in the appropriate dosage and duration.10 At the core of ASPs is the prospective audit of patients who receive antimicrobial treatment, including intervention and feedback. Also, the use of certain antimicrobials may be limited (formulary restriction), and preauthorization can be required for use of specific antimicrobial agents.14 Besides these interventions that directly intervene with prescribing behavior, interventions that support changes in prescribing, such as education, implementation of guidelines and clinical pathways, and dose optimization, are recommended.14 ASPs therefore usually include strategies that inform prescribers about preferred antibiotics to use in certain situations. Furthermore, information sharing regarding symptoms, patient status, laboratory results, and local resistance patterns in order to timely adjust treatment is proposed as an important part of ASP.14,15 The role of nurses in ASPs often remains unaddressed and unclear. Nurses have been proposed as important actors in care involving antibiotic therapy.16-18 However, nurses are not responsible for evaluating antibiotic appropriateness or ensuring timely therapy adjustment (de-escalation). For this, the prescriber, pharmacist, and clinical microbiologist or infectious disease expert jointly consider diagnostic results and changes in current patient status.14 Even though nurses are not directly involved in these core processes, they are important contributors to ASPs. Nurses spend much time with the patient, put antibiotic therapy into practice, and monitor the treatment’s effect on the patient. In case of (sudden) changes or abnormalities, nurses are the eyes and ears of the physician. From this perspective, nurses have great potential in recognizing suboptimal antibiotic therapy. Also, nurses have a potentially important reflective role to play regarding antimicrobial use; they work with different residents and physicians on the ward and can notice differences in prescribing behaviors.19 Other research demonstrates that in nursing home settings and long-term care facilities, nurses are a target group for awareness, education, and information strategies aimed at changing prescription behavior at wards.20 In an earlier needs assessment, we found that to be able to take up a proactive role, nurses need to have access to—and be supported by—information.21 Such information includes concrete instructions on how to prepare and administer antibiotics. Also, information on acute responses to antibiotics, possible side effects, or allergic reactions should be readily accessible. Furthermore, nurses need access to information on how to administer a certain antibiotic and possible drug-drug interactions that may occur. In addition, more general information on the type of antibiotic and working mechanisms is needed, especially when the nurse is new to a certain antibiotic.21 Having access to this kind of information helps nurses carry out regular tasks and enables them to critically monitor and evaluate patient progress. To support information finding and use of information regarding the aforementioned types of information in a clinical setting, the Antibiotic App was developed. This application (app) offers centralized information, developed according to a human-centered design approach.21 In this app, readily available information sources are integrated into a single system, offering the nurse a quick way to find information. After starting the app, the nurse can select an antibiotic. Afterward, a dashboard overview of all information types appear, from which the information button that matches the nurse’s information need can be selected. From here, information supporting practical tasks (eg, instructions for administering or preparing parenteral antibiotics) and background information (eg, information on side effects, allergies, or the working mechanisms of the antibiotic) is available via the app (Fig 1). The app is available without login via the nurses’ medication registration system that is used during medication rounds. The app relies mostly on primary task support as a persuasive strategy, as was previously confirmed in analyses of user evaluations of the app.22 OBJECTIVE In this pilot study, we evaluate the Antibiotic App regarding its contribution to user satisfaction with antibiotic information, effectiveness, and efficiency. The app is part of the nurses’ information situation: all available information sources and their usability and satisfaction. We expect that by providing the app, the information situation will improve. As a result, we expect the app to support nurses in their antimicrobial-related tasks by making information easier and faster to find and allowing it to be applied more effectively as a result. The information support that the app provides should translate into a better overall appreciation of the information situation, should translate into improved communication and teamwork, and is expected to also affect behaviors related to antimicrobial stewardship (alerting and commenting on suboptimal antibiotic therapy). MATERIALS AND METHODS To assess app effectiveness, efficiency, and user satisfaction regarding information support, we applied various methods. A questionnaire was created to measure the information situation regarding satisfaction with information sources and their usability, both before and after implementation. In addition, communication and teamwork, safety behavior, and antimicrobial stewardship– related behavior were assessed via questionnaires. Furthermore, to assess whether nurses are better able to find and apply information more efficiently with the app than without the app, we conducted scenario-based user tests. Design and procedure All baseline measurements were conducted before the app was introduced at the wards via clinical lessons and by distributing fact sheets and flyers. The app was introduced in March 2013 as an addition to readily available information sources. Nurses were free to use their preferred information source (app or regular) during the pilot period. All postimplementation measurements were done 8 months after introducing the app at the wards. The nurses therefore had ample opportunity to use the app and, if so desired, integrate it into their daily work practice. App log data of this pilot period show daily and steady use.23 During the pilot, only log data were collected. All other results stem from data collected during baseline (preimplementation) or postimplementation measurements. The long pilot period prior to postimplementation measurements diminishes possible effects because of the novelty of the system ARTICLE IN PRESS J. Wentzel et al. / American Journal of Infection Control ■■ (2016) ■■-■■ 3 Fig 1. Screenshot of an overview page (left) and target page (right). (A) Open search field to search an antibiotic, (B) drop-down menu to select an antibiotic, (C) open search field to search within the pages of the selected antibiotic, and (D) information blocks, arranged by task or relevance (originally in Dutch). compared with the usual information sources. The setup of the pilot period and timing of measurements followed the following steps: 1. Questionnaire, baseline measurement (March 2013, n = 28). 2. Scenario tests (March 2013, n = 16). 3. App introduction and use on 2 lung wards (March 2013November 2013). 4. Scenario tests intervention (n = 17) and control (n = 17) condition (November 2013). 5. Questionnaire, postimplementation measurement (n = 18). Application could be used. After completing 3 scenarios, a short interview was conducted asking for a reflection on whether the scenario was realistic and how the nurse experienced resolving it. Each individual session lasted for approximately 30 minutes. The university’s ethical committee gave clearance for this study. All participants of the scenario tests signed an informed consent form. All participants received information about the study prior to participating and were informed about their right to withdraw at any moment. Setting A pre- and postimplementation questionnaire was sent via e-mail to all nurses of the wards (N = 62) to assess satisfaction with communication and teamwork, safety behavior, and ASP-related behavior. Up to 2 reminders were sent via e-mail in case of nonresponse. In addition to the questionnaire, the scenario-based tests were conducted. These tests included a baseline measurement and measurements in (randomized) intervention and control groups in postimplementation measurements. Every session took place at the ward of the nurse, in an office, or in an activity room, where at least 1 computer was present (which the nurse normally also uses or logs into). Every nurse was instructed about the test and the option of not participating or withdrawing at any moment. It was stressed that search capacities of individuals were not the focus of this study and that wrong answers or actions did not exist. After this, the audio recorder was started, the nurse was handed out a sheet with the scenarios and was asked to read his or her first scenario aloud. The nurses were asked to do what they would normally do in each scenario to find a solution and to think aloud while doing this. During baseline measurements and in the control condition, nurses were able to use all information sources they had available before the app was introduced. In the intervention condition, only the Antibiotic In this pilot study, 2 lung wards of a local 1,000-bed teaching hospital participated. The wards have a total of 57 beds. During the 8-month pilot phase, on average, a total of 62 nurses (45 full-time equivalent) worked at the wards. These wards were selected as pilot wards in consultation with a pharmacist and a clinical microbiologist of the hospital. The 2 pulmonary wards were chosen because they have a substantial antibiotic expenditure, whereas the patients admitted to these wards in general did not have the highest level of complexity or comorbidity (eg, as opposed to patients of an intensive care unit). Participants In the scenario-based tests, a convenience sample of 16 nurses participated in baseline scenario test measurements. After the 8-month pilot period, postmeasurement scenario tests were carried out with a convenience sample of 34 nurses. These 34 nurses were randomized into 2 conditions: intervention (use of the app, n = 17) and control (use of regular information sources only, n = 17). The data of 1 participant (control) could not be used in the analyses because ARTICLE IN PRESS 4 J. Wentzel et al. / American Journal of Infection Control ■■ (2016) ■■-■■ of technical difficulties. Of the nurses who participated in the baseline measurements, 7 also participated during postmeasurements (n = 4 in the experimental condition, n = 3 in the control condition). The nurses who participated in the scenario tests had a mean age of 31.9 years, and 84% were women. On average, they had 5.4 years of experience working on their ward and 9.1 years of experience as a nurse. No significant differences in characteristics were found between the groups, except for time working at the ward: 5.3, 4.2, and 6.9 years at baseline, control, and intervention, respectively. All nurses of the 2 participating wards (N = 62) were invited to complete a baseline and postimplementation questionnaire. A total of 28 nurses (response rate, 45%) participated in the preimplementation questionnaire; 18 nurses (response rate, 29%) filled out the postimplementation questionnaire. Of these participants, 9 nurses filled out both the pre- and postimplementation questionnaires. The mean age of the participating nurses was 37.7 years, and 87% were women. On average, they have 7.7 years of experience working on their ward and 12.6 years of experience as a nurse. No significant differences in characteristics were found. Materials Nurses choose from different sources to look up information during their tasks. It was not our goal to assess the availability or usability of these separate sources in comparison with the app. Specifically, user satisfaction regarding the Antibiotic Application has been assessed and reported elsewhere.24 The goal was to assess whether the app contributes to the availability and usability of information for nurses as a whole. We refer to this overall appreciation of availability and usability as the nurses’ information situation. To measure this, the Perceived Persuasiveness Questionnaire25,26 and 5 subscales (relevance, user friendliness, hyperlinks, and speed) of the Website Usability Questionnaire27 were administered. These scales were adapted slightly to measure available antibiotic information sources in general as opposed to one particular information source. To assess communication and teamwork, the subscales openness, accuracy, and understanding of the ICU Nurse Physician Questionnaire28 and the subscale teamwork climate of the Safety Attitudes Questionnaire29 were administered. To our knowledge, no instrument to measure antimicrobial stewardship behavior among nurses exists. Therefore, after consulting pharmacists, clinical microbiologists, and nurses about the behaviors that stem from the nurses’ (possible) role in ASP’s we formulated 4 item statements. These reflect the monitoring and alerting behaviors of nurses: “I give suggestions to physicians about the antibiotic treatment patients receive”; “When I have a feeling that something is not right about the antibiotic therapy of a patient, I discuss it with the physician”; “It is unusual to give physicians suggestions about the treatment patients receive”; and “Physicians ask nurses for advice.” Participants were asked to indicate to what extent these statements applied to them on a 5-point scale. For the scenario-based tests, 9 scenarios were created together with pharmacists (R.v.D. and L.N.) to depict situations that are relevant concerning antimicrobial stewardship that (can) occur during nurses’ work. In some cases, the scenario concerned a topic that would eventually result in alerting the physician and discussing the right course of action (eg, noticing and checking up on an inadequate dosage when a patient is overweight). Other scenarios focused on nurse tasks that do not involve direct responsibility on the physician’s end (eg, preparing the right concentration of intravenous antibiotics). Analyses All questionnaire items were measured on a 5-point Likert scale. Negatively formulated items were recoded. Because the number of items per scale differs, average sum scores were calculated per (sub) scale. Concerning the 4 ASP-related items, we report average (per condition) item scores. Differences between groups were statistically tested via independent samples t tests, using SPSS Statistics 22 (IBM, Armonk, NY). To assess scenario-test outcomes, time needed for the search was determined in seconds, based on the audio file records. In a few cases, no search was performed because the nurse immediately resolved the scenario (correctly or not) with ready knowledge or because they directly gave up or indicated they would call in for help instantly. In these cases, no time was recorded. In all other scenario outcomes, time in seconds was calculated from right after reading the scenario aloud (start) until the nurse verbalized his or her solution to the scenario (stop). For each scenario, we assessed whether the nurse found the right information, in the right source, and drew a correct conclusion from it. The following possible scenario outcomes were scored: 1. Wrong information was found by nurse, wrong answer (or no answer) given. 2. Search for information was done, no information was found, nurse gave up. 3. Nurse instantly gave up (did not initiate a search) or immediately called for a physician or pharmacist. In some cases, calling in experts may be better than trying to resolve something you really do not understand. 4. Nurse initiates search, but without success (does not understand or localizes the necessary information), then calls a physician or pharmacist or colleague for assistance. 5. Nurse provides wrong solution to scenario, but found the correct information needed to resolve the scenario. 6. Nurse provides the correct solution to the scenario, but bases it on an incorrect search (wrong information sought, or without searching for information). 7. Nurse provides the right solution to the scenario, after searching, finding, and interpreting the right information. Differences between the intervention and control groups were assessed by performing independent samples t tests (time needed to solve a scenario) using SPSS Statistics 22 and by comparing bar charts of scenario outcomes per condition. RESULTS Questionnaire The questionnaire outcomes provide insight into the nurses’ perception of their information situation. Improvements in user experience as expressed in the mean subscale scores of the Perceived Persuasiveness Questionnaire and Website Usability Questionnaire were found among all subscales (Table 1); these improvements were significant in task support, reliability, unobtrusiveness, relevance, user friendliness, speed, and hyperlinks (Table 1). In particular, the time participants perceive it generally takes to find information (speed) improved, from a 2.4 to 3.7 mean score (on a 1-5 scale). Safety-related scales (Nurse-Physician Questionnaire and Safety Attitudes Questionnaire) showed improvements in all mean scores, but only understanding significantly improved from 3.2 to 3.4. Finally, mean item scores on the ASP behavior items deteriorated or remained the same; none of these means differed significantly between pre and postmeasurement. Additional analyses were carried out to ensure that repeated participation (9 respondents filled out the pre- and postimplementation questionnaires) did not influence the results. Extended results of these analyses are available on request. In sum, the same direction ARTICLE IN PRESS J. Wentzel et al. / American Journal of Infection Control ■■ (2016) ■■-■■ 5 Table 1 Questionnaire results Instrument PPQ WEQ ICU Nurse-Physician Communication Scale SAQ Antibiotic stewardship behavior Subdomain or scale (no. of items)* Preimplementation (n = 29)* Postimplementation (n = 18)* P value Task support (2) Reliability (1) Persuasiveness (3) Unobtrusiveness (4) Relevance (3) User friendliness (3) Speed (2) Hyperlinks (2) Openness (4) Accuracy (5) Understanding (8) Teamwork (14) Make suggestions about antibiotic treatment Discuss suspicion of incorrect antibiotic treatment Making suggestions is common Physicians ask nurses for advice 3.9 (0.7) 3.7 (0.9) 3.8 (0.7) 2.8 (0.7) 3.6 (0.5) 2.9 (0.4) 2.4 (0.8) 3.4 (0.6) 3.7 (0.4) 3.4 (0.4) 3.2 (0.4) 3.8 (0.4) 2.8 (0.8) 3.9 (0.5) 3.4 (0.7) 3.1 (0.8) 4.3 (0.4) 4.3 (0.5) 3.9 (0.4) 3.7 (0.5) 4.1 (0.4) 3.3 (0.2) 3.7 (0.7) 4.0 (0.6) 3.8 (0.4) 3.6 (0.4) 3.4 (0.4) 3.9 (0.3) 2.6 (1.0) 3.6 (0.9) 3.3 (0.8) 3.1 (0.7) .041 .004 n.s. .000 .002 .000 .000 .001 n.s. n.s. .034 n.s. n.s. n.s. n.s. n.s. ICU, intensive care unit; n.s., not significant; PPQ, Perceived Persuasiveness Questionnaire; SAQ, Safety Attitudes Questionnaire; WEQ, Website Usability Questionnaire. *Mean scores are presented per scale. All individual items were scored on a 5-point Likert scale, where higher scores indicate better findings (or an improvement) on that domain. of effects was found in all domains, except for persuasiveness and the 4 ASP-behavior questions. Independent samples t tests showed a significant difference between the repeated and single participants in their baseline mean scores on speed and the ASP item suggestions (speed: 3.00 and 2.20, P = .030; suggestions: 2.33 and 3.00, P = .043, respectively). immediately after reading the scenario, whereas this did happen at baseline (21.3%) especially, and also in the control condition (5.9%). Figure 2 further shows that, at baseline, nurses provided a correct solution based on incorrect (or invalid) information (10.6%) more often than in the control (5.9%) or intervention conditions (2.2%). On average, nurses were faster in resolving their scenarios when using the app. This is especially true when they did not find the correct information and gave a wrong answer, gave a right answer based on the wrong information, or used the right information to reach a right answer (83.1, 91.0, and 74.0 seconds, respectively) (Fig 3). During baseline especially, wrong answers, involvement of external help after search, and right answers after wrong information was found made the search lengthy (241.0, 290.0, and 331.0 seconds, respectively). Over all conditions, scenarios with fairly common tasks or dilemmas were most often answered correctly. Scenario 1 and 2 on dosage, scenario 3 on preparation, and scenario 5 on administration task instructions were answered correctly in 65%, 75%, 73%, and 73% of the cases, respectively. Scenarios 4, 6, 7, and 9 on interaction and compatibility of drugs and renal failure triggered searches, but eventually resulted in calling in for external help in 29%, 50%, 65%, and 31% of the cases, respectively. Related to this solution category are the solutions where the participant gives up after a search (and does not suggest that external help is needed), which occurred most often in scenario 9 (resolved in 23% by giving up). Scenario tests The scenario-based tests demonstrate that with the app, nurses solved the scenarios faster; on average they needed 112.6 seconds with, and 180.0 seconds without, the app to reach a conclusion (P < .001). In all conditions, participants reached a correct answer more often than any other possible outcome; 34.0%, 51.0%, and 43.5% of all scenarios at, respectively, baseline, control, and intervention were resolved correctly (Fig 2). The second most prevalent scenario outcome was performing a search and calling in for external help. The control and intervention conditions differed little, with 19.6% and 21.7% of the scenarios resolved this way, respectively. During baseline, nurses were a bit more prone to involve someone after a search, as was found in 27.7% of the scenarios. In the intervention condition, more wrong answers (15.2%) and nurses giving up after trying to search (10.9%) were found than in the control or baseline conditions. However, not one nurse in the intervention condition gave up baseline control intervention % of all solutions 60 50 40 43.5 34.0 27.7 30 19.6 21.7 21.3 15.2 20 10 51.0 4.3 10.9 5.9 7.8 0.0 5.9 2.1 0.0 3.9 6.5 10.6 5.9 2.2 0 Wrong answer Cannot find, Give up Call for Right info Wrong info Right info, give up instantly, call external help found, wrong found, right right answer for help after search answer answer Solution Fig 2. Scenario solutions per condition. ARTICLE IN PRESS 6 J. Wentzel et al. / American Journal of Infection Control ■■ (2016) ■■-■■ 350 300 250 241.0 231.0 195.7 200 150 100 baseline control application 331.0 290.0 205.2 181.0 157.6 196.7 159.7 147.6 121.0 119.8 91.0 87.0 83.1 74.0 50 Seconds 0 Wrong answer Cannot find, give up Give up Call for Right info instantly, call external help found, wrong for help after search answer Wrong info Right info, right found, right answer answer Solution Fig 3. Time needed to solve per scenario outcome. Solution category 3 was not calculated because participants instantly gave up and called for help. 9. Dose adjustments for renal insufficiency 8. Recognize possible interaction-related complaints 7. Interaction (suspician triggered by renal values) 6. Compatibility of two IV medications 5. Check how fast to adminster a med 4. Check interaction between two meds 3. Duration/preparation augmentin 2. Check standard dosage 1. Check dose appropriateness for weight 0 20 40 60 80 100 1 Wrong answer 2 Cannot find, give up 3 Give up instantly, call for help 4 Call for external help after search 5 Right info found, wrong answer 6 Wrong info found, right answer 7 Right info, right answer Fig 4. Solutions per scenario. IV, intravenous. Scenarios 7 and 9 also triggered an immediate call for help, without searching (18% and 23%, respectively). Wrong answers were provided most often in scenario 6 (19%) on intravenous compatibility and scenario 8 (28%) on drug-drug interactions (Fig 4). With the app, nurses seemed to conclude that they could not find information and gave up or called in for help a bit more often than in general (Figs 4 and 5). This is especially the case in the scenarios that are at the border of nurse responsibility and expertise (renal insufficiency, drug-drug interactions; scenarios 4, 6, and 7). The more straightforward and very familiar tasks of checking how fast to administer a drug and checking the standard dose (scenarios 2 and 5) were performed well in general and perfectly with the app (Figs 4 and 5). DISCUSSION The risks that antibiotic resistance poses need to be countered, and ASPs offer bundles of strategies that contribute to limiting this problem in clinical settings. Because nurses are an often overlooked stakeholder in ASPs, we set out to support their antibioticrelated tasks by developing an information app that incorporates PSD strategies.8 The results of this study show that the app contributed to the overall information situation of nurses regarding aspects of perceived persuasiveness (reliability and unobtrusiveness) and usability/user satisfaction (relevance, user friendliness, speed, and hyperlinks). No new information was supplied via the app (it merely centralizes readily available information). Nurses used Solution per scenario (app) 9. Dose adjustments for renal insufficiency 8. Recognize possible interaction-related complaints 7. Interaction (suspician triggered by renal values) 6. Compatibility of two IV medications 5. Check how fast to adminster a med 4. Check interaction between two meds 3. Duration/preparation augmentin 2. Check standard dosage 1. Check dose appropriateness for weight 1 Wrong answer 4 Call for external help after search 7 Right info, right answer 0 20 40 2 Cannot find, give up 5 Right info found, wrong answer 60 80 100 3 Give up instantly, call for help 6 Wrong info found, right answer Fig 5. Solutions per scenario with the application. IV, intravenous. ARTICLE IN PRESS J. Wentzel et al. / American Journal of Infection Control ■■ (2016) ■■-■■ the app while they were free to do so, and it offered no new information.23 Also, we observed improvements in the information situation. Therefore, we conclude that the mode of offering information is an important contributor to user satisfaction. This positive user experience translates into the time nurses need to seek information to reach a conclusion in the scenario tests: nurses solved scenarios faster with the app than without it. However, the quality of the solutions they reach with and without the app is diverse and not in all cases better with the app than without it. Paradoxically, easier access to information may improve awareness, but it does not necessarily support the ability to process the information, leaving nurses incapable of resolving the scenario. The improvement of the general information situation of the nurses is in line with the previous analysis of app use (steady throughout the pilot period) and positive evaluations of the app regarding usability and user satisfaction. 23,24 The app is valued positively, used steadily, and according to our results, contributes to the overall information support of the nurses. However, no concrete (self-reported) effects on communication, safety attitudes, and antibiotic stewardship behavior were observed. It is possible that the app was mainly used to perform antibiotic-related tasks that are within the nurses’ traditional domain (preparing antibiotics for use and administering them) and less to educate or emancipate them regarding more complex tasks, such as recognizing suboptimal treatment and alerting physicians or pharmacists in case of severe side effects, possible interactions, or other adverse effects that may occur because of suboptimal treatment. This explanation is supported by an analysis of the log data of the app.23 The log data show that mainly pages with instructions for preparation and administration of antibiotics were accessed, hinting again at task support as the main persuasive strategy that comes forward in this app. Congruous with the absence of any improvement in communication, safety attitudes, and antibiotic behaviors, the scenariobased user tests show diffuse findings regarding the quality of solutions for the scenarios, even though the app did enable the nurses to reach their answer faster. However, nurses often indicated that they were unable to find or correctly process the information to be able to draw a conclusion (and therefore call in external help), or they drew a wrong conclusion. Therefore, it seems that more basic knowledge of antibiotic stewardship and information intended for nurses is needed to be able to make correct decisions regarding antibiotics. Acquiring this information and knowledge was not, or insufficiently, supported by the app. As addressed in other publications,30 nurses must first recognize the problem and their need for information (uncertainty). Also, they need to know where to look and what type of information may be relevant in a certain case (eg, to find out whether it is a regular side effect that causes the problem or a drug-drug interaction that is at play). Focusing on solid information systems alone is insufficient in such contexts30; when awareness and empowerment are lacking and information searching does not run smoothly, nurses in our study indicate they would stop and call someone. This is in line with other research,31 illustrating a preference of nurses for human information sources regarding drug information, even though availability and being evidence based are not always ensured. Therefore, to support nurses in ASPs, a combination of (usable) information resources and adequate training and ready knowledge regarding antibiotics should be in place. In particular, the ready knowledge and monitoring skills in ASPs require explicit training. In addition, other persuasive strategies besides better accessible information and task support are needed; the results show that scenarios that required more than locating and carrying out concrete instructions (eg, interpreting information on side effects or interactions, as opposed to preparation instructions) were most difficult to solve. The app may have increased awareness among nurses regarding the more 7 complex tasks, causing them to conclude more quickly that they are not an actor or responsible (and calling in help). In this sense, the technology does trigger awareness and uncovered a new focus for nurse education. Nurses strongly stating that a certain scenario was irrelevant to their task domain was an unforeseen finding in our study. However, it informs us about nurses’ views on their role in ASPs. Some of the scenarios were intentionally created to be at the intersection of nurse and physician responsibility. For example, recognizing suboptimal treatment circumstances and being able to detect possible suboptimal treatment situations do not fall directly under the nurses’ responsibility, but taking up such tasks can be an added value of (often present) nurses in ASPs. Often, nurses in our study instantly indicated that they played no role in the scenario that was presented to them. Subsequently, they were unable to retrieve the information needed or interpret it correctly. Therefore, our results show that supplying information in itself does not support the emancipation or an active role of nurses in ASP, but it does create awareness. Active learning strategies have been proposed as a necessary complementation to create ownership and increase knowledge among nurses in general31 and in ASPs.17 In our study we suspect that the residents in this teaching hospital, who were often present on the wards, may account to some extent for the situation where nurses can easily call in for help. It is possible that in hospitals where physicians or pharmacists are less easily accessible, nurses may be more inclined to start searching and be more persistent in continuing their search until sufficient information is found because there is no (quick) alternative. Limitations and future research This study has some limitations that compromise the robustness of our results. The duration of the pilot (8 months) inevitably means that changes in the population occurred. Also, the amount of nurses working on these wards, the partially independent samples, and the questionnaire nonresponse have limited our ability to draw firm conclusions. However, we were able to provide a proof-ofconcept and identify issues that need to be resolved to improve ASPs with nurse involvement. In addition, we were mainly interested in overall (ward-level) effects rather than individual (within-subject) effects. This focus on ward-level outcomes stems from the understanding that in highly cooperative jobs, such as nursing, where colleagues are easily asked for assistance or help, effects of an intervention on one person will probably not be limited to that person but may spread among colleagues. Following this rationale, for the implementation process, the strategy of link nurses is often used, where the (trained) nurse functions as a linking hub to spread the intervention-change.32 Aside from individual changes and effects, the overall pre- and postquestionnaire results may have suffered from external influences. One example is influence from visits from the Dutch health care inspectorate to the pilot hospital to address (among other things) medication safety.33 These inspectorate activities could have triggered some nurse awareness on antibiotic stewardship and guideline adherence, which may have caused nurses to be more critical in evaluating their information sources. Likewise, (national) media attention for antimicrobial resistance may have triggered increased nurse involvement and awareness over time. These external influences may have caused the nurses to apply a more critical view onto their ward, therefore not acknowledging possible (subtle) improvements in ASP behavior, but rather stressing the imperfect state of ASP behavior. In the ASP behavior items in particular, we observed great variety in the responses. These results imply that nurses therefore do not agree much on the state of ASP behaviors on their ward. After completion of the scenario tests, many nurses ARTICLE IN PRESS 8 J. Wentzel et al. / American Journal of Infection Control ■■ (2016) ■■-■■ (informally) made remarks about the appropriateness of their active role in ASPs. Some nurses felt that it is an important but currently unsupported aspect of nursing, whereas others felt that the physician should be in charge of these types of decisions first and foremost. Finally, this pilot study in this specific setting may suffer from generalizability issues concerning the effectiveness of information centralizing apps. Other studies do show similar evidence regarding these kinds of information integration concepts.5 In addition, this study makes a strong case for a tailored and humancentered approach to develop information systems. The outcomes may not be fully generalizable, whereas the approach is. Just as importantly, the feasibility to perform such approaches on a larger scale remains a challenge.34 For example, the extensive research into the user needs and subsequent tailoring of information was done manually in this project,21 but on a larger scale, automated ways are probably required to do this. In future research, the implementation of information apps combined with educational activities can be carried out and studied on a larger scale and as part of an ASP.35 This way, besides changes in behavior, which are a prerequisite for any other observable effects, changes in antibiotic stewardship processes (antibiotic expenditure and resistance patterns) and quality of care (eg, length of stay, mortality, complications) can be assessed. CONCLUSIONS User-friendly, task-supporting information that is integrated into 1 app does enable nurses to perform their tasks more efficiently (faster). However, empowerment of nurses in antibiotic stewardship needs a more active approach and cannot rely on subtle PSD strategies, such as task support and unobtrusiveness. To benefit from nurse input in ASPs, nurses need to be made aware of their roles and responsibilities in ASPs, via education or training, possibly applying social strategies as well. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. Acknowledgments We thank the various participants in the pilot study for their cooperation in this research: the nurses and chest physicians of the A4/C4 Department of the Medisch Spectrum Twente Hospital in Enschede, The Netherlands. We also thank Marjolein Hartgerink for her contributions to data analysis. 25. 26. 27. References 1. Benner P, Sheets V, Uris P, Malloch K, Schwed K, Jamison D. Individual, practice, and system causes of errors in nursing: a taxonomy. J Nurs Adm 2002;32:509-23. 2. Higuchi K, Donald JG. Thinking processes used by nurses in clinical decision making. J Nurs Educ 2002;41:145-53. 3. Potter P, Wolf L, Boxerman S, Grayson D, Sledge J, Dunagan C, et al. Understanding the cognitive work of nursing in the acute care environment. J Nurs Adm 2005;35:327-35. 4. While A, Dewsbury G. Nursing and information and communication technology (ICT): a discussion of trends and future directions. Int J Nurs Stud 2011;48:130210. 5. Koch SH, Weir C, Haar M, Staggers N, Agutter J, Görges M, et al. Intensive care unit nurses’ information needs and recommendations for integrated displays to improve nurses’ situation awareness. J Am Med Inform Assoc 2012;19:583-90. 6. Potter P, Wolf L, Boxerman S, Grayson D, Sledge J, Dunagan C, et al. An analysis of nurses’ cognitive work: a new perspective for understanding medical errors. In: Henriksen K, Battles JB, Marks ES, Lewin DI, Potter P, Wolf L, et al, editors. Advances in patient safety: from research to implementation (volume 1: research findings). Rockville (MD): Agency for Healthcare Research and Quality; 2005:3951. 7. Fogg B. Persuasive technology: using computers to change what we think and do. San Francisco (CA): Morgan Kaufmann; 2003. 8. Oinas-Kukkonen H, Harjumaa M. Persuasive systems design: key issues, process model, and system features. Commun Assoc Inf Syst 2009;24:485-500. 9. Beerlage-de Jong N, Eikelenboom-Boskamp A, Voss A, Sanderman R, van Gemert-Pijnen J. Combining user-centered design with the persuasive 28. 29. 30. 31. 32. 33. 34. 35. systems design model; the development process of a web-based registration and monitoring system for healthcare-associated infections in nursing homes. Int J Adv Life Sci 2014;6:262-71. MacDougall C, Polk R. Antimicrobial stewardship programs in health care systems. Clin Microbiol Rev 2005;18:638. Alanis AJ. Resistance to antibiotics: are we in the post-antibiotic era? Arch Med Res 2005;36:697-705. Shlaes DM, Gerding DN, John JF, Craig WA, Bornstein DL, Duncan RA, et al. Society for Healthcare Epidemiology of America and Infectious Diseases Society of America Joint Committee on the Prevention of Antimicrobial Resistance: guidelines for the prevention of antimicrobial resistance in hospitals. Clin Infect Dis 1997;25:584-99. Pollack LA, Srinivasan A. Core elements of hospital antibiotic stewardship programs from the Centers for Disease Control and Prevention. Clin Infect Dis 2014;59(Suppl):S97-100. Dellit TH, Owens RC, McGowan JE, Gerding DN, Weinstein RA, Burke JP, et al. Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America guidelines for developing an institutional program to enhance antimicrobial stewardship. Clin Infect Dis 2007;44:159-77. Davey P, Brown E, Charani E, Fenelon L, Gould IM, Holmes A, et al. Interventions to improve antibiotic prescribing practices for hospital inpatients. Cochrane Database Syst Rev 2013;(4):CD003543. Edwards R, Drumright L, Kiernan M, Holmes A. Covering more territory to fight resistance: considering nurses’ role in antimicrobial stewardship. J Infect Prev 2011;12:6-10. Gillespie E, Rodrigues A, Wright L, Williams N, Stuart RL. Improving antibiotic stewardship by involving nurses. Am J Infect Control 2013;41:365-7. Smith MA, McInnis LA. Antimicrobial resistance in critical care. Crit Care Nurs Clin North Am 2007;19:53-60. Charani E, Castro-Sanchez E, Sevdalis N, Kyratsis Y, Drumright L, Shah N, et al. Understanding the determinants of antimicrobial prescribing within hospitals: the role of “prescribing etiquette”. Clin Infect Dis 2013;57:188-96. Nicolle LE. Antimicrobial stewardship in long term care facilities: what is effective. Antimicrob Resist Infect Control 2014;3:6. Wentzel J, van Velsen L, van Limburg M, de Jong N, Karreman J, Hendrix R, et al. Participatory eHealth development to support nurses in antimicrobial stewardship. BMC Med Inform Decis Mak 2014;14:45. de Jong N, Wentzel MJ, Kelders SM, Oinas-Kukkonen H, van Gemert-Pijnen JEWC. Evaluation of perceived persuasiveness constructs by combining user tests and expert assessments. In: Second International Workshop on Behavior Change Support Systems, Persuasive 2014, 22-05-2014. Padua, Italy: 2014. p. 715. Wentzel J, de Jong N, Nijdam L, van Drie-Pierik R, van Gemert-Pijnen J. Understanding eHealth use from a persuasive system design perspective. Int J Adv Life Sci 2014;6:210-9. Wentzel MJ, Gemert-Pijnen JEWC. Antibiotic Information App for Nurses. In: van Gemert-Pijnen JEWC, Hettinga M, Smedberg A, editors. TELEMED 2014, The Sixth International Conference on eHealth, Telemedicine, and Social Medicine, March 23-27, 2014. Barcelona, Spain: IARIA/Thinkmind; 2014. p. 174-9. Drozd F, Lehto T, Oinas-Kukkonen H. Exploring perceived persuasiveness of a behavior change support system: a structural model. In: Bang M, Ragnemalm E, editors. Persuasive technology design for health and safety. Berlin, Germany: Springer; 2012. p. 157-68. Lehto T, Oinas-Kukkonen H, Drozd F. Factors affecting perceived persuasiveness of a behavior change support system. ICIS 2012 Proceedings, 16-19 December 2012, Orlando, Florida. Elling S, Lentz L, De Jong M. Website evaluation questionnaire: development of a research-based tool for evaluating informational websites. In: Wimmer MA, Scholl J, Grönlund A, editors. Proceedings of Electronic Government: 6th International Conference; 2007 Sept 3-7; Regensburg, Germany. Berlin, Germany: Springer; 2007. p. 293-304. Shortell SM, Rousseau DM, Gillies RR, Devers KJ, Simons TL. Organizational assessment in intensive care units (ICUs): construct development, reliability, and validity of the ICU nurse-physician questionnaire. Med Care 1991;29:70926. Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, et al. The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res 2006;6:44. French B. Uncertainty and information need in nursing. Nurse Educ Today 2006;26:245-52. Ndosi M, Newell R. Medicine information sources used by nurses at the point of care. J Clin Nurs 2010;19:2659-61. Sopirala MM, Yahle-Dunbar L, Smyer J, Wellington L, Dickman J, Zikri N, et al. Infection Control Link Nurse Program: an interdisciplinary approach in targeting health care-acquired infection. Am J Infect Control 2014;42:353-9. Dutch Healthcare Inspectorate. Brief: instellen verscherpt toezicht. Utrecht, the Netherlands: Inspectie voor de Gezondheidszorg (Dutch Healthcare Inspectorate); 2013. Peute LW, Driest KF, Marcilly R, Da Costa SB, Beuscart-Zephir M-C, Jaspers MW. A framework for reporting on human factor/usability studies of health information technologies. Stud Health Technol Inform 2013;194:54-60. Schotman N. Inventarisatie van een eLearningcursus over antimicrobiële middelen voor zorgprofessionals van het UMCG. Enschede, The Netherlands: University of Twente; 2015.