Implementation of EMRs, Nigeria
Implementation of EMRs, Nigeria
Implementation of EMRs, Nigeria
Original Paper
Taiwo Adedeji1*, BSc, MSc, MRES, PhD; Hamish Fraser2*, MBChB, MSc; Philip Scott3*, MSc, PhD
1
School of Computing, University of Portsmouth, Portsmouth, United Kingdom
2
Brown Center for Biomedical Informatics, Brown University, Providence, RI, United States
3
Institute of Management and Health, University of Wales Trinity Saint David, Carmarthen, United Kingdom
*
all authors contributed equally
Corresponding Author:
Taiwo Adedeji, BSc, MSc, MRES, PhD
School of Computing, University of Portsmouth
Buckingham Building, Lion Terrace
Portsmouth, PO1 3HE
United Kingdom
Phone: 44 2392846429
Email: taiwo.adedeji@port.ac.uk
Abstract
Background: Digital health has been a tool of transformation for the delivery of health care services globally. An electronic
health record (EHR) system can solve the bottleneck of paper documentation in health service delivery if it is successfully
implemented, but poor implementation can lead to a waste of resources. The study of EHR system implementation in low- and
middle-income countries (LMICs) is of particular interest to health stakeholders such as policy makers, funders, and care providers
because of the efficiencies and evidence base that could result from the appropriate evaluation of such systems.
Objective: We aimed to develop a theory of change (ToC) for the implementation of EHRs for maternal and child health care
delivery in LMICs. The ToC is an outcomes-based approach that starts with the long-term goals and works backward to the inputs
and mediating components required to achieve these goals for complex programs.
Methods: We used the ToC approach for the whole implementation’s life cycle to guide the pilot study and identify the
preconditions needed to realize the study’s long-term goal at Festac Primary Health Centre in Lagos, Nigeria. To evaluate the
maturity of the implementation, we adapted previously defined success factors to supplement the ToC approach.
Results: The initial ToC map showed that the long-term goal was an improved service delivery in primary care with the
introduction of EHRs. The revised ToC revealed that the long-term change was the improved maternal and child health care
delivery at Festac Primary Health Center using EHRs. We proposed a generic ToC map that implementers in LMICs can use to
introduce an optimized EHR system, with assumptions about sustainability and other relevant factors. The outcomes from the
critical success factors were sustainability: the sustained improvements included trained health care professionals, a change in
mindset from using paper systems toward digital health transformation, and using the project’s laptops to collect aggregate data
for the District Health Information System 2–based national health information management system; financial: we secured funding
to procure IT equipment, including servers, laptops, and networking, but the initial cost of implementation was high, and funds
mainly came from the funding partner; and organizational: the health professionals, especially the head of nursing and health
information officers, showed significant commitment to adopting the EHR system, but certain physicians and midwives were
unwilling to use the EHR system initially until they were persuaded or incentivized by the management.
Conclusions: This study shows that the ToC is a rewarding approach to framing dialogue with stakeholders and serves as a
framework for planning, evaluation, learning, and reflection. We hypothesized that any future health IT implementation in primary
care could adapt our ToC approach to their contexts with necessary modifications based on inherent characteristics.
KEYWORDS
theory of change; electronic health records; maternal and child health; primary health center; success criteria
manager, and findings from previous EHR implementations. • Wider benefits: generalizable pointers that can guide the
The ToC map illustrated the problems we were trying to solve, stakeholders with regard to the chances of implementing
the keystakeholders, assumptions, inputs, intervention, outputs, long-term change
measurable effects, and wider benefits of the implementation
The ToC approach is not immune to problems when used as an
to realize the long-term change [44]. We developed a revised
evaluation tool. Problems of theorizing, measurement, testing,
ToC map (Figure 2) to accommodate changes during and after
and interpretation are not unusual [27]. To ensure rigor and
the EHR implementation. These changes related to most of the
evaluate the maturity of the implementation, we adapted the
ToC components and are documented under each component
success criteria used in the studies by Deriel et al [12] and Fritz
subheading in the Results section. We recognize that
et al [47] to supplement the ToC approach. Textbox 1 outlines
implementers should pay attention to sociotechnical issues,
the categories considered for the success criteria of the
especially the interplay between patients’ realities and HCPs’
implementation and provides definitions for each category.
mental models and how these influence the EHR design and
are represented within the system [45,46]. We engaged the health practitioners and decision-makers at
Festac PHC in designing, implementing, and evaluating the
In the context of this study, the ToC components use these
EHR system. In particular, the health practitioners at Festac
definitions:
PHC joined in developing the ToC versions, especially providing
• Long-term change: the desired goal the stakeholders want practical experiences that shaped the theories underpinning the
to achieve ToC versions. This approach facilitated realistic interactions
• Problems: the challenges facing the current paper-based with the stakeholders and gave a proper understanding of the
health records workflow as highlighted by the stakeholders local context in which the study was conducted [48,49]. We
• Stakeholders: the people directly or indirectly involved or had stakeholder meetings involving the heads of department
affected by the success or failure of the EHR and EHR champions at the PHC at the start and during the
implementation implementation process. Each stakeholder discussed the issues
• Assumptions: the beliefs that specify the underlying reasons of the existing paper-based health record system and their
for the logical connections that exist among the ToC expectations and experiences of the new EHR system, which
elements. These beliefs are usually informed by research validated the findings of the first ToC map. Subsequently, health
evidence, clinical practice, and the environment in which informatics experts validated the revised ToC findings at the
the change is taking place. MedInfo 2019 conference in Lyon, France.
• Inputs: the activities or tasks carried out around the
We developed a generic version of the ToC map (Figure 3) to
intervention
reflect a holistic framework as a toolkit for relevant stakeholders
• Interventions: the initiatives or programs embarked on to
who want to embark on this kind of intervention in similar
influence the desired outcomes
contexts beyond Lagos, Nigeria. The stakeholders can adapt it
• Outputs: the tangibles resulting from the inputs and the
for EHR implementations in primary care settings but need to
intervention
pay close attention to inherent characteristics in these
• Measurable effects: the immediate indicators that can be
environments. Despite the nuances in different contexts, the
traced to the implementation process and are readily usable
process and steps involved in the creation of the ToC map are
for evaluation. These measures can be quantitative or
not to be ignored. Chen and Rossi [26] stressed the importance
qualitative.
of giving adequate attention to understanding the implementation
process and not being too concerned about whether the initiative
has yielded excellent results.
Figure 1. An initial version of the theory of change for the scheduled electronic health record (EHR) implementation at Festac Primary Health Centre
(PHC). ANC: antenatal care; FHIR: Fast Healthcare Interoperability Resources; GCRF: Global Challenges Research Fund; OpenMRS: Open Medical
Records System; UoP: University of Portsmouth.
Figure 2. A revised version of the theory of change for electronic health record (EHR) implementation at Festac Primary Health Centre (PHC), including
findings from a workshop at the MedInfo 2019 conference. ANC: antenatal care; CIEL: Columbia International eHealth Laboratory; GCRF: Global
Challenges Research Fund; MCH: maternal and child health care; OpenMRS: Open Medical Records System; UoP: University of Portsmouth.
Textbox 1. Categories for success criteria and their definitions for electronic health record implementation (adapted from Deriel et al [12] and Fritz et
al [47]).
Categories and definitions
• Ethics
• Regulatory and cultural issues such as health data security, privacy, and confidentiality
• Political
• Health policies and countrywide circumstances, including health care infrastructure, characteristics, ministries of health, and primary health
care boards
• Organizational
• Managerial circumstances within the organization itself, including human resources, skilled staff, or local buy-in; leadership and governance;
project management and commitment to implementation; and data use
• Financial
• Functionality
• System features and functions, including modules, data handling, forms, and reports
• Technical
• Infrastructure, software architecture, user interfaces, data standards, and privacy or security
• Training
• Skills training as well as computer literacy and educational background and user support
• Sustainability
• Transition from external stakeholder to local management across all categories, including financing
Figure 3. A generic version of the theory of change for electronic health record (EHR) implementation, without context-specific details. ANC: antenatal
care; MCH: maternal and child health care; PHC: primary health center.
EHR System Selection by completing a participant consent form. The study considered
Open Medical Records System (OpenMRS) is an EHR software the security, privacy, and confidentiality of patient records from
program built for low-resource settings to improve health care the outset. The paper health records were kept locked in a card
delivery with the help of a global community that supports the room at the PHC. Although the reception area is positioned
software [50]. We selected OpenMRS as the EHR application close to the card room, at busy times anyone could access the
for the pilot implementation because it is an open-source room with malicious intentions to cart away or damage the paper
program and therefore freely available, which fits into the records. Hence, the EHR implementation took into account
funding realities of LMICs, including Nigeria. The OpenMRS secure access to the electronic records by creating user accounts
software source code can be modified and tailored to the needs for relevant clinicians, ensuring that only users authorized by
of the particular context in which it is being used. It is an the heads of department could access the system [4].
enterprise platform with flexible modules that have matured
over time and been implemented in similar settings with a Results
vibrant web-based community of developers and implementers
Overview
[51,52]. We adapted existing OpenMRS modules to facilitate
the identified use cases such as patient registration, outpatient In this section, we report the complete ToC life cycle (Figure
clinic, laboratory, and mother and child clinic to manage clinical 4) for this study commencing from idea conception to the
workflows. Moreover, we adapted UgandaEMR’s ANC and development of the initial ToC map and revised ToC map,
immunization e-forms to save development time and initial illustrating how we accomplished the EHR implementation
user-testing requirements. tasks at Festac PHC. At the same time, we hypothesize that
program designers and relevant stakeholders can adapt the
Ethics Approval generic ToC map for EHR implementations in similar contexts.
This study obtained a favorable opinion from the University of Subsequently, we provide a detailed narrative of the long-term
Portsmouth Faculty of Technology ethics committee change and identified preconditions from the ToC process. From
(TECH2019-T.A-01). Participation in the study was voluntary, this process, we produced a summary of the key successes and
and participants were free to withdraw at any time without lessons learned alongside the study’s implications to evaluate
giving any reason. The participants provided written consent the process (Multimedia Appendix 1).
Figure 4. Complete theory of change (ToC) life cycle for electronic health record (EHR) implementation at Festac Primary Health Centre. M&E:
monitoring and evaluation; OpenMRS: Open Medical Records System.
• Practitioners
• Program designer
• Implementer and technicians possess the relevant skills for installing procured equipment
• Practitioners
• Staff give consent to participate in the electronic health record system evaluation
• Practitioners
• Program designer
• Questions relating to the electronic health record implementation are asked by workshop participants
• Practitioners
• Electronic health record system is designed with the relevant system attributes
• Hardware equipment and electronic health record system software remain intact and are maintained regularly
• Clinicians are making use of electronic health records regularly for delivering health services to patients
• Stakeholders are learning from electronic health record use and data
Intervention
Outputs
The main intervention for this study was the introduction of an
The study received a letter of approval for the implementation
EHR system in primary care MCH services. Initially, problems
from the local authority. This approval enabled the release of
were perceived based on explicit and implicit assumptions about
funds and the travel of a research team member (TA) to the
paper medical records and prescriptions and scheduling of
health facility. The funder released the funds to procure the IT
patient encounters being carried out manually. After face-to-face
equipment needed for the study. The interactions with the health
stakeholder meetings on site, the practitioners were of the
practitioners made it possible to obtain the requirements to
unanimous opinion that prescribing and scheduling inefficiencies
design and develop the intervention. After we incorporated the
were not the priority issues; rather, priority should be accorded
active inputs of various stakeholders, the EHR system was ready
to paper records handling, ANC and immunization-tracking
for use by the health practitioners. The outputs in the revised
inefficiencies, and missing patient records. These problems
ToC map were procured equipment, feedback from EHR use,
validated the introduction of the EHR intervention at Festac
optimized EHR system functionality, EHR use outcomes
PHC.
indicators for MCH, and data modeling for EHR system
interoperability. Other key outputs were the critical system Stakeholders
attributes (such as system stability, availability, and usability) The stakeholders are the research team (TA, PS, and HF),
and full and incremental data backup of patient records to the funder, Festac PHC management (local authority), primary
cloud. In the event of system damage, fire, flooding, or any health care board, health care practitioners, patients, and health
adverse incidents, the PHC can restore the records from the informatics experts. They carried out several activities at various
backup. stages of the study. The research team made some informal
contacts with the local primary care facility authorities to
understand their problems and the desired long-term outcomes.
The team reviewed existing studies to gain background extensively evaluated the ToC-based implementation using
knowledge of previous EHR implementations in similar previously defined success criteria across multiple dimensions
contexts. After the review, the team developed the initial version of implementation and use (Multimedia Appendix 1) [12,47],
of the ToC map, informed by the explicit assumptions of the which is a methodological innovation in LMIC settings because
practitioners and implicit assumptions gleaned from previous of the wide range of evaluation criteria. However, combinations
implementations. The research team prepared a funding of some individual criteria have been used. Certain authors have
application and sought approval for the pilot study. Both the argued that theory-based evaluation such as the ToC is more a
funder and the local health authority approved the pilot study. methodology than a theory because it uses different research
methods (eg, randomized controlled trials, interviews, and
Technical Implementation workshops) for its development [30,44]. Weiss [33] argues that
There was a demonstration of the OpenMRS software during the ToC is an approach and a theory because it is built on
the first stakeholder meeting. The activity helped the assumptions (beliefs), preconditions, inputs, and outputs, which
practitioners to have a feel of how the intervention works. Before influence the way people behave.
this meeting, the contact person from the PHC had been testing
the demonstration version of the EHR system; they gave Again, the ToC approach is particularly useful in capturing the
feedback on what the PHC specifically wanted. The main complexities of a program relating to its outcomes, outputs,
technical components of OpenMRS are the database (eg, data inputs, and activities to bring about long-term change by using
concepts mapping, backups, and security) and the EHR software relevant interventions [58]. The research team engaged the
(clinical modules and customizations). The research team relevant stakeholders by asking them to share their experiences
initially designed a cloud solution before the implementation and practices (explicit assumptions). We drew out the implicit
but changed to a local area network design because of poor assumptions, which were not obvious to the practitioners and
internet access at the health facility. Through a combination of experts, through interviews and a workshop (findings to be
on-site and remote support, the research team contributed to published), and then modeled these assumptions and combined
installing and configuring the software. The equipment included them with evidence and logic, all of which were put together
laptops, a desktop PC (dedicated server), networking equipment in readiness to transfer into practice.
(16-port Ethernet switch, wireless router, Category 6 cables, Reflections Based on Experiences of EHR
and RJ45 connectors), a power inverter (to provide power for Implementations in Other LMICs
the server when electricity from the national grid and generator
set is unavailable), and a printer. Despite Festac PHC being an early adopter of the EHR system
and the only one among public PHCs in Lagos State, the
management has not done enough in terms of funding the
Discussion infrastructure and ensuring its sustainability. The issue of
Principal Findings funding and other EHR implementation challenges are not
peculiar to the Nigerian context; rather, they are applicable to
This study shows the value of the ToC process for robust
different LMIC contexts [51,53]. Comparison evaluations of
planning, analysis, and evaluation of EHR implementation
EHR systems in LMICs were provided by 2 papers, published
complexities, as well as challenging the assumptions of all
in 2017 and 2018 (Multimedia Appendix 2 [51,53]). Although
stakeholders. The process requires logical reasoning, effectively
there is anecdotal evidence of EHR implementations across
engaging stakeholders in drawing implicit assumptions,
Nigeria, there is no known peer-reviewed evidence of OpenMRS
designing the preconditions, and mapping the ToC backward
implementation in the country. As of June 2021, the OpenMRS
from the long-term goal to inputs. Political factors play a role
HIV Reference Implementation initiative funded by the Centers
in influencing what practitioners say about their beliefs or
for Disease Control and Prevention is supporting >1000 site
theories regarding the desired change. The practitioners may
rollouts of OpenMRS in Nigeria as well as improvements in
have concerns about the management’s disapproval of their
the user interface, reporting, and other initiatives [59]. A recent
assumptions [33]; for example, we asked the HCPs about the
paper [60] tried to examine the impact of OpenMRS
leadership style of their line managers and the effect it has on
implementations globally over a 15-year period, but no concrete
their use of the EHR system. Some (7/14, 50%) of the HCPs
evidence on Nigeria was available, except for some brief
made positive comments about their managers. Although it is
mentions. This study should help to address this gap, especially
possible to have all-positive feedback about leadership styles
where public primary care in Nigeria is concerned.
in a typical work setting, the lack of concerns or negative
comments may suggest desirability bias or groupthink [57]. Multimedia Appendix 2 compares findings from OpenMRS
implementations in 3 LMICs (Nigeria, Sierra Leone, and
A ToC is useful in articulating assumptions made about a
Kenya), inclusive of this study (Festac PHC in Nigeria).
program or intervention to achieve its desired results. We
Common findings across the 3 studies related to data collection,
generated assumptions from peer-reviewed evidence (documents
staff training, and infrastructure. These studies showed that
and prior research); experience and views of practitioners and
EHR use results in clinical workflow efficiencies. At the same
other stakeholders such as funder, government, and policy
time, the studies discussed the challenges encountered during
makers; and logical reasoning (Textbox 2). However, it can be
implementation, which centered mainly on inadequate
problematic to test assumptions even when they are explicitly
infrastructure, funding, dedicated IT support, and stakeholder
stated. Problems such as measurement, generalization, and
buy-in. A significant issue across the 3 EHR implementations
validation usually plague program theory [27]. Our study
https://medinform.jmir.org/2022/8/e33491 JMIR Med Inform 2022 | vol. 10 | iss. 8 | e33491 | p. 11
(page number not for citation purposes)
XSL• FO
RenderX
JMIR MEDICAL INFORMATICS Adedeji et al
is sustainability, and our Nigerian (Festac PHC) study used the of casual staff who do not have permission to use it because of
ToC approach to underscore this issue extensively. Despite their clinical accountability requirements. Lack of leadership
successful completion, the implementations did not continue motivation or incentive to use the system could prevent health
beyond the first or second phase. Hence, stakeholders must pay information officers, physicians, nurses, and midwives from
close attention to sustainability issues before embarking on EHR understanding the need to work on data entry. System downtime
implementations in LMICs. happens occasionally; when this happens, there is no health IT
support technician on the ground to resolve the issue, and hence
Reflections Based on Experiences of EHR the PHC relies on the implementer, who, although not
Implementations in HICs contractually obliged, may sometimes help out. To resolve
Policy makers and politicians in LMICs can learn from countries system issues, the PHC management could employ an IT support
that incentivized EHR adoption by providing implementation technician on a full-time or part-time basis, but the management
funds to health facilities. A prime example is the United should be keen and be ready to include the employment cost in
Kingdom, where the EHR adoption rate in primary care, the clinic’s budget. In a recent review on the importance of
particularly general practitioner (GP) practices, is nearly 100% primary care records in LMICs, we found that there seems to
[61,62]. Among other factors, financial incentives from the be a particular challenge with EHR data collection in primary
government have proven to be an effective impetus for EHR care organizations [68]; for example, MCH EHR data collection
implementation across GP practices. For many years, thought was challenging because of local factors such as the level of
leaders in the GP profession have collaborated with the technology available for data entry at the point of childbirth.
government to provide incentives for digitizing practices and Hence, this is a larger problem for people who run modest
eliminating barriers. Hence, GPs were more willing to use EHRs primary care EHR systems in LMIC settings, a problem not
than hospital physicians, helping the former leverage the specific to Nigeria. This implementation study successfully
successful health IT intervention [62]. However, despite the demonstrated improvements in MCH services data collection.
successful EHR adoption rate by GP practices in the United However, the lack of effective human, organizational, and
Kingdom, the system has its shortcomings: it sometimes fails system support is responsible for inconsistent data entry in the
as patients show up at the community pharmacy expecting to EHR system, leading to poor clinical benefits and inaccurate
pick up their medications only to find that the electronic reporting.
prescription has not reflected in the pharmacy system. This issue
The ToC approach gave insights into the potential causes of the
can often delay treatment for patients, especially on weekends
breakdown of the system, such as the issues concerning regular
when GP practices are closed, and the pharmacy team chases
use and data entry by key staff, which allowed for provision of
prescriptions. The GP’s on-call team can usually access the
additional planning and training. A simple cost-benefit approach
system and fax the prescriptions to the pharmacy, but the
to framing the overall implementation process to determine the
effectiveness of this process varies across the United Kingdom.
likely gains (value) to staff, patients, health systems, and funders
The US government program based upon the Health Information would be helpful. It would be valuable to determine whether
Technology for Economic and Clinical Health Act of 2009 these costs outweigh the challenges of learning to use the system
provides financial incentives to physician practices and hospitals and the pain of working on data entry. In addition, the proposed
to foster digital health implementation and improve the quality investment in infrastructure and support could be balanced by
of care for patients. These incentives have since led to the the concrete benefits. The costs often fall on staff working on
widespread adoption and meaningful use of EHR systems across data entry who do not benefit much from the outputs. Hence,
all levels of health care in the United States, with the resultant the combined effect of the utility of an application and ease of
digital health transformation and improved clinical outcomes use gives stronger predictability for actual use, which is
[63-65]. However, rapid implementation of existing EHR incorporated in the D&M model.
systems has been associated with many challenges in workflow,
There is a growing interest in alternative data entry approaches,
usability and physician stress or overload. The UK model of
including the “scribe” model (in US primary care) [69], natural
adoption of primary care EHR systems may be better in terms
language processing–enabled data capture, and optical mark
of a limited number of carefully vetted systems, low costs, and
recognition (OMR). These alternative approaches could address
robust interoperability with many hospitals; for example, in
the issue of clinicians’ avoidance of using the EHR system. The
West Yorkshire [66].
“scribe” model introduces a way of working where a human
Reflections on Data Entry at Festac PHC scribe (a volunteer or health professional) manually enters the
Inconsistencies in EHR data entry during patient encounters applicable information such as observations, diagnosis, and test
occur because of several factors, including human, results into the EHR during the patient visit as spoken aloud by
organizational, and system factors. The willingness of clinical the physician or nurse [70]. However, this could affect clinical
staff to use the new system was lacking because of the data quality because the scribe might not be a suitably qualified
perception that the system would add to their existing workload, clinician and prone to making data entry errors, which, in turn,
reflecting the realities of data entry operations and the shortage could affect health outcomes. Natural language processing data
of health workers in LMICs [67]. Only a few HCPs were keen capture applications allow HCPs, especially physicians, to
on using the system. Hence, little or no data entry is completed capture structured data with unstructured dictation into the EHR
if the active HCPs are not on duty. Sometimes, the HCPs attend [71]. OMR is a nondictation, scanning method of data capture
staff verification exercises, leaving the EHR system in the hands where the OMR software processes paper clinical forms that
have been scanned with a modest office scanner or low-cost framework for planning an EHR implementation and the steps
document camera [72]. This approach ensures that clinicians needed to define the requirements and success factors, likelihood
who record clinical data on paper do not also have to enter the of longer-term success, and evaluation metrics. For new
data once or twice in other records. It requires stability of implementers, knowing how to structure this implementation
systems, a person to oversee the scanning and data extraction, process could be very useful. Future health IT implementation
and user confidence. It might develop as a model to overcome in primary care can adapt the ToC approach to their contexts
a data entry backlog in the EHR system, increasing the value with necessary modifications based on inherent characteristics.
for clinicians, particularly if recent improvements in optical The pilot EHR implementation served as a small-scale
character recognition software can be shown to be effective in foundation that can support health information exchange and
interpreting structured handwriting. as a digital health exemplar for other PHCs in Lagos State and
Nigeria. Other health care providers can learn from, and build
Limitations on, the implementation to support the delivery of MCH and
This study includes several limitations with regard to developing other health services. Furthermore, the pilot EHR system
the ToC. First, the research team was extensively involved in represented a digital enabler that provides computable and
developing and revising the ToC map, which may have machine-readable health data, the necessary first step toward
contributed to a social desirability bias. Second, the first author more complex aspects such as interoperability, clinical decision
(TA) mainly worked on the analysis of the ToC maps under the support, and a learning health system. Further work is needed
guidance of the last author (PS) and the second author (HF). to extend the scope of the implementation to cover other public
We would have engaged the HCPs and stakeholders in the PHCs. There is a need to secure more funds for additional
analysis, but they were not well versed with the technicalities infrastructure alongside solid leadership to ensure sustainability
of the ToC approach. Future studies will ensure that HCPs are and scalability. In addition, it will be helpful to explore the
familiarized with the ToC analysis. The relevant stakeholders interoperability of health data across public PHCs by designing
were fully engaged in the clinical, data collection (interviews a national health data model for an MCH services data set. The
and observations), and managerial aspects of the design. model should be based on established data standards and an
examination of the preconditions and drivers for implementing
Conclusions
such a model and build on existing work on clinical decision
This research presented the ToC as a rewarding approach to support for MCH services [73].
framing dialogue with stakeholders. It functioned as a valuable
Acknowledgments
The authors thank the management and staff of Festac Primary Health Centre for granting approval to, and supporting, the study.
The pilot study was funded by the Global Challenges Research Fund allocation to the University of Portsmouth.
Authors' Contributions
TA and PS conceived the study. TA drafted the manuscript. All authors revised and approved the final manuscript.
Conflicts of Interest
HF is a cofounder of the Open Medical Records System open-source software project that developed the software used in this
study; he contributed to the drafting and revision process.
Multimedia Appendix 1
Summary of successes achieved and lessons learned from the pilot study at Festac Primary Health Centre, as well as implications
for electronic health record implementations in Nigeria and other low- and middle-income countries.
[DOCX File , 21 KB-Multimedia Appendix 1]
Multimedia Appendix 2
A comparison of electronic health record implementation findings from 3 studies conducted in low- and middle-income countries.
[DOCX File , 21 KB-Multimedia Appendix 2]
References
1. Rumball-Smith J, Ross K, Bates DW. Late adopters of the electronic health record should move now. BMJ Qual Saf 2020
Mar;29(3):238-240 [FREE Full text] [doi: 10.1136/bmjqs-2019-010002] [Medline: 31732701]
2. Odekunle FF, Odekunle RO, Shankar S. Why sub-Saharan Africa lags in electronic health record adoption and possible
strategies to increase its adoption in this region. Int J Health Sci (Qassim) 2017;11(4):59-64 [FREE Full text] [Medline:
29085270]
3. Attah A. Implementing an electronic health record in a Nigerian secondary healthcare facility. Prospects and challenges.
UiT Norges arktiske universitet. 2017. URL: https://munin.uit.no/handle/10037/12245 [accessed 2020-03-07]
https://medinform.jmir.org/2022/8/e33491 JMIR Med Inform 2022 | vol. 10 | iss. 8 | e33491 | p. 13
(page number not for citation purposes)
XSL• FO
RenderX
JMIR MEDICAL INFORMATICS Adedeji et al
4. Adamu J, Hamzah R, Rosli MM. Security issues and framework of electronic medical record: a review. Bulletin EEI 2020
Apr 01;9(2):565-572. [doi: 10.11591/eei.v9i2.2064]
5. Nguyen L, Bellucci E, Nguyen LT. Electronic health records implementation: an evaluation of information system impact
and contingency factors. Int J Med Inform 2014 Nov;83(11):779-796. [doi: 10.1016/j.ijmedinf.2014.06.011] [Medline:
25085286]
6. Waterson P. Health information technology and sociotechnical systems: a progress report on recent developments within
the UK National Health Service (NHS). Appl Ergon 2014 Mar;45(2):150-161. [doi: 10.1016/j.apergo.2013.07.004] [Medline:
23895916]
7. Ebenso B, Allsop MJ, Okusanya B, Akaba G, Tukur J, Okunade K, et al. Impact of using eHealth tools to extend health
services to rural areas of Nigeria: protocol for a mixed-method, non-randomised cluster trial. BMJ Open 2018 Oct
18;8(10):e022174 [FREE Full text] [doi: 10.1136/bmjopen-2018-022174] [Medline: 30341123]
8. Reis ZS, Maia TA, Marcolino MS, Becerra-Posada F, Novillo-Ortiz D, Ribeiro AL. Is there evidence of cost benefits of
electronic medical records, standards, or interoperability in hospital information systems? Overview of systematic reviews.
JMIR Med Inform 2017 Aug 29;5(3):e26 [FREE Full text] [doi: 10.2196/medinform.7400] [Medline: 28851681]
9. Oluoch T, de Keizer NF. Evaluation of health IT in low-income countries. Stud Health Technol Inform 2016;222:324-335.
[Medline: 27198114]
10. Culture of quality and safety: a prerequisite for any informatics intervention. In: Global Health Informatics: Principles of
eHealth and mHealth to Improve Quality of Care. Cambridge, Massachusetts, United States: The MIT Press; 2017.
11. What if a smart phone could save a life? Virtual Doctors. URL: https://www.virtualdoctors.org/ [accessed 2020-03-07]
12. deRiel E, Puttkammer N, Hyppolite N, Diallo J, Wagner S, Honoré JG, et al. Success factors for implementing and sustaining
a mature electronic medical record in a low-resource setting: a case study of iSanté in Haiti. Health Policy Plan 2018 Mar
01;33(2):237-246. [doi: 10.1093/heapol/czx171] [Medline: 29253138]
13. Mirzoev T, Etiaba E, Ebenso B, Uzochukwu B, Manzano A, Onwujekwe O, et al. Study protocol: realist evaluation of
effectiveness and sustainability of a community health workers programme in improving maternal and child health in
Nigeria. Implement Sci 2016 Jun 07;11(1):83 [FREE Full text] [doi: 10.1186/s13012-016-0443-1] [Medline: 27268006]
14. Evans RS. Electronic health records: then, now, and in the future. Yearb Med Inform 2018 Mar 06;25(S 01):S48-S61. [doi:
10.15265/iys-2016-s006]
15. Lee SH, Nurmatov UB, Nwaru BI, Mukherjee M, Grant L, Pagliari C. Effectiveness of mHealth interventions for maternal,
newborn and child health in low- and middle-income countries: systematic review and meta-analysis. J Glob Health 2016
Jun;6(1):010401 [FREE Full text] [doi: 10.7189/jogh.06.010401] [Medline: 26649177]
16. Eze E, Gleasure R, Heavin C. Reviewing mHealth in developing countries: a stakeholder perspective. Procedia Comput
Sci 2016;100:1024-1032. [doi: 10.1016/j.procs.2016.09.276]
17. Were MC, Nyandiko WM, Huang KT, Slaven JE, Shen C, Tierney WM, et al. Computer-generated reminders and quality
of pediatric HIV care in a resource-limited setting. Pediatrics 2013 Mar;131(3):e789-e796 [FREE Full text] [doi:
10.1542/peds.2012-2072] [Medline: 23439898]
18. Oluoch T, Katana A, Kwaro D, Santas X, Langat P, Mwalili S, et al. Effect of a clinical decision support system on early
action on immunological treatment failure in patients with HIV in Kenya: a cluster randomised controlled trial. Lancet HIV
2016 Feb;3(2):e76-e84. [doi: 10.1016/s2352-3018(15)00242-8]
19. Amoroso C, Akimana B, Wise B, Fraser H. Using electronic medical records for HIV care in rural Rwanda. In: Studies in
Health Technology and Informatics. Amsterdam, Netherlands: IOS Press; 2010.
20. Douglas GP, Gadabu OJ, Joukes S, Mumba S, McKay MV, Ben-Smith A, et al. Using touchscreen electronic medical
record systems to support and monitor national scale-up of antiretroviral therapy in Malawi. PLoS Med 2010 Aug
10;7(8):e1000319 [FREE Full text] [doi: 10.1371/journal.pmed.1000319] [Medline: 20711476]
21. Fraser H, Habib A, Goodrich M, Thomas D, Blaya J, Fils-Aime J, et al. E-health systems for management of MDR-TB in
resource-poor environments: a decade of experience and recommendations for future work. Stud Health Technol Inform
2013;192:627-631. [Medline: 23920632]
22. National health ICT strategic framework 2015-2020. Federal Ministry of Health. URL: https://www.health.gov.ng/doc/
HealthICTStrategicFramework.pdf [accessed 2020-05-27]
23. Implementing the basic health care provision fund in Nigeria: a framework for accountability and good governance internet.
Resilient & Responsive Health Systems. URL: https://resyst.lshtm.ac.uk/resources/
implementing-the-basic-health-care-provision-fund-in-nigeria-a-framework-for [accessed 2022-07-14]
24. Mackey TK, Vian T, Kohler J. The sustainable development goals as a framework to combat health-sector corruption. Bull
World Health Organ 2018 Jun 04;96(9):634-643. [doi: 10.2471/blt.18.209502]
25. Olaronke I, Ishaya G, Rhoda I, Janet O. Interoperability in Nigeria healthcare system: the ways forward. Int J Inf Eng
Electron Bus 2013 Oct 01;5(4):16-23. [doi: 10.5815/ijieeb.2013.04.03]
26. Chen H, Rossi PH. Evaluating with sense. Eval Rev 2016 Jul 26;7(3):283-302. [doi: 10.1177/0193841x8300700301]
27. Weiss CH, Connell JP. Nothing as practical as good theory: exploring theory-based evaluation for comprehensive community
initiatives for children and families. In: New Approaches to Evaluating Community Initiatives: Concepts, Methods, and
Contexts. Washington, DC: The Aspen Institute; 1995.
28. Review of the use of ‘Theory of Change’ in international development. UK Department of International Development.
URL: https://www.gov.uk/government/news/
dfid-research-review-of-the-use-of-theory-of-change-in-international-development [accessed 2020-02-03]
29. Taplin D, Clark H, Collins E, Colby D. Theory of change TECHNICAL PAPERS a series of papers to support development
of theories of change based on practice in the field. ActKnowledge. URL: http://www.theoryofchange.org/wp-content/
uploads/toco_library/pdf/ToC-Tech-Papers.pdf [accessed 2020-03-23]
30. De Silva MJ, Breuer E, Lee L, Asher L, Chowdhary N, Lund C, et al. Theory of Change: a theory-driven approach to
enhance the Medical Research Council's framework for complex interventions. Trials 2014 Jul 05;15:267 [FREE Full text]
[doi: 10.1186/1745-6215-15-267] [Medline: 24996765]
31. Daruwalla N, Jaswal S, Fernandes P, Pinto P, Hate K, Ambavkar G, et al. A theory of change for community interventions
to prevent domestic violence against women and girls in Mumbai, India. Wellcome Open Res 2019 Mar 25;4:54. [doi:
10.12688/wellcomeopenres.15128.1]
32. Connell JP, Kubisch AC. Applying a theory of change approach to the evaluation of comprehensive community initiatives:
progress, prospects, and problems. Educ Crisis Conflict 1998;2(15-44):1-6 [FREE Full text]
33. Weiss CH. How can theory-based evaluation make greater headway? Eval Rev 2016 Jul 26;21(4):501-524. [doi:
10.1177/0193841x9702100405]
34. Nkwake A. Working with Assumptions in International Development Program Evaluation. Cham: Springer; 2020.
35. Uvhagen H, Hasson H, Hansson J, von Knorring M. What happened and why? A programme theory-based qualitative
evaluation of a healthcare-academia partnership reform in primary care. BMC Health Serv Res 2019 Nov 01;19(1):785
[FREE Full text] [doi: 10.1186/s12913-019-4665-1] [Medline: 31675956]
36. DeLone WH, McLean ER. The DeLone and McLean model of information systems success: a ten-year update. J Manag
Inf Syst 2014 Dec 23;19(4):9-30. [doi: 10.1080/07421222.2003.11045748]
37. Monitoring and Evaluating Digital Health Interventions: A Practical Guide to Conducting Research and Assessment.
Geneva: World Health Organisation; 2016.
38. Goeschel CA, Weiss WM, Pronovost PJ. Using a logic model to design and evaluate quality and patient safety improvement
programs. Int J Qual Health Care 2012 Aug;24(4):330-337. [doi: 10.1093/intqhc/mzs029] [Medline: 22745358]
39. Ojo AI. Validation of the DeLone and McLean information systems success model. Healthc Inform Res 2017 Jan;23(1):60-66
[FREE Full text] [doi: 10.4258/hir.2017.23.1.60] [Medline: 28261532]
40. Nigeria Health Facility Registry. Federal Ministry of Health. 2019. URL: https://hfr.health.gov.ng/facilities/hospitals-list
[accessed 2021-06-21]
41. Lagos State Maternal and Child Mortality Reduction (MCMR) program: good practices memo. Lagos State Ministry of
Health. URL: https://health.lagosstate.gov.ng/lagos-state-maternal-and-child-mortality-reduction-mcmr-program/ [accessed
2021-06-23]
42. Global Challenges Research Fund. UK Research and Innovation. 2021. URL: https://www.ukri.org/our-work/
collaborating-internationally/global-challenges-research-fund/ [accessed 2021-03-22]
43. Purposive sampling. Lund Research. 2012. URL: http://dissertation.laerd.com/purposive-sampling.php [accessed 2017-02-20]
44. Breuer E, Lee L, De Silva M, Lund C. Using theory of change to design and evaluate public health interventions: a systematic
review. Implement Sci 2016 May 06;11(1):63 [FREE Full text] [doi: 10.1186/s13012-016-0422-6] [Medline: 27153985]
45. Smith SW, Koppel R. Healthcare information technology's relativity problems: a typology of how patients' physical reality,
clinicians' mental models, and healthcare information technology differ. J Am Med Inform Assoc 2014;21(1):117-131
[FREE Full text] [doi: 10.1136/amiajnl-2012-001419] [Medline: 23800960]
46. Scott PJ, Briggs JS. STAT-HI: a socio-technical assessment tool for health informatics implementations. Open Med Inform
J 2010;4:214-220 [FREE Full text] [doi: 10.2174/1874431101004010214] [Medline: 21603280]
47. Fritz F, Tilahun B, Dugas M. Success criteria for electronic medical record implementations in low-resource settings: a
systematic review. J Am Med Inform Assoc 2015 Mar;22(2):479-488. [doi: 10.1093/jamia/ocu038] [Medline: 25769683]
48. Cordeiro L, Soares CB. Action research in the healthcare field: a scoping review. JBI Database System Rev Implement
Rep 2018 Apr;16(4):1003-1047. [doi: 10.11124/JBISRIR-2016-003200] [Medline: 29634517]
49. Thobias J, Kiwanuka A. Design and implementation of an m-health data model for improving health information access
for reproductive and child health services in low resource settings using a participatory action research approach. BMC
Med Inform Decis Mak 2018 Jun 25;18(1):45 [FREE Full text] [doi: 10.1186/s12911-018-0622-x] [Medline: 29941008]
50. OpenMRS homepage. OpenMRS. 2016. URL: https://openmrs.org/ [accessed 2020-03-09]
51. Muinga N, Magare S, Monda J, Kamau O, Houston S, Fraser H, et al. Implementing an open source electronic health record
system in Kenyan health care facilities: case study. JMIR Med Inform 2018 Apr 18;6(2):e22 [FREE Full text] [doi:
10.2196/medinform.8403] [Medline: 29669709]
52. Purkayastha S, Allam R, Maity P, Gichoya JW. Comparison of open-source electronic health record systems based on
functional and user performance criteria. Healthc Inform Res 2019 Apr;25(2):89-98 [FREE Full text] [doi:
10.4258/hir.2019.25.2.89] [Medline: 31131143]
53. Oza S, Jazayeri D, Teich JM, Ball E, Nankubuge PA, Rwebembera J, et al. Development and deployment of the
OpenMRS-Ebola electronic health record system for an Ebola treatment center in Sierra Leone. J Med Internet Res 2017
Aug 21;19(8):e294 [FREE Full text] [doi: 10.2196/jmir.7881] [Medline: 28827211]
54. Welcome to FHIR®. HL7. URL: https://www.hl7.org/fhir/ [accessed 2020-10-09]
55. What is openEHR? OpenEHR. URL: https://www.openehr.org/about/what_is_openehr [accessed 2020-10-09]
56. Scott PJ, Curley PJ, Williams PB, Linehan IP, Shaha SH. Measuring the operational impact of digitized hospital records:
a mixed methods study. BMC Med Inform Decis Mak 2016 Nov 10;16(1):143 [FREE Full text] [doi:
10.1186/s12911-016-0380-6] [Medline: 27829453]
57. King A, Crewe I. The Blunders of Our Governments. London: Oneworld Publications; 2014.
58. Chibanda D, Verhey R, Munetsi E, Cowan F, Lund C. Using a theory driven approach to develop and evaluate a complex
mental health intervention: the friendship bench project in Zimbabwe. Int J Ment Health Syst 2016;10:16 [FREE Full text]
[doi: 10.1186/s13033-016-0050-1] [Medline: 26933448]
59. CDC in Nigeria. Centers for Disease Control and Prevention. URL: https://www.cdc.gov/globalhealth/countries/nigeria/
default.htm [accessed 2022-03-22]
60. Verma N, Mamlin B, Flowers J, Acharya S, Labrique A, Cullen T. OpenMRS as a global good: impact, opportunities,
challenges, and lessons learned from fifteen years of implementation. Int J Med Inform 2021 May;149:104405. [doi:
10.1016/j.ijmedinf.2021.104405] [Medline: 33639327]
61. Schade CP, Sullivan FM, de Lusignan S, Madeley J. e-Prescribing, efficiency, quality: lessons from the computerization
of UK family practice. J Am Med Inform Assoc 2006;13(5):470-475 [FREE Full text] [doi: 10.1197/jamia.M2041] [Medline:
16799129]
62. Benson T. Why general practitioners use computers and hospital doctors do not--Part 1: incentives. BMJ 2002 Nov
09;325(7372):1086-1089 [FREE Full text] [doi: 10.1136/bmj.325.7372.1086] [Medline: 12424171]
63. Adler-Milstein J, Everson J, Lee SD. EHR adoption and hospital performance: time-related effects. Health Serv Res 2015
Dec;50(6):1751-1771 [FREE Full text] [doi: 10.1111/1475-6773.12406] [Medline: 26473506]
64. Nuckols TK, Smith-Spangler C, Morton SC, Asch SM, Patel VM, Anderson LJ, et al. The effectiveness of computerized
order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and
meta-analysis. Syst Rev 2014 Jun 04;3(1):56 [FREE Full text] [doi: 10.1186/2046-4053-3-56] [Medline: 24894078]
65. Murphy EV. Clinical decision support: effectiveness in improving quality processes and clinical outcomes and factors that
may influence success. Yale J Biol Med 2014 Jun;87(2):187-197 [FREE Full text] [Medline: 24910564]
66. Martin PM, Sbaffi L. Electronic health record and problem lists in Leeds, United Kingdom: variability of general practitioners'
views. Health Informatics J 2020 Sep;26(3):1898-1911 [FREE Full text] [doi: 10.1177/1460458219895184] [Medline:
31875417]
67. Wallis L, Blessing P, Dalwai M, Shin SD. Integrating mHealth at point of care in low- and middle-income settings: the
system perspective. Glob Health Action 2017 Jun 25;10(sup3):1327686 [FREE Full text] [doi:
10.1080/16549716.2017.1327686] [Medline: 28838302]
68. Fraser H, Adedeji T, Amendola P. The importance of primary care records in low- and middle-income settings for care,
resource management and disease surveillance: a review. In: ICT in Health: Survey on the Use of Information and
Communication Technologies in Brazil Healthcare Facilities. Brazil: Brazilian Network Information Center; 2021.
69. Ziemann M, Erikson C, Krips M. The use of medical scribes in primary care settings: a literature synthesis. Med Care 2021
Oct 01;59(Suppl 5):S449-S456 [FREE Full text] [doi: 10.1097/MLR.0000000000001605] [Medline: 34524242]
70. Howard K, Helé K, Salibi N, Wilcox S, Cohen M. Adapting the EHR scribe model to community health centers: the
experience of Shasta community health center's pilot. Blue Shield Foundation of California. URL: https:/
/blueshieldcafoundation.org/publications/
adapting-ehr-scribe-model-to-community-health-centers-experience-shasta-community [accessed 2020-02-03]
71. Kaufman DR, Sheehan B, Stetson P, Bhatt AR, Field AI, Patel C, et al. Natural language processing-enabled and conventional
data capture methods for input to electronic health records: a comparative usability study. JMIR Med Inform 2016 Oct
28;4(4):e35 [FREE Full text] [doi: 10.2196/medinform.5544] [Medline: 27793791]
72. Clinical data capture: OMR and OCR and your flatbed scanner. Medscape. URL: https://www.medscape.com/viewarticle/
497865_2 [accessed 2020-08-19]
73. Bartlett L, Avery L, Ponnappan P, Chelangat J, Cheruiyot J, Matthews R, et al. Insights into the design, development and
implementation of a novel digital health tool for skilled birth attendants to support quality maternity care in Kenya. Fam
Med Community Health 2021 Aug 03;9(3):e000845 [FREE Full text] [doi: 10.1136/fmch-2020-000845] [Medline: 34344764]
Abbreviations
ANC: antenatal care
D&M: DeLone and McLean
EHR: electronic health record
GP: general practitioner
Edited by C Lovis; submitted 14.09.21; peer-reviewed by A Kanter, A Garcia; comments to author 31.01.22; revised version received
25.03.22; accepted 31.05.22; published 11.08.22
Please cite as:
Adedeji T, Fraser H, Scott P
Implementing Electronic Health Records in Primary Care Using the Theory of Change: Nigerian Case Study
JMIR Med Inform 2022;10(8):e33491
URL: https://medinform.jmir.org/2022/8/e33491
doi: 10.2196/33491
PMID:
©Taiwo Adedeji, Hamish Fraser, Philip Scott. Originally published in JMIR Medical Informatics (https://medinform.jmir.org),
11.08.2022. This is an open-access article distributed under the terms of the Creative Commons Attribution License
(https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information,
a link to the original publication on https://medinform.jmir.org/, as well as this copyright and license information must be included.