Creating Accountability in Image Quality Analysis Part 3: Creation of a Standardized Image-Centric Mark-Up and Annotation Tool

While quality assurance (QA) has existed in some form from the inception of modern medicine, its importance and degree of scrutiny is greater than ever today, as evidenced by numerous quality-centric mandates issued from the Institute of Medicine (IOM) [1–3]. These IOM mandates have in turn fostered numerous healthcare and legislative quality initiatives including evidence-based medicine (EBM), meaningful use, and pay for performance [4–7]. The common denominator to these quality initiatives is data, which serves as the means with which measurement takes place and performance is judged. The goal of this data-driven analysis and intervention is to improve quality in healthcare delivery, which in turn is expected to improve clinical outcomes.

Unfortunately, in its present form, a great deal of medical data exists in nonstandardized formats, which precludes creation of referenceable databases and meta-analysis [8, 9]. At the same time, despite the “digitization” of vast amounts of medical data, data integration and accessibility remains a problem due to the relative lack of integration between disparate information systems [10, 11]. The combined inabilities to record, access, correlate, and analyze standardized data in medical practice adversely affects these quality initiatives. Despite almost universal support for the principles behind EBM, its widespread applicability is limited by these existing data deficiencies.

Attempts to improve quality and standardize data in medical imaging practice have been limited to date by a number of factors including the preferred method of radiology reporting (i.e., nonstandardized and narrative free text), inconsistency of QA standards (with the exception of mammography), lack of supporting quality-centric technology, and heightened emphasis on productivity and workflow enhancement in the face of declining reimbursements and increasing data volume and complexity [12–15]. If the IOM mandates are to be addressed in medical imaging practice, radical innovation is required which simultaneously addresses issues of data standardization and quality improvement without sacrificing workflow and productivity.

When attempting to create quality accountability strategies for medical imaging, it is essential to remember that medical imaging is a collective process, composed of several individual steps which we will call the medical imaging chain for simplistic purposes [16]. The various steps or links within this chain begin with order entry data (for the purpose of scheduling a medical imaging exam) by the referring clinician and ends with communication of medical imaging report findings by the radiologist, which in turn will trigger a clinical action based upon appropriate care standards. If any of the steps or links in this chain is limited by a quality deficiency, there will be some downhill effect, which can compromise patient care. These downhill adverse quality effects can take a number of forms; both obvious and insidious. An obvious quality effect may consist of an additional imaging exam, which may result in additional cost, radiation, and/or management time delay. A more insidious quality effect may consist of equivocal or ambiguous report findings, which may produce confusion or even error on the part of the clinician when instituting clinical management [17, 18]. It is somewhat ironic that an insidious quality effect such as report ambiguity can produce a negative impact of greater magnitude that a more obvious quality effect and this illustrates the clinical imperative of quality improvement in medical imaging.

In addition to these individual steps or links in the medical imaging chain, a multitude of stakeholders (or players) are involved, along with numerous technologies. In addition to the obvious stakeholders (e.g., imaging technologist, radiologist, and clinician) are less obvious players including support staff (e.g., nurse, clerical person, administrator, IT specialist, and patient). The technologies are generally predictable and include imaging modalities, image processing and reporting software, and numerous information systems (e.g., CPOE, PACS, RIS, and EMR).

If one was to combine these multiple steps, players, and technologies into a grid; the complexity and potential interaction effects would become apparent. This illustrates the fact that quality improvement is a multistep, multiplayer, and multitechnology process requiring methodical data collection, correlation, and analysis. Now if one was to attempt to identify “across the board” quality trends and opportunities for intervention, the only possible way to do so is to create a standardized data methodology at the point of care transcending the collective multistep process and independent of individual person, institutional, or technology differences.

The preferential focus on productivity and workflow (due to data overload and declining economics) has resulted in somewhat limited quality investments on the parts of medical imaging service providers and technology producers. Service providers do not want to be slowed down or have their limited resources diverted by quality initiatives, while technology producers do not want to invest in products with limited (or unproven) return on investments [19]. As a result, quality is often put aside during routine operation and retrieved during times of heightened scrutiny (e.g., accreditation, review, and adverse event). In the present state of medical imaging, quality measures are largely focused on time-stamped operational efficiency measures (e.g., scheduling backlog, patient throughput, and report turnaround times), which can be easily recorded and analyzed in a standardized fashion. While these efficiency measures are in part a good starting point, they do not represent a comprehensive measure of quality in the medical imaging chain and preclude clinical outcomes analysis. Simply stated, medical imaging QA in its present form is deficient and in desperate need of innovation, in order for medical imaging to remain clinically and economically viable.

While innovation strategies can focus on any of the medical imaging steps, a logical starting point is in the assessment of “technical” quality of the imaging dataset. Without optimization of the medical imaging dataset, accurate and confident diagnosis cannot be rendered by the interpreting radiologist, which in turn prevents timely, cost-effective, and definitive patient care. While creation of the imaging dataset takes place during the step of image acquisition (performed by technologists using image acquisition modalities), other steps affect technical image quality, including protocol optimization and image processing. The collective goals for the proposed innovation strategy should be the creation of standardized data accounting for each individual step in the collective imaging chain while also maintaining (or preferably exceeding) existing end-user workflow and productivity measures. Ideally, this would leverage both automation (to improve workflow) and adaptability (to enhance end-user options) in order to gain acceptance by the diverse and heterogeneous population of end-users.

The proposed innovation strategy should first evaluate the current QA model and look for opportunities to improve quality deliverables in a standardized fashion. By doing so, one comes to the realization that conventional QA programs related to technical imaging quality are idiosyncratic, time consuming, and inconsistent [13]. This creates a “low hanging fruit” scenario, where essentially any innovation which standardizes data collection and provide objective data-driven analysis (for quality improvement) will be a significant improvement to the status quo.

In addition to data standardization, another goal of the innovation strategy is to improve data integration, which is currently lacking in the traditional workflow and technology model, which separates clinical, imaging, and report data. The goal would be to directly integrate these various data elements onto a single all inclusive application, with the ability to customize data review in accordance with each individual end-user’s preferences. For medical imaging, the obvious application for combining multiple data elements is the imaging dataset. A methodology for directly integrating radiology report data onto the imaging dataset has been previously described as image-centric reporting [20], and could be applicable to the proposed innovation model for technical image quality assessment.

In this innovation strategy, individual images from the comprehensive medical imaging dataset would be selected by an end-user for input of technical image quality data. In order to standardize the input data, qualitative technical image quality data categories have been created which are in large part common to all imaging modalities and exam types (Table 1). The quantitative image quality data would utilize a standardized scoring system which is already being used by RadLex [21]; which utilizes a Likert scale for numerically rating subjective image quality (Table 2). These combined qualitative and quantitative data would in effect create a standardized method for technical image quality analysis which would be consistent among all end-users, institutional service providers, and technologies in use.

Since medical image annotation and mark-up has been relatively ubiquitous throughout conventional practice, the proposed innovation would take advantage of a data input model which is widely accepted. The end-user would simply select the image/s of interest (i.e., key images) and apply annotations using a standardized image mark-up and annotation schema similar to that used in AIM [22]. In order to maximize workflow flexibility of the proposed innovation model, multiple data input options would be presented to the end-user including speech (i.e., voice commands), user interface toolbar (e.g., icons), and graphical input (using an electronic stylus or finger with a touch screen monitor). The end-product would consist of the selected key image/s which has been annotated in a standardized fashion using a combination of graphical and alpha numeric data; which collectively reports the categories of image quality being evaluated and the corresponding technical image quality score.

The corresponding technical image quality folder would consist of a series of annotated “key images”, which can be reviewed in isolation or in conjunction with the comprehensive imaging dataset. The method of data presentation can be customized in accordance with individual end-user preferences. As an example, a radiologist may elect to initially review an imaging dataset in its original un-annotated presentation state, followed by sequential presentation of the annotated key images. By choosing this presentation model, the radiologist has the opportunity to review the entire imaging dataset devoid of visual distractions in the first visual pass, followed by a condensed review of “key images”. The annotations can be turned on or off at any point in time through a single input command and be customized (e.g., color coded) in the style of preferred graphical display. The annotation of these key images need not be restricted to quality data alone but incorporate other data elements (e.g., report finding and clinical data), thereby providing additional clinical context to the quality data being reported.

The proposed innovation model would also provide the ability for multiple individual end-users to record image quality data, which can in turn be selectively displayed and analyzed in accordance with individual end-user’s preferences. As an example, a radiologist may request that only image quality data be presented to him/her from other radiologists; while a radiology administrator may want to have image quality data from multiple input sources (e.g., radiologist, technologist, and clinician) presented and incorporated into quality analytics. If one elects to simultaneously review the quality annotation/mark-up of multiple input sources; they could easily do so, and differentiate different users’ input data through different display features such as differential font or color coding.

The use of a standardized quality annotation/mark-up tool directly embedded onto the medical imaging dataset would also provide an excellent education and consultation tool. As an example, a technologist performing a CT exam on a noncompliant patient may elect to solicit QA feedback from the interpreting radiologist during the image acquisition process. This could be readily accomplished by having the technologist select desired key images, annotate these images using the standardized QA schema, and electronically send these to the radiologist (who may be geographically distant) for direct feedback and recommendations. The radiologist could in turn annotate the same images and/or respond with written text for the purpose of directing protocol changes at the point of care. Alternatively, the same technologist may visualize an artifact of the imaging dataset they are unsure of and consult the QA imaging database for educational purposes. The QA imaging database could identify similar images using artificial intelligence techniques [23–25] and present these images along with associated supplemental data for correlation. This would in theory provide the technologist with an educational image-centric resource which can be used in real-time for QA review, assessment, and intervention.

The supplemental data associated with the medical imaging dataset can utilize a number of data sources (e.g., RIS, CPOE, and DICOM header) for incorporation of clinical, technical, and demographic data specific to the patient, institution, technology in use, and exam (Table 3). While this supplemental data is not in itself critical to the image quality assessment, it is beneficial to the comprehensive QA analytical process, which would provide one with the ability to analyze the interaction effects between image quality and patient, service provider, technology, and clinical context. A more detailed discussion of the QA analytics and practical applications will be covered in a separate manuscript.

One last feature of importance to consider is the determination of how the QA process will be validated to ensure data accuracy, consistency, and integrity. In current practice, QA is almost exclusively “internal” in performance, and performed almost exclusively by the medical imaging service provider. In the absence of external scrutiny, this internal QA process may be inherently flawed, resulting in deficient image quality which goes unnoticed. In light of the fact that the current accreditation/review processes are triennial and limited in scope [26], it is not unreasonable to assume that current QA practice can be biased and inconsistent in its present form. One solution to address the current deficiencies in everyday QA practice is to outsource some (or even all) of the QA process to a neutral entity which has an established performance record for accurate and consistent QA analysis. While this concept of “external” QA would have its own challenges and potential pitfalls (e.g., data confidentiality, accessibility, communication, validation, and mediation), it does provide a theoretical mechanism for unbiased, more consistent, and verifiable image quality assessment. The key to success is in large part creation of a standardized data collection QA instrument which directly integrates QA and imaging data using an exportable and easy-to-use process. The external QA model could be used intermittently or continuously, with the external QA provider remotely accessing the imaging database, performing the image QA mark-up and annotation, and transmitting the completed QA imaging folders to the service provider in a hierarchical fashion. One could theorize that the added expertise the dedicated QA provider would offer could provide added education and consultation benefits above and beyond local resources. In addition, the ability to perform external QA by an established expert third party would provide an economic framework for improved and expanded pay for performance. Service providers and payers could be provided with a real and tangible incentive and mechanism for aggressively improving image quality and rewarding those providers who can consistently demonstrate high-quality performance measures. At the same time, the technology community would be provided with an economic incentive to create new quality-centric technologies with the goal of further perpetuating quality improvement. The ultimate goal is to improve clinical outcomes through improved technical image quality. Down the road, additional steps in the medical imaging chain can undergo similar innovation efforts with the goal of creating quality synergy throughout the medical imaging continuum.