j o u r n a l o f s u r g i c a l r e s e a r c h  o c t o b e r 2 0 2 0 ( 2 5 4 ) 4 0 8 e4 1 6

Available online at www.sciencedirect.com

ScienceDirect

journal homepage: www.JournalofSurgicalResearch.com

Deep Learning-Based Risk Model for Best

Management of Closed Groin Incisions After
Vascular Surgery

Bora Chang, MD,a,* Zhifei Sun, MD,a,b Prabath Peiris, BS,a

Erich S. Huang, MD-PhD,a,b Ehsan Benrashid, MD,b
and Ellen D. Dillavou, MDb
a
KelaHealth, Durham, North Carolina
b
Department of Surgery, Duke University Medical Center, Durham, North Carolina

article info abstract

Article history: Background: Reduced surgical site infection (SSI) rates have been reported with use of closed
Received 29 October 2019 incision negative pressure therapy (ciNPT) in high-risk patients.
Received in revised form Methods: A deep learning-based, risk-based prediction model was developed from a large
13 January 2020 national database of 72,435 patients who received infrainguinal vascular surgeries
Accepted 16 February 2020 involving upper thigh/groin incisions. Patient demographics, histories, laboratory values,
Available online 17 March 2020 and other variables were inputs to the multilayered, adaptive model. The model was then
retrospectively applied to a prospectively tracked single hospital data set of 370 similar
Keywords: patients undergoing vascular surgery, with ciNPT or control dressings applied over the
Closed incision negative pressure closed incision at the surgeon’s discretion. Objective predictive risk scores were generated
therapy for each patient and used to categorize patients as “high” or “low” predicted risk for SSI.
High risk vascular patients Results: Actual institutional cohort SSI rates were 10/148 (6.8%) and 28/134 (20.9%) for high-
Infrainguinal vascular surgery risk ciNPT versus control, respectively (P < 0.001), and 3/31 (9.7%) and 5/57 (8.8%) for low-
Prediction model risk ciNPT versus control, respectively (P ¼ 0.99). Application of the model to the institu-
Surgical site infection tional cohort suggested that 205/370 (55.4%) patients were matched with their appropriate
intervention over closed surgical incision (high risk with ciNPT or low risk with control),
and 165/370 (44.6%) were inappropriately matched. With the model applied to the cohort,
the predicted SSI rate with perfect utilization would be 27/370 (7.3%), versus 12.4% actual
rate, with estimated cost savings of $231-$458 per patient.
Conclusions: Compared with a subjective practice strategy, an objective risk-based strategy
using prediction software may be associated with superior results in optimizing SSI rates
and costs after vascular surgery.
ª 2020 The Author(s). Published by Elsevier Inc. This is an open access article under the CC
BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Introduction increase health care costs to both individuals and health care
systems. Frequent application of prosthetic grafts is an
Surgical site infection (SSI) is a major source of morbidity after important feature of vascular surgery that elevates the risk of
infrainguinal vascular surgeries and can dramatically infection and makes SSI more morbid. Furthermore, vascular

* Corresponding author. KelaHealth, 301 Howard Street, Suite 950, San Francisco, CA 94105. Tel.: þ1 919 972 8231; fax: þ1 919 470 7099.
E-mail address: bora@kelahealth.com (B. Chang).
0022-4804/$ e see front matter ª 2020 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND
license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
https://doi.org/10.1016/j.jss.2020.02.012
 chang et al  vascular surgery incision management 409

surgery patients are often elderly with multiple underlying high amounts of stored data are more available. The relevance
comorbidities, such as diabetes, hypertension, heart disease, of such predictive models to impact surgical care grows sub-
and immune system disorders.1 A study of vascular surgery stantially when it can easily be used at the point of care.
readmissions after common vascular procedures reported Primary objectives of this study were to apply the DLPM to
32% and 41%  90-day hospital readmission rates for patients a single hospital cohort of vascular surgery patients to retro-
undergoing endovascular or open infrainguinal revasculari- spectively assess the appropriate use of ciNPT for post-
zation, respectively.2 Turtiainen et al. reported a 27% surgical operative management of closed vascular incisions, and to
wound infection rate in 184 consecutive patients after estimate the financial impact of adopting this risk-based
infrarenal aortic and lower limb vascular surgery procedures, strategy model.
81 of which were infrainguinal bypass surgeries.3
Surgical site infections in vascular surgery are associated
with more hospital readmissions, prolonged hospital stays, Methods
massive hemorrhage, systemic sepsis, the need for additional
operations, increased mortality, and an average increase in Data source
cost of $10,497 per event.4 In addition, there are several
uniquely grave consequences of SSIs in vascular surgery, For this retrospective study, a deep learningebased, risk-based
particularly graft loss and major amputation. prediction model was retrospectively applied to an institu-
Proper management of incisions as patients leave the tional data set of patients who underwent vascular surgery at
hospital is a focus of interventions aimed at reducing post- Duke University Hospital between January 1, 2016 and
operative complications. Infrainguinal incisions are at high December 31, 2017. The Duke University Institutional Review
risk of SSI because of skin tension, and the proximity of lymph Board approved this study (Duke Pro00089692). Funding for this
nodes and urogenital organs.5 To help address these risks, study came from the National Science Foundation (Award #
over the past decade, the use of negative pressure therapy 1721737) and Acelity Inc. Neither Duke University Hospital
over closed incisions after vascular surgery has expanded.6-12 administration nor Acelity Inc was involved in any aspect of
The closed incision negative pressure therapy (ciNPT) device study design, data collection, data analysis, or application of
is a disposable, battery-powered pump that applies negative the deep learning model. This study was approved by the Duke
pressure through a sterile dressing placed over a closed sur- Health Institutional Review Board (Duke Pro00089692).
gical incision. The system protects the incision from external
contamination, helps wound edge tension, and removes Description of institutional cohort
excess fluid and infectious materials while applying contin-
uous negative pressure. Use of ciNPT after vascular surgery Based on current procedural terminology (CPT) codes that
has been shown in prospective randomized trials to reduce were established in the inclusion criteria, records of 370
surgical site infection in patients at elevated risk of post- vascular surgery patients were extracted from the Duke Uni-
surgical complications.7-9,11,13 versity Hospital patient database and included for analysis in
Although the use of ciNPT is growing overall, SSI events this study. For model building, the Safe Harbor method was
continue to occur too frequently in high-risk populations. used to deidentify protected health information. Specific
Increased utilization of ciNPT among high-risk vascular pa- identifiers were removed from the data set and then individ-
tients may potentially reduce SSI rates, but it is important to ual columns under the Safe Harbor list were removed to
use this tool appropriately. Underutilization of ciNPT in high- generate a data set for use. Institutional cohort baseline pa-
risk patients, as well as overuse among low-risk patients, may tient and wound characteristics as well as operative proced-
lead to costly, inefficient care. To address this broader issue of ures are shown in Table 1. There was a significantly greater
appropriate health resource utilization, various predictive percentage of men, a higher median BMI, and more patients
models14,15 have been developed to guide clinical care. with a history of congestive heart failure in the standard of
Recently, Kuy et al. demonstrated that surgical mortality can care (SOC) group. There were more smokers in the ciNPT
be improved with the prospective use of predictive models as group. Otherwise, group baseline characteristics were similar.
part of preoperative surgical planning.16 All endovascular surgeries were performed with a femoral
For this present study, a novel deep learningebased pre- cutdown. Suprainguinal bypasses were those performed with
diction model (DLPM) was developed by a software data ana- an abdominal incision with both proximal and distal targets
lytics company to assist in predicting the patient-specific risk above the inguinal ligament, for example, aortoiliac graft.
of surgical site infection after vascular surgery. Machine
learning was used to analyze a large national data set selected Description of postsurgical procedure for institutional cohort
from the US National Surgical Quality Improvement Program
(NSQIP), to determine each patient’s comprehensive risk All operations on patients in the institutional cohort were
profile based on patient and wound characteristic inputs. performed by five different board-certified vascular surgeons.
Although such linear regression-based prediction models for As part of the institutional effort to reduce wound infections,
surgical complications have been present for decades, owing all patients were treated with the same “institutional bundled
to the cumbersome process of manual data entry into the risk care protocol”,17 which included optimization of perioperative
calculators, the adoption of such surgical risk calculators in risk factors, dedicated wound closure trays, and voluntary use
routine clinical practice has been limited. However, with of ciNPT in the operating room. Use of ciNPT (PREVENA Inci-
expanded implementation of the electronic health record, sional Management System, KCI, an Acelity company, San
410 j o u r n a l o f s u r g i c a l r e s e a r c h  o c t o b e r 2 0 2 0 ( 2 5 4 ) 4 0 8 e4 1 6

Table 1 e Baseline institutional cohort patient and wound characteristics.

Characteristic Total (n ¼ 370) SOC (n ¼ 191) ciNPT (n ¼ 179) P-value
Age (median [IQR]) 67 (59, 74) 67 (59, 75) 66 (58, 73) 0.244
Sex 0.008
Female 116 (31.4%) 48 (25.1%) 68 (38.0%)
Male 254 (68.6%) 143 (74.9%) 111 (62.0%)
Race 0.219
Asian 4 (1.1%) 4 (2.1%) 0 (0.0%)
Black 104 (28.1%) 50 (26.2%) 54 (30.2%)
Other 16 (4.3%) 9 (4.7%) 7 (3.9%)
White 246 (66.5%) 128 (67.0%) 118 (65.9%)
ASA class (median [IQR]) 3 (3, 3) 3 (3, 4) 3 (3, 4) 0.173
Body mass index (median [IQR]) 27.4 (23.9, 30.9) 27.5 (24.9, 31.6) 26.8 (23.0, 30.6) 0.037
Current smoker 140 (37.8%) 59 (30.9%) 81 (45.3%) 0.013
Diabetes 120 (32.4%) 60 (31.4%) 60 (33.5%) 0.665
Renal insufficiency 38 (10.3%) 23 (12.0%) 15 (8.4%) 0.455
Dialysis dependent 21 (5.7%) 11 (5.8%) 10 (5.6%) 0.871
Immunosuppression use 20 (5.4%) 13 (6.8%) 7 (3.9%) 0.416
History of stroke 49 (13.2%) 27 (14.1%) 22 (12.3%) 0.769
Preoperative wound infection 11 (3.0%) 4 (2.1%) 7 (3.9%) 0.508
Peripheral vascular disease 218 (58.9%) 107 (56.0%) 111 (62.0%) 0.396
History of congestive heart failure 68 (18.4%) 45 (23.6%) 23 (12.8%) 0.008
Ischemic heart disease 87 (23.5%) 48 (25.1%) 39 (21.8%) 0.67
General anesthesia 360 (97.3%) 187 (97.9%) 173 (96.6%) 0.757
Emergency surgery 85 (23.0%) 48 (25.1%) 37 (20.7%) 0.535
Operative procedure
Amputation of lower extremity 5 (1.4%) 2 (1.0%) 3 (1.7%)
Embolectomy and endarterectomy of lower limbs 17 (4.6%) 8 (4.2%) 9 (5.0%)
Endovascular abdominal aortic aneurysm repair 45 (12.2%) 34 (17.8%) 11 (6.1%)
Femoral exploration 6 (1.6%) 2 (1.0%) 4 (2.2%)
Femoral thromboendarterectomy 37 (10.0%) 13 (6.8%) 24 (13.4%)
Infra-inguinal bypass 127 (34.3%) 68 (35.6%) 59 (33.0%)
Open abdominal aortic aneurysm repair 5 (1.4%) 5 (2.6%) 0 (0.0%)
Supra-inguinal bypass 81 (21.9%) 35 (18.3%) 46 (25.7%)
Thoracic and complex endovascular aneurysm repair 4 (1.1%) 2 (1.0%) 2 (1.1%)
Other 43 (11.6%) 22 (11.5%) 21 (11.7%)
Wound class 0.226
Clean 314 (84.9%) 162 (84.8%) 152 (84.9%)
Clean contaminated 29 (7.8%) 16 (8.4%) 13 (7.3%)
Contaminated 5 (1.4%) 0 (0.0%) 5 (2.8%)
Dirty 11 (3.0%) 7 (3.7%) 4 (2.2%)
Missing 11 (3%) 6 (3.1%) 5 (2.8%)
Preoperative albumin (median [IQR]) 3.2 (2.6, 3.7) 3.1 (2.6, 3.7) 3.4 (2.8, 3.7) 0.137
Preoperative creatinine (median [IQR]) 1.1 (0.9, 1.4) 1.1 (0.9, 1.4) 1.0 (0.8, 1.3) 0.158
Preoperative hematocrit (median [IQR]) 31.3 (27.1, 37.0) 31.1 (27.0, 36.7) 31.6 (28.4, 37.1) 0.071
Preoperative white blood cell count (median [IQR]) 9.7 (8.0, 12.2) 9.8 (7.8, 12.3) 9.7 (8.2, 12.1) 0.997

Bold ¼ Statistically significant difference between groups (P < .05).

IQR ¼ Interquartile range; Q1 ¼ 25th percentile; Q3 ¼ 75th percentile; ASA ¼ American Society of Anesthesiologists.

Antonio, TX) over closed vascular surgery incisions had based on surgeon preference and subjective individual
increased within the vascular surgery department at Duke assessment of patient risk. In this institutional cohort, 179/370
University Hospital in 2016 because of concern for unaccept- (48.4%) patients received ciNPT, and 191/370 (51.6%) received
able SSI event rates. However, adoption of this device was SOC over their closed incision.
 chang et al  vascular surgery incision management 411

Closed incision NPT was applied over the surgically closed

incision in the operating room immediately after the vascular
procedure. Each ciNPT device was set to deliver -125 mmHg
negative pressure, and the dressing remained over the inci-
sion for up to 7 days after surgery. The goal was to apply ciNPT
for 7 days, but it may have been removed earlier if there were
difficulties maintaining a seal or if there was a need to inspect
the wound, usually for suspicion of bleeding or infection. In-
cisions of patients who did not receive ciNPT were managed
with standard surgical wound care (SOC), consisting of a
sterilely applied occlusive dressing, usually gauze and a
transparent film dressing, which was removed on post-
operative day 2.

Model development and calculations

Fig. 1 e Area under the receiver operating characteristic
KelaHealth (San Francisco, CA), a software data analytics curve of the national surgical site infection (SSI) prediction
company, developed the DLPM predictive algorithm using model. (Color version of figure is available online.)
derivative algorithms and methods from initial research per-
formed on the NSQIP data set, and the Duke University Measures of the performance and accuracy of the model
Hospitalespecific vascular cohort. Within this cohort, 72,435 were the following: sensitivity (recall) ¼ 0.8298;
national cases were included in the training set based on specificity ¼ 0.25; precision (positive predictive value) ¼ 0.1354;
defined CPT codes specific to infrainguinal vascular surgeries negative predictive value ¼ 0.9120; F1 score ¼ 0.2328.
that involved upper thigh/groin incisions, excluding amputa-
tions. All NSQIP variable features were taken into consider-
ation in the original machine learning training set. Definition of SSI
The national data set was split into an 80-20 training-
validation cohort. Using a multilayer perceptron learning al- An SSI event was defined according to American College of
gorithm, the model was trained using “development of SSI” as Surgeons-NSQIP guidelines, as the development of any su-
the outcome of interest. Fine-tuning from the national model perficial incisional, deep incisional, or organ/space infection,
based on the 72,435 cases was then applied to the institutional or wound dehiscence within 30 days from the date of
cohort of 370 vascular surgery patients from Duke University operation.18
Hospital. The same CPT codes that were used to select cases
from the NSQIP database also defined the hospital cohort.
Application of DLPM algorithm to institutional cohort
A comprehensive set of discrete and nondiscrete variables
which included patient demographic information, comorbid
Using the deep-learning SSI predictive model, objective pre-
conditions, disease characteristics, operative variables, past
dictive risk scores were generated for each patient.
medical and surgical histories, wound classification, and
The 0.07 risk score threshold was applied to stratify the
recent laboratory values were collected from the institutional
institutional patient cohort into groups considered to be
patient data and used as inputs to the model. The neural
“high” (282/370, 76%) or “low” (88/370, 24%) predicted risk for
network model architecture was identified with a set of
SSI.
hyperparameters after conducting a grid search. Batch
These patients were further divided into four groups for
normalization technique was used to improve performance
analysis: (1) low-risk patients who received SOC, (2) low-risk
and the stability of the network.
patients who received ciNPT, (3) high-risk patients who
The event rate of SSI in the national NSQIP database was
received SOC, and (4) high-risk patients who received ciNPT.
5.7%, and the area under the receiver operating characteristic
The actual SSI event rates were then calculated for each group
(AUROC) curve of DLPM was 0.68 (Fig. 1). After fine-tuning this
(Fig. 2). Data analysis was performed by data analytics com-
model using the predictor variables from the institutional
pany personnel under the supervision of an independent
cohort, the resulting model AUROC curve was 0.61, within a
statistical advisor.
normal variation. Given this AUROC curve, a risk score
threshold of 0.07 (percentage risk score between 0 and 1) was
determined by balancing the care cost of SSI and the cost of Definition of “appropriate” use of ciNPT
the ciNPT system, the level of sensitivity versus specificity
desired by the hospital, plus the clinical estimation of infec- The following clinical assumptions were used as a basis for
tion risk. Specifically, a combination of Youden’s J statistic determining “appropriate” use of ciNPT: ciNPT is appropriate
(which balances sensitivity and specificity for high-risk patients to help decrease occurrence of SSI, and
[J ¼ sensitivity þ specificity e 1]) and clinical input from the SOC is appropriate for low-risk patients. These assumptions
institution regarding its safety threshold was used for greater are based on study results7,9 and consensus recommenda-
maximization of sensitivity to result in more false positives tions by Willy et al.19 that support the selective application of
and fewer false negatives. ciNPT in high-risk patients.
412 j o u r n a l o f s u r g i c a l r e s e a r c h  o c t o b e r 2 0 2 0 ( 2 5 4 ) 4 0 8 e4 1 6

Fig. 2 e Patient stratification and outcomes based on a risk threshold of 0.07.

Cost assumptions events. It was assumed that ciNPT would be used for high-risk
patients, and SOC would be used for low-risk patients.
The cost of the ciNPT system was assumed to be $495,
whereas the SOC surgical dressing was assumed to be negli-
gible ($0.00). The cost of SSI was calculated at $10,497, a pre-
viously published estimated average excess cost of an SSI in Results
patients undergoing general and vascular surgery.4 In this
study by Boltz et al.,4 the average cost was calculated using In the high-risk institutional cohort, patients receiving ciNPT
linear regression analysis of institutional data from the NSQIP. had an SSI rate of 6.8%, and patients that received SOC had an
These data represented fully loaded operating costs estimated SSI rate of 20.9% (P < 0.001). In the low-risk cohort, there was
in the database from a ratio of costs-to-charges methodology, no significant difference in SSI rates between patients
whereby costs were estimated as a percentage of hospital receiving ciNPT compared with SOC (9.7% ciNPT versus 8.8%
charges.4 SOC [P ¼ 0.99]). With the application of DLPM, results suggest
Estimations of the potential SSI and cost reduction with the that while 205 (55.4%) patients were appropriately matched
implementation of risk stratification were based on actual SSI with their intervention (either high-risk with ciNPT or low-risk

Table 2 e Actual versus predicted surgical site infection rate with application of deep learningebased prediction model in
vascular surgery patients who received ciNPT or SOC.
Actual

Predicted risk Intervention n Actual SSI Actual SSI Appropriate match P-value (ciNPT
events (n) rate (%) of intervention & risk, based versus SOC)
on risk threshold ¼ 0.07
High ciNPT 148 10 6.8 Yes <0.001
SOC 134 28 20.9 No
Low ciNPT 31 3 9.7 No 0.99
SOC 57 5 8.8 Yes
Total ciNPT 179 13 7.3
Total SOC 191 33 17.3
Total — 370 46 12.4
 chang et al  vascular surgery incision management 413

with SOC), 165 (44.6%) patients were inappropriately matched classification problems with sparse inputs and nonlinear
(either high-risk with SOC or low-risk with ciNPT). feature interactions to improve predictive performance.24 The
If all patients were matched with the appropriate inter- precision metric for this model is defined as the ratio between
vention over their closed incision, assuming the actual SSI the “true positive” value and a total number of predictive
rates in the retrospective cohort of 6.8% for high-risk patients positives. Having a lower precision but higher recall allows
and 8.8% for low-risk patients, the predicted SSI events in the this model to lower false negatives and help avoid missing
entire cohort by perfect utilization based on risk would be 27/ patients who would have a “catastrophic outcome” such as a
370 (7.3%; Table 2). Because there was no significant difference readmission. This precision metric is meant to balance cost
in the SSI rates between high-risk patients receiving ciNPT trade-offs and determine the point at which the cost of false
(6.8%) and low-risk patients receiving SOC (8.8%), it was positives ($495 each) is worth avoiding false negatives which
assumed that the predicted SSI rate would fall within this can lead to readmissions.
range (Table 3). These three predicted SSI rates (6.8%, 7.3%, Wound infections after vascular surgery result from the
and 8.8%) were used to estimate the potential impact of complex interplay of countless patient- and procedure-related
applying the DLPM risk-based adoption strategy versus actual factors. Independent predictors of SSI include obesity, dia-
use in the cohort size of 370 patients. For the 370 patients, the betes, tobacco use, and chronic obstructive pulmonary dis-
actual total cost of infections and ciNPT units was estimated ease. End-stage renal disease, coronary artery disease,
to be $571,467, or $1545 per patient. With the risk-based model hypertension, critical limb ischemia, previous infrainguinal
applied, the projected cost per patient ranges from $1087 to arterial surgery, dyspnea, prolonged operative time >4 h,
$1,313, which is a cost reduction of 15.0% to 29.7%, or $231 to groin anastomosis, or American Society of Anesthesiologists
$458 per patient. This corresponds to a relative SSI rate class 4 or 5 are common risk factors.5,25 In developing this
reduction of 29.2% to 45.3% (Table 3). model, machine learning was used to sift through data from
72,435 patient records to determine which combinations of
factors most commonly predict SSIs. Although this model is
Discussion not perfect, it allows better stratification of predicted risk than
is possible without the model.
Artificial intelligence has been dubbed a top technology trend Since 2004, numerous randomized controlled trials (RCTs)
in the coming decades. Although advancements in electronic and individual cohort studies have described ciNPT use
health records have been remarkable, these data provide little encompassing various incision types, including abdominal,
more than paper charts. However, when enhanced by the sternal, traumatic, orthopedic, breast, cesarean-section, and
power of analytics and machine learning, these data can be vascular.26-31 Preclinical studies evaluating ciNPT compared
better leveraged for medical professionals at the point of pa- with standard wound care reported reduced incision scar
tient care. Within the field of machine learning, deep learning thickness and incision scar width, increased collagen at the
is a growing area of research that aims to learn rich repre- incision site, increased mechanical properties, and increased
sentations of data by extracting hidden or “latent” relation- tensile strength in the ciNPT groups.32,33 The highest per-
ships from a set of large and complex features, more easily centage of published clinical RCTs comparing the use of ciNPT
and practically performed at scale by a machine than a human and control groups involves vascular surgery incisions.13
brain. These methods have enjoyed widespread success in a Enrolled patients have often had comorbidities including
variety of challenging applications such as computer vision, obesity, diabetes mellitus, peripheral vascular disease, or
automatic speech recognition, and natural language process- chronic obstructive pulmonary disease.19 Most of the pub-
ing, and are routinely used at companies such as Google and lished prospective and retrospective studies have reported
Facebook to process large volumes of web-scale data.20 that patients treated with ciNPT have showed reduced SSI
The value of machine learning in health care is its ability to rates, with the disclaimer that a greater number of large RCTs
process huge data sets beyond the scope of human capability, are necessary. The largest to-date, prospective RCT of vascular
and then reliably convert analysis of the data into clinical patients receiving ciNPT assessed 188 patients for superficial
insights that aid medical professionals in providing care, with SSIs of groin incisions following vascular surgery for periph-
the goal of enhancing outcomes and lowering care costs. eral artery disease.9 The control group experienced more
Medical practitioners who have had access to this advanced frequent SSIs (30/90; 33.3%) than the ciNPT group (13/98;
information have recognized that machine learning holds 13.2%; P ¼ 0.0015; absolute risk differenced20.1 per 100; 95%
enormous potential in the field of health care.21 Google has CI -31.9 to 8.2).9
developed a machine learning algorithm to help identify An international consensus document regarding use of
cancerous tumors on mammograms,22 and Stanford Univer- ciNPT designated high tension, open groin incisions as high-
sity is using a deep learning algorithm to identify skin risk incisions where ciNPT use is recommended, particularly
cancer.23 if accompanied by a synthetic graft or vascular graft inserted
For this present study, a software data analytics company below the inguinal ligament.19 Consensus panel members
collaborated with Duke University Hospital administrators to noted there is a level of uncertainty surrounding the decision
develop a deep, learningebased prediction model to predict to use ciNPT. This is the first report of machine learning to
postoperative complications for a spectrum of general sur- help guide the use of ciNPT and reduce uncertainty in deci-
geries, including vascular surgeries. Deep neural networks sion-making.
(complex mathematical functions with hundreds of parame- The novelty of this model is that it applies a specific
ters) were utilized in a manner that excels at large-scale intervention that drives a clinical outcome and financial
 414
j o u r n a l o f s u r g i c a l r e s e a r c h  o c t o b e r 2 0 2 0 ( 2 5 4 ) 4 0 8 e4 1 6
Table 3 e Financial impact estimations on institutional cohort with application of a deep learningebased prediction model.
Group/Cost item Actual Low-range estimate Medium-range estimate High-range estimate

SSI rate SSI events Projected Projected Projected Projected Projected Projected
SSI rate SSI events SSI rate SSI events SSI rate SSI events
High ciNPT (n ¼ 148) 6.8% 10 8.8% 13 6.8% 10 6.8% 10
High SOC (n ¼ 134) 20.9% 28 8.8% 12 6.8% 9 6.8% 9
Low ciNPT (n ¼ 31) 9.7% 3 8.8% 3 8.8% 3 6.8% 2
Low SOC (n ¼ 57) 8.8% 5 8.8% 5 8.8% 5 6.8% 4
Total (n ¼ 370) 12.4% 46 8.8% 33 7.3% 27 6.8% 25
SSI cost ($10,497 per infection) $482,862 $346,401 $283,419 $262,425
ciNPT devices used/recommended 179 282 282 282
ciNPT device cost ($495) $88,605 $139,590 $139,590 $139,590
Total cost (infection þ devices) $571,467 $485,991 $423,009 $402,015
Cost per patient $1544.51 $1313.49 $1143.27 $1086.53
SSI % reduction (absolute) 3.6% 5.1% 5.6%
SSI % reduction (relative) 29.2% 41.3% 45.3%
Cost reduction (absolute) $85,476 $148,458 $169,452
% Cost reduction 15.0% 26.0% 29.7%
Cost reduction per patient $231.02 $401.24 $457.98

Bold ¼ Statistically significant difference between groups (P < .05).

 chang et al  vascular surgery incision management 415

incentive. Although both the clinical evidence for and the use
of ciNPT in vascular procedures has expanded in recent years, Conclusion
there is a critical need to provide financial models that can
optimize its utility in clinical practice. The data used to fuel In determining whether to use ciNPT versus a standard dres-
this model are extracted from the medical record, allowing sing over closed vascular groin incisions, using an SSI risk
enhanced decision-making for busy clinicians. It produces prediction software model platform to guide decision-making
data that is actionable and helps close the loop between may result in superior results in optimizing clinical outcomes
intervention and financial outcome. In addition to informing and costs, compared with a subjective decision-making
resource utilization, identification of high-risk patients may strategy. If the risk-based model that was developed for this
improve preoperative counseling and encourage other modi- study were to prospectively replace subjective human
fications in perioperative management to optimize decision-making regarding the use of ciNPT versus SOC for this
outcomes.34 analyzed hospital data set of 370 vascular surgery patients, it
Administrators at Duke University Hospital requested use was projected that perfect utilization would yield a 7.3% SSI
of this risk-strategy technology to determine if the effect of rate, versus the actual rate of 12.4%.
ciNPT was being maximized at their facility in vascular sur- The actual total cost of infections and ciNPT units was
gery. During the study, surgeon assessment of high-risk estimated to be $571,467, or $1545 per patient for the institu-
wounds was the primary driver of the decision to use ciNPT tional cohort. With the risk-based adoption strategy model
in the institutional cohort, but subjective surgeon assessment applied to this institutional data set, the projected cost per
varied between surgeons and for each patient. A goal of this patient was estimated to range from $1087 to $1,313, a cost
study is to provide supporting data so that surgeons will reduction of 15.0% to 29.7%, or $231 to $458 per patient. The
change practice, in a consistent, objective, data-driven novelty of this model is that it applies a specific intervention
manner, for patient improvement. This risk-strategy model that drives a clinical outcome and financial incentive. Such
may be suitable for any hospital administration and surgical prediction software can potentially be used as a companion
team that has higher than desired complication or read- tool for determining the best use of interventions such as
mission rates and wants to improve surgical quality. Knowing ciNPT and can be promoted as a means for optimizing their
the levels of risk at the point of care for each patient allows utilization.
clinicians to make better decisions about treatment options
while understanding potential outcomes and cost.
The SSI event rate in the NSQIP database (5.7%) was sur-
Acknowledgment
prisingly lower than expected for vascular surgeries of this
nature, and even lower than the 8.8% SSI rate of the low risk
The authors thank Karen Beach and Leah Griffin (Acelity, San
institution group. These investigators suspect that a differ-
Antonio, TX) for their assistance with editing the article.
ence between the two groups in defining an SSI is a major
Authors’ contributions: B.C. was involved in the conception
reason for the variance. Patients in the institutional group
and design of the manuscript, analysis and interpretation of
were prospectively followed for 90 days with all possible in-
data, writing the manuscript, critical revisions, and approval
fections noted, regardless of whether they required an esca-
of the final manuscript. Z.S. participated in concept and
lation of therapy. Similar trends in SSI rates were consistently
design, writing the manuscript, critical revisions, and
observed between the two data sets, although the exact
approval of the final manuscript. P.P. performed the analyses
numbers were disparate.
and interpretation of data as well as critical revisions, and he
There are several limitations to this study. A limitation of
approved the final manuscript. E.S.H. participated in the
the use of deep neural networks is that it lacks high inter-
conception and design as well as writing, revising and
pretability and does not allow distinct quantitative assess-
approval of final manuscript. E.B. performed analyses and
ment of the contributing impact of each variable to the final
interpretation of data and helped to revise and approve final
outcome. In addition, the performance of the models both in
manuscript. E.D.D. performed analyses and interpretation of
the national cohort (AUC 0.679) and fine-tuned to a vascular
data and helped to revise and approve final manuscript.
population (0.61) could be further enhanced and boosted for
Funding for this study came from the National Science
better performance. Although there were no explicit discrep-
Foundation [United States, Award # 1721737 to KelaHealth]
ancies between NSQIP variables and institutional variables,
and Acelity Inc. Neither National Science Foundation nor
collection of health information always has some level of
Acelity had any involvement in the study design; collection,
variability. The difference in performance may be attributed
analysis and interpretation of data; writing of the report; or in
to natural variation that occurs when attempting to make
the decision to submit the article for publication.
direct comparisons between a large retrospective data set and
a small set of prospectively tracked patients. There was likely
significant heterogeneity within the national cohort in the Disclosure
definition of “surgical site infection.” In addition, the small
sample size of 370 patients could have limited the differences B.C. is the Chief Executive Officer and Co-founder of Kela-
between the rates of SSI of the risk-based intervention co- Health. P.P. is a software engineer at KelaHealth. E.D.D. is a
horts. Prospective application of this prediction software consultant for and received a research grant from KCI, an
model across multiple institutions is needed to validate its Acelity Company. E.D.D. is also a consultant for Gore Medical
role in clinical practice. and Angiodynamics. E.B. received a research grant from KCI,
416 j o u r n a l o f s u r g i c a l r e s e a r c h  o c t o b e r 2 0 2 0 ( 2 5 4 ) 4 0 8 e4 1 6

an Acelity Company. Z.S. and E.S.H. are affiliated with Duke 16. Kuy S, Romero RAL. Decreasing 30-day surgical mortality in a
University and are also founding advisors of KelaHealth. VA medical center utilizing the ACS NSQIP surgical risk
calculator. J Surg Res. 2017;215:28e33.
17. Benrashid E, Youngwirth LM, Guest K, Cox MW, Shortell CK,
Dillavou ED. Negative pressure wound therapy reduces
surgical site infections. J Vasc Surg. 2019;71:896e904.
references
18. Ban KA, Minei JP, Laronga C, et al. American College of
surgeons and surgical infection society: surgical site infection
guidelines, 2016 update. J Am Coll Surg. 2017;224:59e74.
1. Zhang YG, Guo XL, Song Y, Miao CF, Zhang C, Chen NH. 19. Willy C, Agarwal A, Andersen CA, et al. Closed incision
Diagnosis and treatment of vascular surgery related infection. negative pressure therapy: international multidisciplinary
Open Biomed Eng J. 2015;9:250e255. consensus recommendations. Int Wound J. 2017;14:385e398.
2. Orr NT, El-Maraghi S, Davenport DL, Xenos ES. PS62: vascular 20. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature.
surgery readmissions after common vascular procedures: risk 2015;521:436e444.
factors and cost analysis. J Vasc Surg. 2014;59:50S. 21. Obermeyer Z, Emanuel EJ. Predicting the future - big data,
3. Turtiainen J, Saimanen E, Partio T, et al. Surgical wound machine learning, and clinical medicine. N Engl J Med.
infections after vascular surgery: prospective multicenter 2016;375:1216e1219.
observational study. Scand J Surg. 2010;99:167e172. 22. Chen PH, Gadepalli K, MacDonald R, et al. An augmented
4. Boltz MM, Hollenbeak CS, Julian KG, Ortenzi G, Dillon PW. reality microscope for real-time automated detection of
Hospital costs associated with surgical site infections in cancer. 2018. Available at: https://drive.google.com/file/d/
general and vascular surgery patients. Surgery. 1L5Yudm5k4ZGejYWMgy8z6KYMGJl4y5mc/view. Accessed
2011;150:934e942. August 12, 2019.
5. Aicher B, Curry P, Croal-Abrahams L, et al. Infrainguinal 23. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level
wound infections in vascular surgery: an antiquated classification of skin cancer with deep neural networks.
challenge without a modern solution. J Vasc Nurs. Nature. 2017;542:115e118.
2017;35:146e156. 24. Cheng HT, Koc L, Harmsen J, et al. Wide & deep learning for
6. Sabat J, Tyagi S, Srouji A, et al. IP123 - prophylactic negative- recommended systems. In: Proceedings of the 1st workshop on
pressure therapy for femoral incision in vascular surgery: deep learning for recommender systems. 2016:7e10.
preliminary results of a prospective, randomized trial. J Vasc 25. Wiseman JT, Fernandes-Taylor S, Barnes ML, et al. Predictors
Surg. 2016;63:94Se95S. of surgical site infection after hospital discharge in patients
7. Pleger SP, Nink N, Elzien M, Kunold A, Koshty A, Boning A. undergoing major vascular surgery. J Vasc Surg.
Reduction of groin wound complications in vascular surgery 2015;62:1023e1031.
patients using closed incision negative pressure therapy 26. Grauhan O, Navasardyan A, Hofmann M, Muller P, Stein J,
(ciNPT): a prospective, randomised, single-institution study. Hetzer R. Prevention of poststernotomy wound infections in
Int Wound J. 2018;15:75e83. obese patients by negative pressure wound therapy. J Thorac
8. Lee K, Murphy PB, Ingves MV, et al. Randomized clinical trial Cardiovasc Surg. 2013;145:1387e1392.
of negative pressure wound therapy for high-risk groin 27. Pachowsky M, Gusinde J, Klein A, et al. Negative pressure
wounds in lower extremity revascularization. J Vasc Surg. wound therapy to prevent seromas and treat surgical
2017;66:1814e1819. incisions after total hip arthroplasty. Int Orthop.
9. Gombert A, Babilon M, Barbati M, et al. Closed incision 2012;36:719e722.
negative pressure therapy reduces surgical site infections in 28. Conde-Green A, Chung TL, Holton III LH, et al. Incisional
vascular surgery: a prospective randomised trial (aims trial). Negative-Pressure Wound Therapy versus conventional
Eur J Vasc Endovasc Surg. 2018;56:442e448. dressings following abdominal wall reconstruction. A
10. Engelhardt M, Rashad NA, Willy C, et al. Closed-incision comparative study. Ann Plast Surg. 2013;71:394e397.
negative pressure therapy to reduce groin wound infections 29. Stannard JP, Volgas DA, McGwin G, et al. Incisional negative
in vascular surgery: a randomised controlled trial. Int Wound J. pressure wound therapy after high-risk lower extremity
2018;15:327e332. fractures. J Orthop Trauma. 2012;26:37e42.
11. DiMuzio P, Staley C, Reiter D, et al. RS06: a randomized study 30. Gabriel A, Sigalove S, Sigalove N, et al. The impact of closed
evaluating negative-pressure therapy to decrease vascular incision negative pressure therapy on postoperative breast
groin wound complications. J Vasc Surg. 2017;65:133S. reconstruction outcomes. Plast Reconstr Surg Glob Open.
12. Matatov T, Reddy KN, Doucet LD, Zhao CX, Zhang WW. 2018;6:e1880.
Experience with a new negative pressure incision 31. Gunatilake RP, Swamy GK, Brancazio LR, et al. Closed-incision
management system in prevention of groin wound infection negative-pressure therapy in obese patients undergoing
in vascular surgery patients. J Vasc Surg. 2013;57:791e795. cesarean delivery: a randomized controlled trial. AJP Rep.
13. Singh DP, Gabriel A, Silverman RP, Griffin LP, D’Agostino 2017;7:e151ee157.
McGowan L, D’Agostino Jr RB. Meta-analysis comparing 32. Kilpadi DV, Lessing C, Derrick K. Healed porcine incisions
outcomes of two different negative pressure therapy systems previously treated with a surgical incision management
in closed incision management. Plast Reconstr Surg Glob Open. system: mechanical, histomorphometric, and gene
2019;7:e2259. expression properties. Aesthetic Plast Surg.
14. D’Agostino Sr RB, Grundy S, Sullivan LM, Wilson P. Validation 2014;38:767e778.
of the Framingham coronary heart disease prediction scores: 33. Suh H, Lee AY, Park EJ, Hong JP. Negative pressure
results of a multiple ethnic groups investigation. J Am Med wound therapy on closed surgical wounds with dead
Assoc. 2001;286:180e187. space: animal study using a swine model. Ann Plast
15. Magovern JA, Sakert T, Benckart DH, et al. A model for Surg. 2014;76:717e722.
predicting transfusion after coronary artery bypass grafting. 34. Mueck KM, Kao LS. Patients at high-risk for surgical site
Ann Thorac Surg. 1996;61:27e32. infection. Surg Infect. 2017;18:440e446.