Abstract

Introduction

The unequal distribution of medical resources remains a global issue. Although new technologies and ideas constantly evolve, inexperienced medical caregivers and outdated technologies and ideas in medically underserved areas may prevent patients from accessing optimal treatment^1–3. This disparity increases the risks of complications, larger surgical incisions, and suboptimal cosmetic outcomes, depriving patients of equitable medical services. Moreover, seeking in-person consultation or transferring patients to a better-equipped hospital increases transportation hazards, medical burden, nursing care inconvenience, and travel costs^3–8. Remote surgical instruction can help address this challenge, yet its full potential remains underutilized.

In the past, remote surgical instructions provided via audio and video methods (such as speaking commands or directional cues) did not accurately, quickly, and effectively facilitate the performance of remote surgical procedures⁶. Recent technological advancements have introduced augmented reality (AR), enabling virtual instructions for remote surgery. Previous AR remote surgical guidance relied primarily on endoscopic systems, microscopes, AR glass, and Google Glass^9–13. When these methods are used for non-endoscopic procedures, such as internal fixation of fractures, surgeons must constantly shift their focus between the operating field and the video screen during remote instruction. This process increases the operating time and distracts the operator’s attention since it involves applying the instructions on the screen to the operating field, increasing the surgeon’s cognitive burden. Past AR remote surgical guidance delivered suboptimal viewing due to lacking a clear distance view, especially for detailed guidance. Furthermore, the Google Glass remote mode guidance field of view is not aligned with the operating field view^9–13.

Given these issues, we developed a new AR remote system featuring a screen positioned within the surgeon’s visual field to guide fracture surgery remotely using an adjustable stent at the operating end. To our knowledge, no study has reported the performance of remote-guided fracture surgery with AR and remote-guided surgery with screens located directly in the surgical visual field^9–19. This study aimed to evaluate the effectiveness and safety of this new AR remote mode. We hypothesise that this new remote mode will improve the implementation of remote fracture surgery effectively and safely.

Materials and methods

Data sources and study population

This retrospective cohort analysis used information from the electronic medical record databases of three centres in Hubei, China. The database contains data on population characteristics at hospital admission, injury details, and surgical notes. The patients’ postoperative data were obtained from the follow-up records. Following fracture surgery, patients undergo routine outpatient review within the first 3 months, with appointments typically scheduled every 2–4 weeks. After 3 months, outpatient visits are scheduled every 2–6 months. The study was approved by the institutional review board (IRB number: 2022087501) and followed the Strengthening the Reporting of Observational Studies in Epidemiology reporting guideline for cohort studies. The work has been reported in line with the STROCSS criteria, Supplemental Digital Content 1, http://links.lww.com/JS9/C712 ²⁰. This study is registered with the Chinese Clinical Trial Registry. Patients who were admitted between 1 January 2018 and 31 March 2022, due to elbow fracture, tibial plateau fracture, patella fracture, or ankle fracture and underwent open reduction and internal fixation were included in the study. The study excluded patients with other injuries to the ipsilateral limb, pathological fractures, fractures older than 14 days, a history of fractures, or surgery performed at the site of the current fracture.

A total of 800 patients who underwent surgery for fractures of the elbow, tibial plateau, patella, and ankle fracture were selected. The patients were divided into an AR group (specialists remotely guided junior doctors in performing surgeries through AR) and a traditional non-remote group (specialists performed the surgery). Of the 800 patients, 249 were excluded (40 in the AR group; 209 in the non-remote surgery group) because they met one or more of the exclusion criteria: a follow-up period of less than 1 year (126 patients, 50.6%), fracture more than 14 days (33 patients, 13.3%), other injuries in the ipsilateral limb (50 patients, 20.1%), and repeated fracture or surgery on the currently fractured site (40 patients, 16.1%). The final study cohort comprised 551 patients (184 patients (33.4%) in the AR remote surgery group and 367 patients (66.6%) in the non-remote surgery group) (Table ​(Table1).1). To minimise potential confounding factors between the two groups, we used propensity score matching (PSM) to match patients in the AR remote surgery group with those in the non-remote surgery group who shared similar characteristics.

Table 1

Baseline demographic characteristics of patients before and after propensity score matching based on type of AR remote group and non-remote surgery group.

	Before propensity score matching			After propensity score matching
Characteristic	AR group (n=184)	Non-remote surgery group (n=367)	P	AR group (n=182)	Non-remote surgery group (n=182)	P
Age (Mean±SD)	48.23±14.04	47.95±15.21	0.863	48.11±14.07	48.29±15.05	0.896
(Median, range)	50.5 (18, 82)	48 (18, 83)		50 (18, 82)	47 (18, 77)
BMI（Mean±SD)	23.17±3.20	22.80±2.84	0.342	23.17±3.19	22.78±2.84	0.303
≤18.5	8 (4.3)	17 (4.6)		8 (4.4)	8 (4.4)
18.5–23.9	113 (61.4)	232 (63.2)		107 (58.8)	110 (60.4)
≥24.0	63 (34.2)	118 (32.2)		67 (36.8)	64 (35.2)
Sex, n (%)			0.311			0.525
Male	102 (55.4)	220 (59.9)		101 (55.5)	107 (58.8)
Female	82 (44.6)	147 (40.1)		81 (44.5)	75 (41.2)
Educational level, n (%)			0.480			0.764
Primary school	31 (16.8)	77 (21.0)		30 (16.5)	25 (13.7)
Junior high school	80 (43.5)	157 (42.8)		80 (44.0)	83 (45.6)
High school or higher	73 (39.7)	133 (36.2)		72 (39.6)	74 (40.7)
Injury mechanism, n (%)			0.239			0.237
No	107 (58.2)	194 (52.9)		106 (58.2)	117 (64.3)
Yes	77 (41.8)	173 (47.1)		76 (41.8)	65 (35.7)
Affected side, n (%)			0.017			1.000
Left	83 (45.1)	205 (55.9)		82 (45.1)	82 (45.1)
Right	101 (54.9)	162 (44.1)		100 (54.9)	100 (54.9)
Fracture site, n (%)			0.653			0.520
Patella	50 (27.2)	82 (22.3)		49 (26.9)	56 (30.8)
Elbow	42 (22.8)	86 (23.4)		42 (23.1)	46 (25.3)
Tibil	40 (21.7)	86 (23.4)		40 (22.0)	41 (22.5)
Ankle	52 (14.2)	113 (30.8)		51 (28.0)	39 (21.4)
Classification, n (%)			0.728			0.552
A	15 (8.2)	37 (10.1)		15 (8.2)	13 (7.1)
B	57 (31.0)	116 (31.6)		56 (30.8)	48 (26.4)
C	112 (60.9)	214 (58.3)		111 (61.0)	121 (66.5)
Medical history, n (%)			0.140			1.000
No	134 (72.8)	288 (78.5)		133 (73.1)	133 (73.1)
Yes	50 (27.2)	79 (21.5)		49 (26.9)	49 (26.9)
Cancer	6 (3.3)	6 (1.6)		5 (2.7)	3 (1.6)
Diabetes	12 (6.5)	20 (5.4)		12 (6.6)	11 (6.0)
Chronic kidney disease	2 (1.1)	1 (0.3)		2 (1.1)	1 (0.5)
Pulmonary disease	3 (1.6)	7 (1.9)		3 (1.6)	6 (3.3)
Heart disease	6 (3.3)	13 (3.5)		6 (3.3)	8 (4.4)
Hypertension	35 (19.0)	51 (13.9)		35 (19.2)	33 (18.1)
Liver disease	1 (0.5)	1 (0.3)		1 (0.5)	0
Anaemia	4 (2.2)	6 (1.6)		4 (2.2)	6 (3.3)

Open in a separate window

AR, augmented reality.

AR remote surgery implementation process

The portable AR remote system (Xinglian AR system; Beijing, Weizuozhiyuan) with an internet protocol–based connection comprises only two tablets with AR software installed and a stand: one remote surgical guidance terminal and one at the operative end (eFigure1 in the Supplement, Supplemental Digital Content 2, http://links.lww.com/JS9/C713).

In each surgical procedure, the senior surgeon used the AR remote system to supervise junior surgeons remotely as they performed various portions of the surgical procedure. The senior surgeon transmitted gestures, notes, and instructions through the AR remote system. A movable and adjustable plate bracket was affixed to the operating bed. During surgery, a sterile protective sleeve was used to cover the bracket. In addition, a portable AR plate was covered by a transparent sterile protective film and fixed on the plate bracket (Fig. ​(Fig.11 and eFigures 1, Supplemental Digital Content 2, http://links.lww.com/JS9/C713 and 2, Supplemental Digital Content 2, http://links.lww.com/JS9/C713 in the Supplement).

Open in a separate window

Figure 1

The clinical application of the augmented reality (AR) remote system in patellar fracture surgery. (A) Experts can draw virtual incision marks through the AR remote system to guide skin incision remotely. (B) Experts guide fracture reduction remotely through AR virtual annotation. (C). Experts guide fracture internal fixation remotely through AR virtual annotation. (D) Experts transmit gesture information through AR video fusion to guide operations remotely in real time.

Following each procedure, the senior and junior surgeons completed a Likert scale questionnaire (with responses ranging from 1 for “strongly disagree” to 5 for “strongly agree”), assessing their opinions on the effect of the portable AR system on education, safety, and efficiency. In addition, the operating room staff (anaesthesiologist, circulating nurse, and scrub technician) completed a Likert scale questionnaire to evaluate the effect of the portable AR remote system on the surgical procedure and its perceived safety.

The operative time, blood loss, number of fluoroscopies, complications, and functional results were documented. Functional outcomes were evaluated at the 12-month follow-up using the Lysholm (patella fracture), Mayo (elbow fracture), Hospital for Special Surgery (HSS) (tibial plateau fracture), and Kofoed (ankle fracture) rating scales.

Results

The fractures among the 551 patients included those in 132 patellae, 128 elbows, 126 tibial plateaus, and 165 ankles. The mean (SD) age was 48±14.8 years, and 322 patients (58.4%) were women (Table ​(Table1).1). Among the 364 patients (182 patients in each group) matched by PSM, seven (3.8%) from the AR group and six (3%) from the non-remote group developed complications, with a risk difference of 0.005 and 95% CI of −0.033 to 0.044, below the pre-defined non-inferiority margin of a 10% absolute increase. A similar distribution in the individual components of all complications was observed between the AR remote and non-remote groups for ankylosis (0.5% and 0.5%, respectively), nonunion (1.1% and 0.5%), internal fixation failure (1.1% and 0.5%), loss of reduction (0.5% and 1.1%), and heterotopic ossification (1.1% and 1.1%) (Table ​(Table2).2). The adjusted risk difference in all complications between the two groups was 0.5% (95% CI: −3.3 to 4.4) less for the AR remote group, below the pre-defined non-inferiority margin of a 10% absolute increase. A similar distribution in the individual components of all complications was observed between the groups (Table ​(Table22).

Table 2

Analysis of complications after propensity matching score.

	AR remote group (n=182)	Non-remote group (n=182)	Risk difference (95% CI)
Overall complications, n (%)	7 (3.8)	6 (3.0)	0.005 (−0.033 to 0.044)
Separate components of the overall complications, n (%)
Intraoperative accident	0	0	—
Ankylosis	1 (0.5)	1 (0.5)	0 (−0.015 to 0.015)
Wound infection	0	0	—
Vasoneural injury	0	0	—
Nonunion	2 (1.1)	1 (0.5)	0.005 (−0.013 to0.024)
Internal fixation failure	1 (0.5)	0	0.005 (−0.005 to 0.016)
Loss of reduction	1 (0.5)	2 (1.1)	−0.005 (−0.024 to 0.013)
Heterotopic ossification	2 (1.1)	2 (1.1)	0 (−0.021 to 0.021)

Open in a separate window

AR, augmented reality.

Patella fracture

Following PSM, a hierarchical analysis was performed in 105 patients with patella fractures (49 AR remote; 56 non-remote). Compared with non-remote surgery, AR remote surgery showed no statistical difference in operative time (99.55±42.74 vs. 96.63±39.46 min, P=0.65), amount of bleeding (17.63±4.51 vs. 16.86±4.67 ml, P=0.50), number of fluoroscopies (7.86±1.51 vs. 7.71±1.46, P=0.65), injury surgery interval (4.04±2.50 vs. 4.30±2.38 d, P=0.46), complications (4.1% vs. 3.6%, P=0.89), and Lysholm functional results (92.73±3.90 vs. 92.77±4.11, P=0.96) (Fig. ​(Fig.22 and eTable 1 in the Supplement, Supplemental Digital Content 2, http://links.lww.com/JS9/C713).

Open in a separate window

Figure 2

Comparison of operative time, blood loss, fluoroscopy number, functional results, and injury surgery intervals between the augmented reality remote and non-remote groups after propensity score matching. (A) After propensity score matching for all fractures. (B) After propensity score matching for patella fractures. (C) After propensity score matching for elbow fractures. (D) After propensity score matching for tibial plateau fractures. (E). After matching scores for ankle fractures.

Ankle fracture

Following PSM, a hierarchical analysis was performed in 90 patients with ankle fractures (51 AR remote; 39 non-remote). Compared with non-remote surgery, AR remote surgery showed no statistical difference in operative time (132.10±48.85 vs. 129.82±48.87 min, P=0.59), amount of bleeding (87.82±30.10 vs. 85.64±27.46 ml, P=0.76), number of fluoroscopies (6.71±1.03 vs. 6.64±1.06, P=0.79), injury surgery interval (6.43±3.46 vs. 5.72±3.36 days, P=0.33), complications (3.9% vs. 2.6%, P=0.72), and Kofoed functional results (89.33±2.87 vs. 89.31±2.77, P=0.84) (Fig. ​(Fig.22 and eTable 4 in the Supplement, Supplemental Digital Content 2, http://links.lww.com/JS9/C713).

A 1:1 PSM was performed in patients according to the fracture type: 92 patellae (46 AR remote; 46 non-remote), 62 elbows (31 AR remote; 31 non-remote), 58 tibial plateaus (29 AR remote; 29 non-remote), and 92 ankles (46 AR remote; 46 non-remote). Compared with non-remote surgery, the AR remote group showed no statistical difference in operative time, blood loss, fluoroscopy number, complications, and functional results (eFigure 3 and eFigure 4).

The senior and junior surgeons’ Likert scale data showed generally positive results (median Likert scores: 4 to 5 or 5 to 5) for safety, efficiency, and education. Responses to questions regarding reliability, ease of use, sufficient image resolution, safety concerns, communication, understanding of instructions, increasing the surgeon’s confidence, and increasing the surgeon’s sense of security did not differ significantly between the senior and junior surgeons (P≥0.05). When asked about the presence of a perceptible time lag between motions, the mean score for the junior surgeons was 4.32±0.64 compared with 4.47±0.55 for the senior surgeons (P=0.01). Although these mean scores were significantly different, 95% of the junior surgeons’ responses and 98% of the senior surgeons’ responses indicated disagreement or strong disagreement with the statement that a perceptible lag occurred between motions (Fig. ​(Fig.33 and eTable 5 in the Supplement, Supplemental Digital Content 2, http://links.lww.com/JS9/C713). The staff survey results (Fig. ​(Fig.33 and eTable 6 in the Supplement, Supplemental Digital Content 2, http://links.lww.com/JS9/C713) suggested that the AR did not interfere with efficiency or compromise safety (median Likert scores: 4 to 5).

Open in a separate window

Figure 3

Surgeon and operative staff Likert scale questionnaire scores.

Discussion

To our knowledge, this study is the first to report on an AR system with a screen located directly in the surgical visual field for remote guidance during fracture surgery. Our study provides primary clinical evidence regarding this domain. This study presents evidence suggesting that AR remote fracture surgery is as safe and effective as that performed by a specialist in person. In addition, this method appears to enhance the skills and increase the confidence of junior surgeons. The risk difference in all complications between groups favoured telemonitoring and was less than the pre-defined non-inferiority margin. This technique is a promising new medical care model utilising AR remote for fracture surgery, educational purposes, and other open surgical procedures, potentially mitigating the challenges of inadequate medical care in remote areas.

Initially introduced in the 1990s, AR superimposes virtual images onto reality or enhances the virtual world by integrating digital elements into the real world, blending the two, and aligning their positions instantaneously¹⁶. Compared with the traditional video and voice mode, the AR remote mode allows for implementing remote surgical guidance more accurately, quickly, and effectively. The use of AR remote in orthopaedic surgery has been reported only for long-distance education in shoulder arthroscopy via the endoscopy system and one case involving shoulder replacement patients via Google Glass. In addition, studies have used AR glasses for remote lumbar arthrodesis in patients with lumbar degeneration^9,11,13,19. To date, no study has documented using AR remote technology in fracture surgery. In our new AR remote system, the screen is positioned directly in the operating visual field using an adjustable stent at the operating end. This integration aligns the surgeon’s field of view with the AR virtual remote guidance field, enabling real-time remote guidance for fracture surgery. During the operation, surgeons can instantaneously transmit remote virtual guidance commands while utilising real-time annotation, sketching, measurement, and posture information transfer. Thus, with its real-time capabilities, we strongly believe that this innovative remote mode surpasses previous remote methods in efficacy for guiding fracture surgeries remotely. The technology also facilitates long-distance travel medicine, even animal medical education, especially during the 2019 coronavirus outbreak, highlighting its importance^23,24.

For this AR remote system, experts need only a tablet to execute remote guidance anytime or anywhere. The remote mode is very simple and practical; the receiver surgeon in the operating room requires only a tablet and support. Junior surgeons will be able to perform independent operations under the supervision and guidance of experts. This approach increases the junior surgeon’s confidence level, provides a sense of security, and enables more opportunities for practical practice. Both junior and senior experts strongly believe this new remote mode helps cultivate the growth of young doctors. Our analyses showed no difference in surgical time, amount of blood loss, and functional outcomes in treating patellar, tibial plateau, ankle, or elbow fractures when comparing junior surgeons who received remote proctoring and senior surgeons who performed surgeries without remote assistance. These results are consistent with previous studies that explored remote surgery guidance using the AR endoscopic system for arthroscopy, further substantiating its efficacy^9,13. Through this remote mode, patients in areas with poor medical care can receive optimal treatment while avoiding the associated risks and costs of traditional transportation and the inconvenience of requiring medical care escorts. Neither medical care providers nor patients need to travel long distances, saving time, physical strength, and economic costs. Thus, this portable remote mode provides a new strategy to solve the medical care disparities.

Although this study yielded a promising result, our analysis focused mainly on the fracture of the limbs; further research is needed for the fracture of the spine, pelvis, and other parts. Randomised controlled trials with larger samples will be necessary to corroborate these findings. In addition, future studies must consider remote surgery’s legal norms and communication networks. A reliable network communication environment is essential for implementing remote surgery. Fourth-generation networks showed occasional instability in our study, underscoring the need to incorporate fifth-generation technology in future endeavours.

Conclusion

In conclusion, this novel AR remote can allow senior surgeons to remotely guide junior surgeons effectively, safely, and efficiently during fracture surgery with similar results, without the specialist being physically present. It could also help junior surgeons develop the necessary surgical skills and confidence. This technique is a promising new model for remote instruction and education in fracture surgery and other open surgeries, addressing the challenge of inadequate medical care in remote areas.

Ethical approval

The study was approved by the institutional review board (IRB number 2022087501).

Consent

The IRB approved the waiver of the participants’ consent.

Source of funding

This study was supported by the National Natural Science Foundation of China (Grant No.NO.82172524), Major Technology Innovation of Hubei Province(Grant No.NO.2021BEA161 ).

Author contribution

All authors conceived and designed the trial. S.L., M.X., F.G., Y.F. and M.X. ran the clinical study and collected the data. S.L., M.X., F.G., H.-G.C.. A.Y., and Z.Y. drafted the manuscript and all authors edited the manuscript. All authors had access to all of the data in the study and conducted the statistical analysis. Z.Y., H.-G.C., and A.Y. had the final responsibility to submit for publication.

Conflicts of interest disclosure

The authors declare no competing interests.

Research registration unique identifying number (UIN)

This trial is registered with the Chinese Clinical Trial Registry ChiCTR2300075384.

Guarantor

Zhewei Ye.

Data availability statement

The datasets used and/or analysed during the current study are available from the corresponding author upon reasonable request.

Provenance and peer review

Not commissioned, externally peer-reviewed.

Supplementary Material

Footnotes

References