Over the past 5 years, the prevalence and incidence of heart disease have risen due to the longevity of the population, creating a great economic burden for society^[1,2]. A significant transformation has occurred in the field of minimally invasive cardiac surgery, from continuous refinements of the results of minimal access methods to the application of transcatheter solutions for heart disease^[3,4]. With the development of minimally invasive surgery and optical imaging techniques, endoscopic imaging is widely used in the interior of the human body, enabling disease diagnosis and surgical image guidance^[5–9]. For example, nonlinear optical endoscopy was developed for three-dimensional (3D) optical imaging^[10], and fluorescent endoscopy can provide real-time cellular imaging^[11,12]. Optical coherence tomography (OCT)-based endoscopy can be applied in elastic scattering imaging^[13]. In addition, fiber-optic-based two-photon fluorescence endomicroscopy can provide functional neuronal imaging and histological imaging of internal organs^[14–16]. Usually, medical workers can obtain an initial diagnosis according to the spatial images detected with an endoscopy system. However, the propagation of light within the tissue^[17] and optical imaging through blood inside the heart^[18] are significant problems in diagnostic approaches and medical applications.

Angioscopy is an endoscopic imaging system^[19] that includes a flexible optical fiber bundle, a distal lens, and a white light source; angioscopy can achieve direct visualization of the situations inside the blood vessels. Although angioscopy can acquire real-time images during a surgical operation, the process is quite complicated. Once the proximal vessel is occluded with a balloon, normal saline is required to be injected at the region of interest to eliminate blood and obtain a clear view. Meanwhile, angioscopy is limited to a small field of view (FOV), and blood seriously distorts visible light images. Multiple angioscopic runs may be needed when visualizing large areas inside a vessel, making the procedures somewhat less efficient and resulting in the risk of spasm and transient atrioventricular block^[20]. Optical imaging offers the potential to ultrasound and augments fluoroscopic imaging by providing intuitive local images of the intervention site when repairing inside the heart. But, the challenge of optical imaging inside the heart is seeing through the blood.

Visualization through blood has continuously been the subject of intensive investigations^[21,22]. Blood can be mathematically modeled as a particulate suspension. Separation of the cellular elements can be observed when blood is remaining stagnant. The theory of Mie scattering has been used to see through particulate suspensions such as mist, fog, and space dust^[21]. It is necessary to utilize a wavelength long enough to see past the red blood cells that are suspended in the plasma for imaging through blood. The wavelengths of the visible spectrum projected into blood are scattered and absorbed by red blood cells, creating only a red reflection. Nevertheless, the scattering decreases greatly as the imaging wavelength is increased in the infrared spectrum, exceeding the effective optical diameter of the scattering particles^[21]. Consequently, an optical imaging system with an infrared spectrum can see through blood, as erythrocytes constitute the major particulate matter in the blood. But, the optical imaging tradeoff for seeing through blood with longer wavelengths is that the plasma absorption of wavelengths increases as the wavelength increases. Therefore, when designing an infrared imaging system for observing the inside of a heart, the effects of scattering and absorption both needed to be minimized in certain ranges.

With the development of multimodal imaging capabilities and miniaturization of fiber-based devices, fiber-optic endoscopic probes have enabled breakthroughs in medical imaging due to their ability to reach deep into the human body^[22,23]. A challenge faced by fiber-optic endoscopes is the desire to provide a wide FOV and large depth of field (DOF). The lack of a wide FOV and large DOF makes it difficult to quickly survey a large tissue area while delicately performing dissection and suturing, causing surgeons to be pressed for time and requiring more correctional movement. Under the strict requirements of high imaging quality and small spatial size, designing an imaging system with wide FOV and large DOF for detection in heart surgery through blood is quite difficult, especially for an infrared imaging system.

In this paper, a real-time fiber-optic infrared imaging system, named RFIS, has been proposed. We focused on improving the optical structure of the RFIS for wide FOV and large DOF with high relative illuminance over all of the fields. This paper is organized as follows. The design concept and specification are described in Section 2. In Section 3, experiments and analyses of the optical properties are presented. Section 4 illustrates the tolerance analysis, and then the conclusion is presented in Section 5.

The conceptual system diagram is shown in Fig. 1. The RFIS is designed to simultaneously achieve both wide FOV and large DOF infrared imaging with high quality. The RFIS mainly includes an objective lens (OL), laser source, fiber bundle, coupling lens (CL), illuminance fibers, short wavelength infrared (SWIR) camera, and display device. The RFIS is designed with optimized structures so that a compact diameter package could be adapted for housing the OL and CL along with the fiber bundle; this facilitates the rapid adoption of the developed technique for clinical application. The light source is from a laser source (ES-7700-0001, QPC Lasers). The imaging fiber bundle (A2 Fiber, SCHOTT) has a high numerical aperture (NA) of 0.22, a section diameter of 3 mm with 10,000 optical fibers, and a single-core diameter of 27.5 µm, which makes it greatly suited to be coupled to the optical device. The order packing structure of the fiber bundle is a hexagonal structure, and the maximum spatial resolution of the fiber bundle is 21 lp/mm. The sensitive area of the SWIR camera (Xeva-2.5-320, Xenics) is 9.6mm×7.68mm, the resolution is 320×256, and the dimension of a pixel is 30µm×30µm.

The fiber bundle acquires real-time imaging of the diseased areas through an OL and projects the images to the SWIR camera through the CL during the surgery. Additionally, the designed system is optimized for a wide FOV of 120° and a long working distance (WD) of 8 to 25 mm. Thus, the efficiency of intracardiac visualization and surgery can be improved significantly.

The optical design of the RFIS system was quite challenging due to the wide FOV, large DOF, limited lens diameter, high relative illuminance, and telecentric requirements in addition to the need to match different optical components.

Figure 2(a) shows the cross-sectional schematic of the OL. As shown in Table 1, the OL was designed to have a focal length of 0.797 mm, an F-number of three, a total track length of 13.32 mm, and an FOV of 120°. Generally, the image space NA in the OL should match the acceptance NA in the fiber bundle to ensure that the incident rays that are both on the axis and outside the axis symmetrical to the chief rays can project from the fiber bundle without loss. The telecentric optical structure provides constant image magnification regardless of the object and/or image position, eliminating the perspective angle error^[24]. Furthermore, to obtain a wide FOV, small size, and large DOF without major obscurity, the OL was designed with a reversed telephoto telecentric structure in the image space to successfully track all of the parallel principal rays and reduce the FOV pressure in the design. The OL consists of five elements, the front-end negative lens L1 can collect and bend rays with wide FOVs effectively, and the two-doublet combination has good performance on aberration correction. The calculation and optimization of the OL were performed by Zemax (OpticStudio, USA). Figure 2(b) shows the polychromatic diffraction modulation transfer function (MTF) curves for each configuration. MTF is an important parameter for the quality evaluation of an optical system, which is described as^[25]MTF=(Imax−Imin)/(Imax+Imin), where Imax and Imin represent the highest and lowest luminance, respectively. The cutoff frequency of the MTF was calculated to be 21 lp/mm according to Nyquist’s theorem^[26]. The MTF values for all of the FOVs at 21 lp/mm are above 0.759, 0.797, 0.827, 0.826, and 0.820 and are all close to the diffraction-limited cases, which demonstrates that the designed OL enables excellent image quality. Figure 2(c) shows the field curvature and distortion at different wavelengths and for different FOVs. The distortion patterns for all of the configurations are quite similar, and the differences are negligible. The distortions of all the FOVs are less than −15%, which can be effectively calibrated through post-image processing and quantification. The spot diagram is another important parameter for the OL. A smaller spot diagram represents better imaging quality of the optical system.

Figure 2(d) shows the spot diagram of the OL. Different colors in the spot patterns indicate the different wavelengths, including 1.2 µm, 1.5 µm, and 1.8 µm. These spots are scaled with the Airy disk at 1.5 µm (dark circle), whose radius is 1.22λ/(2NA) for a uniformly illuminated, circular entrance pupil^[25]. The Airy disk is 5.868 µm in diameter, and the resulting RMS values of the spot radius are 3.377, 3.980, 4.904, 5.718, and 4.643 µm, which indicates that, basically, all the rays are well within the Airy disk and that the designed lens is said to be diffraction-limited. The relative illuminance curve shown in Fig. 2(e) is computed as the ratio of the corner luminance to the center luminance^[27], which is affected by distortion, vignetting, and pupil aberration. The relative illuminance values are higher than 0.98 in all fields, as the telecentric structure of the designed image space can reduce the change in relative illuminance on the focal plane. Furthermore, the wavefront map at an FOV of 60° is taken to evaluate the imaging quality of the OL, as shown in Fig. 2(f). In theory, the peak-to-valley (P-V) value of the wavefront should be less than λ/4, which represents the high optical quality of the imaging system. The P-V value of the wavefront of the OL is less than 0.1825λ, which indicates that the imaging quality of the designed OL can be guaranteed. The CL was designed to transmit the rays from the fiber bundle to the SWIR camera.

Figure 3(a) shows the cross-sectional schematic of the CL. To meet the condition where the incident beam of the fiber bundle can be collected perfectly by the CL without loss, the CL was designed with a telecentric structure in the object space. As presented in Table 2, the designed CL has a focal length of 4.370 mm, the NA in the object space is 0.25, and the FOV is 28.95°. Figure 3(b) presents the MTF of the CL. The cutoff frequency is calculated to be 16.7 lp/mm, and all of the fields at 16.7 lp/mm are above 0.661. The high flatness of the optimized MTF demonstrates that a high uniformity can be obtained for the CL.

To further improve the optical imaging performance of the RFIS, the system was optimized at three different WDs to cover a feasibly large DOF. The largest WD for optimization is set at 25 mm, which is larger than the design requirements. The objective space is filled with blood. The optimization results are shown in Fig. 4 and consist of three WDs of 8, 15, and 25 mm covering five fields. The RFIS has good contrast over most of the field at 21 lp/mm, and the MTF values are above 0.74 when the WDs are 8 and 15 mm, respectively. Additionally, when the WD is 25 mm, the MTF values are above 0.7. The relative illuminance values are higher than 0.95 in all fields, the RMS values of the spot radius are below 10.492 µm, and the distortion of all configurations is less than −18%. The results significantly indicate that the optimized system is diffraction-limited, with small spherical aberration and residuals of axial color; further demonstrating that the RFIS can work with wide FOVs and large DOFs while quickly surveying a large operating area so that the detection can be delicately performed.

To further demonstrate the feasibility of the RFIS, we imaged the USAF 1951 resolution board in Zemax. The test image has a nominal WD of 10 mm, and the result is shown in Fig. 5(b), where the details of the center and edge of the detection areas can be observed. Meanwhile, the distortion appears in the wide-angle optical system inevitably, and it is complicated to reduce distortion while still maintaining a wide FOV^[28]. In this Letter, a small distortion is achieved, while the distortion would not affect the imaging clarity of the RFIS. The distortion analysis of the RFIS was done according to Standard Mobile Imaging Architecture (SMIA) TV distortion, which is given by^[29,30] SMIA TV distortion (%)=[(h′−h)/h]×100, where h′ denotes the half-height at the image corner, and h represents the half-height on the center. Maximum SMIA TV distortion is −15%. As seen in Fig. 5(c), the distortion can be corrected effectively by the image processing algorithm in real time. The distortion correction rate is 90.20%, and the average distortion rate after correction is −1.47%, which significantly meets the best imaging quality requirements.

From the preceding analysis, we designed an RFIS that can meet the specification and possesses good optical quality. However, the feasibility of manufacturing needs to be confirmed through a tolerance analysis^[26,31]. In the tolerance analysis, the MTFs of the Nyquist frequencies of 21 lp/mm for the OL and 16.7 lp/mm for the CL are used as the merit function. First, the RFIS is analyzed with inverse sensitivity analysis. Second, the proximate ranges of manufacturing and assembling tolerance of the RFIS can be obtained. Then, the tolerance ranges were used to perform sensitivity analysis, and some sensitivity tolerances must be further restricted. Monte Carlo simulation is a numerical method guided by probability theory and statistical theory, and it is one of the random sampling experiment methods. Finally, there are 5000 samples for the Monte Carlo analysis, and every sample simulated five fields: on-axis, 0.3 field, 0.5 field, 0.707 field, and full field. The number of rays tracked in the simulation is 8 million. The optical constants of blood were set in the software of Zemax, the extinction coefficient is 0.0010660, the absorption coefficient is 66.959cm−1, the chromatic dispersion is 0.014249µm−1, the transmittance is 0.99333, and the reflectance of non-polarized lights is 0.017109.

Table 3 presents the tolerance distribution for the OL and CL. Figure 6(a) shows the tolerance analysis results for different surfaces of the OL. TSDY denotes the decentration of the surface. TEDY represents the decentration of the lens in the Y direction. The TSDY values for surfaces 8 and 9 and the TEDY values for surfaces 4 to 6 have a great effect on the performance of the MTF, while all of the top ten changes in the MTF drop are less than 0.1. The results indicate that the surface decentration control is the most significant factor to control so that a high optical imaging quality for the OL can be obtained. Furthermore, the Monte Carlo analysis was performed in different fields. As shown in Fig. 6(b), the analysis results indicate that the MTF is greater than 0.76, 0.73, and 0.71, with a possibility of obtaining 90% in the central field, 0.3 field, and 0.5 field, respectively. Additionally, for 0.707 field and the full field, the MTF can only be maintained above 0.61 and 0.43, respectively, in different fields. As shown in Fig. 6(b), the analysis results indicate that the MTF is greater than 0.76, 0.73, and 0.71, with a possibility of obtaining 90% in the central field, 0.3 field, and 0.5 field, respectively. Moreover, Fig. 6(c) presents the tolerance analysis results of the CL, and the Monte Carlo results are shown in Fig. 6(d). TFRN denotes the radius tolerance, TTHI represents the thickness tolerance, and TIRY represents the inclination of the surfaces in the Y direction. The results show that the TFRN value for surface 9 and the TSDY value for surface 9 mostly affect the MTF drop, while the MTF is greater than 0.61 with a possibility of 90% in the central field, 0.58 for the 0.5 field, and 0.53 for the 0.707 field with a possibility of 90%. Therefore, we have achieved a reasonable MTF for the manufacturing and assembly tolerances of the designed RFIS. Furthermore, all of the elements are designed to be spherical, which greatly saves the manufacturing cost and reduces the processing time.

To further demonstrate the feasibility of the RFIS in imaging biological tissue through blood, we performed an imaging simulation to see the change in image quality. The corresponding gross anatomy of the tricuspid valve in a postmortem examination from Ref. [21] and the tricuspid valve from an autopsy of the right ventricle from Ref. [32] was simulated. The test images have a nominal WD of 10 mm, and the results are shown in Figs. 7(a) and 7(c), respectively, where the details of the center and edge of the detection areas can be observed. Meanwhile, the distortion appears in the wide-angle optical system inevitably, and it is complicated to reduce distortion while still maintaining a wide FOV. As seen in Figs. 7(b) and 7(d), the distortion is corrected effectively by the image processing algorithm in real time.

The design of an RFIS is presented in this paper, which mainly includes OL, fiber bundle, CL, laser source, display device, SWIR camera, and illuminance fibers. The RFIS performs well with wide FOVs, large DOFs, and high relative illuminance in the overall fields. Firstly, the experiments and analysis of the RFIS are performed by the software of Zemax. The assessment criteria of the imaging quality for the OL and CL, such as MTF values, spot diagram, and relative illuminance, have been comprehensively optimized and analyzed. Then, we further evaluate the performance of the RFIS with different WDs, and the results demonstrate a large DOF of the designed system. Moreover, to confirm the feasibility of manufacturing and assembly, tolerance and Monte Carlo analyses have been performed, further guaranteeing the rapid adoption of the developed technique for clinical application. Finally, we simulate and evaluate the imaging quality, and the distortion is corrected effectively by the image processing algorithm in real time. Therefore, the proposed RFIS can achieve a large WD of 8 to 25 mm, a wide FOV of 120°, and the relative illuminance is over 0.98 in the overall FOV.