Clinical Validation of a New Enhanced Stent Imaging Method
Round 1
Reviewer 1 Report
The author present a paper about new software for the enhancement of the stent in X-ray angiography images and compare the results qualitative and quantitative to an existing software package. Overall the paper is written quite clear, but from a quite clinical point of view, less from an algorithmic point of view.
Some question about the manuscript:
(72) To what extend is the CAAS StentEnhancer independent of the X-ray angiographic system. For the qualitative analysis, It has only be compared to the Philips StentBoost technology with Philips images using one system. For the quantitative analysis, it is not clear which acquisition systems have been used.
(78) For the SE algorithm, how many images are used for weighted averaging?
(79) The contrast adjustment. Is this automatic or could the user adjust the contrast?
Overall, the level of detail about the algorithm is very limited.
(103) Why qualitative scoring on 0 to 2?
(107) Intra observer variability. Depends on the skill of this person. Which person was used for the intra-observer variability?
(109) How were the region for the SNR defined? Was there a protocol available for selecting the region of interest?
(115) What has been done with the two orthogonal views? It is not clear whether the measurements were combined to calculate average diameters.
The authors report a significant difference in the overall stent visibility between the software packages? What will be the consequence of this?
The quantitative SE analysis. Was this done with the same QCA software, or a modified version adjusted to the SE algorithm?
(243) The authors state that the stent diameter was more in-line with the expected chart diameter. But what was the actual realized diameter? Without intravascular imaging, this will be difficult.
Author Response
We would like to thank the reviewer for his/her constructive and important points.
(72) To what extend is the CAAS StentEnhancer independent of the X-ray angiographic system. For the qualitative analysis, It has only be compared to the Philips StentBoost technology with Philips images using one system. For the quantitative analysis, it is not clear which acquisition systems have been used.
(78) For the SE algorithm, how many images are used for weighted averaging?
(79) The contrast adjustment. Is this automatic or could the user adjust the contrast?
We have addressed the 3 above comments in the same paragraph. CAAS SE is completely independent and runs on a side station. The SE system was used for the quantitative analysis. 40 images are used in the SE algorithm and both automatic as well as manual contrast adjustment can be done. These can be found in lines 75 and after:
“Pie Medical Imaging (Maastricht, The Netherlands) introduced the CAAS StentEnhancer® (SE), a method similar to SB with the main advantage of being completely independent of the X-ray angiographic system of the vendor hence running on a side station. SE uses a maximum of 40 frames from a Digital Imaging and Communications in Medicine (DI-COM) file. Its algorithm automatically detects the stent balloon markers or of the balloon used for post-dilation in order to compute a single image in which the visibility of a deployed stent is improved. Following background subtraction, all frames are transformed into a common reference frame. The resulting images are combined into a single image after weighted averaging. The image is then sharpened and automatically contrast adjusted for the enhanced stent image to be generated. Furthermore, the SE system allows for a manual contrast adjustment of generated images as well as a quantitative assessment of the deployed stent through the measurement of different diameters along the stent length.
Quantitative coronary angiography (QCA) is an objective quantitative tool to evaluate coronary arteries based on the use of dedicated softwares allowing a precise measurement of vessel diameter, percent stenosis, and stent diameters[32]. After image acquisition, a digital quantification of selected frames can be easily performed with or without magnification.
The aim of this study was to qualitatively compare image results from the SE system to the currently available SB and report the results of the measured diameter by the SE system of the deployed stents as compared to the expected chart diameters upon deployment and after post-dilation using SE as well as to final QCA measurements.”
We also added at line 143 that among the different hospitals participating in this study, one used a Siemens X-ray system but we were still able to compute SE on the DICOM files received from that center.
Overall, the level of detail about the algorithm is very limited.
We have added more details that can be found in the previous paragraph.
(103) Why qualitative scoring on 0 to 2?
We have added the following sentence “Images were graded on a scale from 0 to 2 (0 = undetectable; 1 = seen unclearly and 2 = clearly seen) “
(107) Intra observer variability. Depends on the skill of this person. Which person was used for the intra-observer variability?
Both observers are established experienced interventional cardiologists. This was changed in the text : “Two independent blinded and experienced interventional operators compared and graded the stent images obtained by each technique”
(109) How were the region for the SNR defined? Was there a protocol available for selecting the region of interest?
As shown on figure 1, the protocol was relatively straightforward and we clarified it in the methods: a reference noise square of 100 by 100 pixels was manually placed in a region without interventional material (wire, previous stent,…) and without a bone structure such as a rib. Another rectangle was then traced around the stent, as close as possible to the struts. The same two regions were used on the SB and SE images to be compared. The standard deviation σbg of the background pixel values was calculated in the square region of interest (ROI) of noise whereas the average signal value μsig was calculated as the average of the values of the pixels in the ROI traced around the enhanced stent.
(115) What has been done with the two orthogonal views? It is not clear whether the measurements were combined to calculate average diameters.
The mean diameters and mean percent stenosis from both views were used. This was changed in the text at line 145 “Mean stent diameter as well as percent stenosis were calculated in both views and compared to the expected stent chart diameter at the pressure of deployment of the stent and after post-dilatation when available as well as to the QCA measurements. A Bland-Altman analysis was performed between SE and QCA diameters.”
The authors report a significant difference in the overall stent visibility between the software packages? What will be the consequence of this?
The significant difference in stent visibility is different between the 2 observers grading and not the software packages as noted “Table 3 shows the evaluation results of the two observers. Proximal and distal edge visualization grades did not differ between the 2 observers (1.42 ± 0.77 vs 1.38 ± 0.62 and 1.46 ± 0.76 vs 1.44 ± 0.61 for observer 1 and 2 respectively); however a statistically significant difference (p<0.05) was found between the mean total grade by observer 1 vs observer 2 (4.12 ± 1.73 vs 3.67 ± 1.49 respectively) for the evaluation of SB images (Figure 1). A statistically significant difference (p<0.05) was also found between the mean total grade by observer 1 vs observer 2 (4.10 ± 1.86 vs 3.76 ± 1.58 respectively) for the evaluation of SE images with no statistically significant difference noted for the proximal and distal edge visualization grades (1.46 ± 0.79 vs 1.42 ± 0.64 and 1.43 ± 0.77 vs 1.46 ± 0.59 for observer 1 and 2 respectively).”
Although both observers appeared to slightly prefer SE images as compared to SB, this calls for a quantitative analysis as stated “There was no significant difference between the two methods SB or SE used by a Wilcoxon test for each observer (Table 4). A final Kendall test demonstrated a significant difference between the two observers for the assessment of underexpansion and calcifications (Figure 2). While the first reviewer found a correlation of ± 60%, the second one found ± 90% (Table 5). This called for a more quantitative assessment of underexpansion that we validated prospectively in the second part of this research project.”
The quantitative SE analysis. Was this done with the same QCA software, or a modified version adjusted to the SE algorithm?
Two different softwares, the SE software, and the QCA software that is the CAAS workstation as stated in the methods part on line 141 “ESI images were transferred to the SE and QCA workstations (CAAS software v.8.4) and reconstructed.”
(243) The authors state that the stent diameter was more in-line with the expected chart diameter. But what was the actual realized diameter? Without intravascular imaging, this will be difficult.
The reviewer is absolutely right, IVUS remains the gold standard, reason why we added a figure illustrating the advantages of the more precise measurements obtained with intravascular imaging. Not enough of our patients had IVUS for a meaningful subanalysis. ESI was found to have a good correlation with IVUS as stated in the introduction :” SB was found to have good correlation with IVUS regarding stent diameter and found superior to quantitative coronary angiography (QCA)”
Since SE and SB did not differ in our qualitative analysis, we have opted for using SE measurements to compare to the expected chart diameters.
Reviewer 2 Report
The authors presented the paper "Clinical validation of a new enhanced stent imaging method"
1) The reference list should be improved. Many more 2-3 years review papers should be cited in the Introduction section to show the progress in the area. I highly recommend not using references older than 10 years for all sections, except for historically important works.
2) Introduction section should be improved. I highly recommend to insert 2022-2023 year references (review papers) and data about the problem.
3) The discussion section is not clearly present the data and the comparison beetween the results. Authors don't discuss quality of obtained images and parameters. Moreover, no comparison with a literature data are presented.
4) Have you tried to enhance image quality by special programs for image processing? It doesn't looks like the quality of your images becomes much better.
5) I don't see any discussion about Different parameters and stent (Scaffold) material.
6) Conclusion section is poor. The novelty and limitations of the work should be clearly mentioned in the Conclusion section and Abstract.
Minor
Table 4 Observer words are not seen a bit. NS?
Fig 3 increase the text size
Fig 1. Qiality of the figure is poor, low magnification.
Minor editing of English language required
Author Response
We thank the reviewer for his/her valuable time commenting on our work
1) The reference list should be improved. Many more 2-3 years review papers should be cited in the Introduction section to show the progress in the area. I highly recommend not using references older than 10 years for all sections, except for historically important works.
2) Introduction section should be improved. I highly recommend to insert 2022-2023 year references (review papers) and data about the problem.
For the above 2 points : The reviewer is absolutely right, unfortunately there is limited studies/reviews of ESI in the literature recently and that was the idea of our article : to revive the technology for newer and advance the technology. A new reference was however added (ref 32)
3) The discussion section is not clearly present the data and the comparison beetween the results. Authors don't discuss quality of obtained images and parameters. Moreover, no comparison with a literature data are presented.
We have elaborated the discussion and added a Bland-Altman analysis between the SE and QCA measurments: “The results of the two ESI algorithms were compared as per each observer and finally a SNR for the two methods was calculated and compared to an SNR calculated from the angiographic image. ESI has been demonstrated to enhance contrast on fluoroscopic images allowing better visualization of stent struts. This study demonstrated a non-inferiority of SE as compared to SB for the criteria evaluated but a clear interobserver variability calling for more quantitative methods. Despite this difference, both observers had a tendency towards SE images studied parameters. Easily integrated into the procedure, independently of X-ray angiography machine vendor, SE was found in our study to provide good stent expansion assessment as well as a better stent struts visualization. SNR was found to be superior for SE as compared to SB.”
4) Have you tried to enhance image quality by special programs for image processing? It doesn't looks like the quality of your images becomes much better.
The goal of any ESI is image enhancement. We have compared 2 enhanced images originating from angiography (DICOM file) by SB and SE. The quality would not differ much as would be expected but there was a clear tendency towards SE
5) I don't see any discussion about Different parameters and stent (Scaffold) material.
Scaffold material plays an important role in stent visualization hence in ESI evaluation. We have added this to the discussion : “Since newer generation scaffolds tend to use thinner struts or bioresobable material in order to reduce the risk of stent thrombosis in addition to a trend towards the use of lower X-ray power during angiographic procedures, proper stent visualization becomes challenging[3,21–24]. The use of ESI becomes pivotal for the assessment of proper stent expansion, a major risk factor for stent thrombosis[6-10].”
6) Conclusion section is poor. The novelty and limitations of the work should be clearly mentioned in the Conclusion section and Abstract.
We have changed the conclusion as requested: “StentEnhancer is a novel ESI modality that provides an enhanced stent visualization and allows quantitative assessment of stent underexpansion. It is a simple, cost effective and minimally invasive method. We demonstrate that it is non-inferior qualitatively to the market-available and validated StentBoost software providing good stent visualization. StentEnhancer workstation also allowed for a quantitative analysis of images by stent expansion measurement as well as stent underexpansion quantitative assessment correlating well with the expected chart diameter upon stent implantation. A correlation study for the evaluation of StentEnhancer images as compared to IVUS images is needed in order to further validate the StentEnhancer measurements.”
Minor
Table 4 Observer words are not seen a bit. NS?
Thank you for noticing, it was fixed and NS (non significant) added to the legend
Fig 3 increase the text size
This was fixed as requested
Fig 1. Quality of the figure is poor, low magnification.
This was fixed as requested
Reviewer 3 Report
The paper presents the clinical validation of a new enhanced stent visualization method which provides more detailed view of stent deployment. The paper is clearly written, all validations experiments are carefully designed and correctly performed. The method was assessed both qualitatively and quantitatively on a sufficiently large dataset. Thus, the presented conclusions are sound. Only one point needs to be clarified: why SNR is higher for SE compared to SB (as shown in Fig. 3).
Author Response
We thank this last reviewer for his very positive comments and raising this interesting discussion point.
We cannot provide a definite answer to this question, as we are unaware of the exact method of StenBoost. However, based on the available papers, there are methodological differences. For instance, StentBoost does not seem to perform background substraction as it can be seen in Figure 1. We added this comment at line 314.
Round 2
Reviewer 2 Report
Thank you for the revised paper.
Minor editing of English language required