Open AccessArticle

An Unsupervised Approach for Treatment Effectiveness Monitoring Using Curvature Learning

Hersh Sagreiya

^1,2,*

Isabelle Durot

^2,3 and

Alireza Akhbardeh

^2,4,5

Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA

Department of Radiology, Stanford University School of Medicine, Stanford, CA 94305, USA

Department of Radiology, Regionalspital Emmental Burgdorf, 3400 Burgdorf, Switzerland

⁴

Department of Diagnostic and Interventional Imaging, The University of Texas Health Science Center, Houston, TX 77030, USA

⁵

Ambient Digital LLC, Daly City, CA 94014, USA

Author to whom correspondence should be addressed.

Computers 2024, 13(9), 227; https://doi.org/10.3390/computers13090227

Submission received: 17 March 2024 / Revised: 3 September 2024 / Accepted: 4 September 2024 / Published: 9 September 2024

(This article belongs to the Special Issue Machine and Deep Learning in the Health Domain 2024)

Download

Browse Figures

Versions Notes

Abstract

Contrast-enhanced ultrasound could assess whether cancer chemotherapeutic agents work in days, rather than waiting 2–3 months, as is typical using the Response Evaluation Criteria in Solid Tumors (RECIST), therefore avoiding toxic side effects and expensive, ineffective therapy. A total of 40 mice were implanted with human colon cancer cells: treatment-sensitive mice in control (n = 10, receiving saline) and treated (n = 10, receiving bevacizumab) groups and treatment-resistant mice in control (n = 10) and treated (n = 10) groups. Each mouse was imaged using 3D dynamic contrast-enhanced ultrasound with Definity microbubbles. Curvature learning, an unsupervised learning approach, quantized pixels into three classes—blue, yellow, and red—representing normal, intermediate, and high cancer probability, both at baseline and after treatment. Next, a curvature learning score was calculated for each mouse using statistical measures representing variations in these three color classes across each frame from cine ultrasound images obtained during contrast administration on a given day (intra-day variability) and between pre- and post-treatment days (inter-day variability). A Wilcoxon rank-sum test compared score distributions between treated, treatment-sensitive mice and all others. There was a statistically significant difference in tumor score between the treated, treatment-sensitive group (n = 10) and all others (n = 30) (p = 0.0051). Curvature learning successfully identified treatment response, detecting changes in tumor perfusion before changes in tumor size. A similar technique could be developed for humans.

Keywords:

machine learning; unsupervised learning; curvature learning; ultrasound; contrast-enhanced ultrasound; cancer; treatment monitoring; early diagnosis

Graphical Abstract

1. Introduction

The Response Evaluation Criteria in Solid Tumors (RECIST) is a widely used set of criteria to determine tumor response to therapy [1]. However, these criteria are based on tumor size, and during treatment, there is not always a measurable change in tumor size despite tumor response to therapy [2,3]. Furthermore, the first RECIST follow-up evaluation usually happens 6–8 weeks after therapy initiation at the earliest [4]. Several studies showed dynamic contrast-enhanced ultrasound (DCE-US) to be useful for monitoring treatment response, especially in patients undergoing anti-angiogenic chemotherapy [5,6]. This form of chemotherapy curtails the delivery of oxygen and nutrients to tumor cells by obliterating their vascular supply by targeting vascular endothelial growth factor (VEGF) [7]. As anti-angiogenic therapy suppresses tumor cell growth by inhibiting vascular supply, decreases in tumor size under this therapeutic regime often take longer than other forms of chemotherapy [8].

DCE-US has several advantages and a few disadvantages compared to cross-sectional modalities for tumor response evaluation, such as CT, MRI, and PET-CT [9]. Unlike CT and PET-CT, there is no ionizing radiation. DCE-US is also portable and can be performed at the patient’s bedside, is less expensive than alternative imaging modalities, and is widely available. DCE-US does not cause nephrotoxicity, an advantage for patients with renal failure [10]. DCE-US has been investigated for enhancing drug delivery into tumors and increasing tumor sensitivity to radiotherapy [11]. Hence, a physician could simultaneously diagnose and treat a tumor on the same clinical visit. Regarding disadvantages, other modalities such as CT can provide a quick survey of a large amount of patient anatomy across multiple organs, whereas ultrasound is typically more targeted. In addition, the finding under investigation needs to be accessible by ultrasound, an issue for deeper lesions, especially if they are surrounded by loops of bowel. However, DCE-US is ideally suited for early tumor response evaluation in lesions easily accessible by ultrasound, such as those in the liver and kidneys. DCE-US can detect changes in tumor perfusion as a response to treatment, allowing the early detection of cancer treatment effectiveness even before there is a change in tumor size [8].

Machine learning has recently become popular for medical applications, including in monitoring responses to cancer therapy [12,13,14]. Treatment effectiveness monitoring most commonly employs supervised machine learning. Three major categories of features that are assessed in this way are pixel-intensity features (mean, median, skewness, and kurtosis), hemodynamic parameters (peak enhancement, time to peak, mean transit time, and area under the curve), and texture/shape analysis (gray level co-occurrence matrix, wavelet features, and fractal-based features). Subsequently, feature selection techniques are used, such as least absolute shrinkage and selection operator (LASSO) regression [15] or principal component analysis (PCA) [16], and a supervised learning algorithm is subsequently run.

However, the current approach presents challenges. First, it requires manual tumor segmentation, which is time consuming even for expert readers. This is particularly burdensome for 3D dynamic contrast-enhanced ultrasound, as multiple planes must be segmented. Three-dimensional DCE-US has advantages over conventional two-dimensional DCE-US, as it is less prone to sampling error related to subtle deviations in transducer positioning [17]. This is important if the robust and reproducible quantification of tumor perfusion is desired. Since a busy ultrasound clinic may image over fifty patients per day for each radiologist, manual segmentation of tumor volumes is impractical, hindering the clinical translation of these techniques. Moreover, manual segmentation introduces inter-observer variability. Supervised machine learning requires an adequate sample size for training and validation and does not perform well with small sample sizes, leading to overfitting and lacking generalizability to external datasets [13]. The current approach also requires feature selection and is sensitive to the specific features that are chosen. Finally, the current approach is complex and requires multiple steps, including tumor segmentation, perfusion modeling, feature extraction and selection, and finally machine learning. More recently, deep learning has increased in popularity, which obviates the feature selection process. However, it is sensitive to small datasets, and its “black box” approach makes it hard to understand the reasoning behind its predictions [18]. Deep learning also typically employs supervised learning, while the algorithm presented in this paper uses an unsupervised learning approach.

We present an unsupervised approach using curvature learning that addresses several of these concerns [14,19], which is comprehensively described in Section 2.2, Section 2.3 and Section 2.4. As an unsupervised method that does not require training, it is not subject to overfitting, a major concern for small sample sizes when using supervised machine learning. This technique requires far fewer steps than traditional supervised learning, as it does not require manual segmentation, saving time and eliminating inter-observer variability, and it does not require feature selection, eliminating feature dependency. This method calculates perfusion parameters on a pixel-by-pixel basis without needing curve fitting to existing perfusion models [20,21]. The utility of this approach, which uses dimensionality reduction, was previously described for breast cancer cases undergoing breast MRI [22]. In this paper, the unsupervised curvature learning approach was applied to dynamic contrast-enhanced breast MRI scans, and the technique successfully distinguished breast tumors from normal background tissue. Therefore, the purpose of this study was to develop a new algorithm and decision support tool for the quantitative and noninvasive assessment of treatment response using contrast-enhanced ultrasound.

2. Materials and Methods

2.1. Mouse Experimental Protocol

All procedures were approved by the Administrative Panel on Laboratory Animal Care (APLAC) at Stanford University under protocol #21637, entitled “The use of contrast-enhanced ultrasound with microbubbles for applications in cancer imaging and therapeutic delivery”. Forty mice (Charles River Laboratories, Wilmington, MA, USA) implanted with colon cancer tumors were imaged via DCE-US using an EPIQ 7 ultrasound scanner with an X6-1 matrix array transducer (Philips Healthcare, Andover, MA, USA) and injected with perflutren (Definity) microbubbles (Lantheus Medical Imaging Inc., North Billerica, MA, USA), as previously described [2]. These microbubbles were diluted 1:4 in a sterile 0.9% saline solution after activation, and a 120 µL suspension was delivered in a five-second bolus at an injection rate of 24 µL/sec using an injection pump. Twenty mice were implanted with the LS174T tumor (American Type Culture Collection, Manassas, VA, USA) that responds to the anti-angiogenic drug bevacizumab (Avastin, Genentech, South San Francisco, CA, USA), a monoclonal antibody targeting VEGF. Twenty were implanted with the CT26 tumor (American Type Culture Collection, Manassas, VA, USA) that does not respond to the drug. For both the responder and non-responder groups, ten mice were treated with bevacizumab and ten served as controls that were administered saline. This resulted in four groups of ten mice: treatment-sensitive, bevacizuab-treated mice (LSBolusAV); treatment-sensitive, saline-treated mice (LSBolusCTRL); treatment-resistant, bevacizumab-treated mice (CTBolusAV); and treatment-resistant, saline-treated mice (CTBolusCTRL). In this nomenclature, the first set of capitalized letters represents the cell type (LS174T vs. CT26) and the second set of capitalized letters represents the type of treatment given (chemotherapeutic agent bevacizumab/Avastin vs. saline control). Mice administered bevacizumab can be considered “treated”, whereas mice administered saline can be considered “untreated.” Only the treatment-sensitive, bevacizumab-treated mice (LSBolusAV) were expected to respond to treatment. As shown in Figure 1, before treatment (day 0: baseline) and on post-treatment days 1, 3, 7, and 10, the tumors were imaged with DCE-US. Of note, prior research using this dataset showed that there was no change in tumor size between day 0 and post-treatment day 1 [2]. After day 3, groups with untreated tumors showed a progressive increase in size compared to the treated group.

Further description and details about the original experimental setup can be found in [2].

2.2. Dimensionality Reduction

Data analysis was performed using MATLB R2015b (The MathWorks, Inc., Natick, MA, USA). To detect the underlying data structure within DCE-US images, we applied dimensionality reduction, a method that transforms the high-dimensional DCE-US images into a lower-dimensional representation reflecting the data’s intrinsic dimensionality. This representation denotes the least amount of data that can meaningfully represent the true data structure, reducing redundant data whilst maintaining data variance [22]. The most commonly used dimensionality reduction methods are linear, assuming that the data are within a linear subspace of a higher-dimensional topological space, and these methods include principal component analysis (PCA) [23] and linear discriminant analysis (LDA) [24]. However, nonlinear dimensionality reduction techniques do not make this assumption, allowing for more complex representations of higher-dimensional data. A previous work analyzed both linear and nonlinear dimensionality reduction methods on dynamic contrast-enhanced MRI, finding that nonlinear dimensionality reduction methods were superior and that amongst the dimensionality reduction techniques, diffusion maps performed particularly well [22].

2.3. Diffusion Maps

Diffusion maps take inspiration from physics [22,25,26]. Conceptually, pixel intensities correspond to the diffusivity of different particles. Diffusion maps locate the subspace that most effectively preserves diffusion interpoint distances based on a Markov random walk on a graph of the data known as a Laplacian graph [22,26]. A Gaussian kernel function is used to estimate the weights K of the graph edges, which are defined by

K_{i j} = e^{\frac{‖ x_{i} - x_{j} ‖^{2}}{2 σ^{2}}} i \geq 1, j \leq L

(1)

in which L represents the number of multidimensional points and σ is a free parameter. Next, normalization is performed such that the rows of the matrix K sum to 1 by the following equation:

p_{i j}^{(t)} = \frac{K_{i j}}{\sum_{n = 1}^{L} K_{i n}}

(2)

in which p represents the forward transition probability of t time steps of a random walk from one data point to another. Lastly, the diffusion distance is defined as the following:

D_{i j}^{(2)} = \sum_{r = 1}^{L} \frac{{(p_{i r}^{(t)} - p_{j r}^{(t)})}^{2}}{ψ (x_{r})}

(3)

ψ (x_{m}) = \frac{\sum_{j = 1}^{L} p_{j m}}{\sum_{k = 1}^{L} \sum_{j = 1}^{L} p_{j k}}

(4)

The high-density parts of the graph defined by the diffusion distance are weighted more heavily, whereas pairs of data points with a large forward transition probability have a smaller diffusion distance. Diffusion distance is less sensitive to noise than geodesic distance, using numerous paths within the graph to generate the embedded image. Using spectral theory, the embedded image Y can be generated by using d nontrivial eigenvectors of distance matrix D.

Y : x \mapsto {λ_{2} V_{2}, λ_{3} V_{3}, \dots λ_{n} V_{n}}

(5)

The first eigenvector V₁ of the largest eigenvalue (λ₁) is discarded, and the remaining eigenvectors are normalized by their eigenvalues.

2.4. Generation of the Embedded Image and Scattergram

The embedded vector was created by projecting pixel intensities from higher-dimensional space to a one-dimensional embedding space using diffusion maps for dimensionality reduction. The embedded image, or fused heat map, was then formed by taking this embedded vector and changing it to size m x n, the original dimensions of the image. Essentially, the higher-dimensional space has been mapped to a 2D space, creating an unfolded version of the original data manifold [27,28,29]. To create the scattergram, the higher-dimensional space is mapped to a 2D embedded space. This is then quantized into three classes for tissue characterization—blue, yellow, and red—representing normal, intermediate, and high probability for cancer, respectively. This procedure was conducted for all mice both at baseline and post-treatment. To compare groups of mice, statistical features were calculated, described in the next section. Figure 2A shows the data analysis pipeline, Figure 2B shows examples of the embedded image and the scattergram generated by the curvature learning algorithm, and Figure 2C demonstrates how statistical features from different days were used to calculate intra-day curvature learning scores and the inter-day curvature learning score.

2.5. Statistical Analysis

To compare the four groups of mice, a tumor score termed the curvature learning score (CLS) was calculated, representing the variation in the different color classes across each image frame from the ultrasound cine images obtained during contrast administration on a given day (intra-day variability) and the variation between the pre-treatment and post-treatment days (inter-day variability). For this discussion, statistical measures include, but are not limited to, “mean”, “median”, “max”, “mode”, and “min”. The mode is the most frequently occurring value in the dataset. We tested different combinations of these statistical features to calculate the CLS.

The following steps were performed. (1) Intra-day curvature learning score: on a given treatment day, first calculate the percentages for the three color-coded clusters (blue, yellow, and red) for each time point and then calculate the statistical measures over time. (2) Inter-day curvature learning score: across different days, calculate the statistical measures comparing the previously collected intra-day data. (3) Final curvature learning score: this score represents both intra-day variability and inter-day variability for a given mouse. (4) The distribution of scores between treated, treatment-sensitive mice and all other groups was compared using a Wilcoxon rank-sum test.

3. Results

A total of 40 mice were imaged in the following categories: treatment-sensitive mice in control (N = 10, receiving saline) and treated (N = 10, receiving bevacizumab) groups and treatment-resistant mice in control (10) and treated (10) groups. Each mouse was imaged using contrast-enhanced ultrasound on five different days. The curvature learning algorithm proposed in this paper, along with the pipeline described in Figure 2A, was used to characterize each pixel in the 3D image, and values were quantized into three classes: red, yellow, and blue. This procedure was repeated for all mice at baseline and on post-treatment days.

Figure 3 shows a characteristic example of a treated, treatment-sensitive mouse (LSBolusAV) on days 0 (baseline), 3, and 10. Yellow and red colors represent increasing suspicion for tumor, while dark blue represents normal tissue. In the fused heat maps on the left, there is a decrease in red and yellow signals within the tumor over time. In the bar plots on the right, there is a decrease in the yellow and red color classes over time.

Figure 4 shows an untreated, treatment-resistant mouse (CTBolusCTRL) on days 0 (baseline), 3, and 10. There is an increase in the yellow and red colors (representing tissue more suspicious for tumor) during the treatment course both in the fused heat maps and the bar plots. Once again, dark blue represents pixels with normal tissue.

To compare between the four groups of mice, a tumor score termed the curvature learning score (CLS), explained in Section 2.5, was calculated for each mouse. This score encompasses intra-day and inter-day variation in the different color classes. The distribution of scores between each group of mice was compared using a Wilcoxon rank-sum test, as shown in Figure 5. This figure compares the mice pre-treatment and three days after treatment.

Finally, we compared the first day after treatment with the pre-treatment day, as there was no significant change in tumor size on day one [2]. After examining differences in the intra-day curvature learning scores between the baseline exam and the first exam post-treatment, there was a statistically significant difference between the final curvature learning score (CLS) for the treated, treatment-sensitive group and all other groups (p = 0.0051), as shown in Figure 6. Treatment-resistant mice, as expected, did not respond to treatment, which was demonstrated quantitatively by the CLS. Treatment-sensitive mice, as expected, responded to treatment, also demonstrated quantitatively by the CLS.

4. Discussion

This study showed that there was a significant difference between treated, treatment-sensitive mice and all other groups after treatment with the chemotherapeutic agent bevacizumab. This was evident by measuring differences in tumor perfusion in a fully unsupervised fashion. By treatment day 1, there was a clear difference in tumor perfusion despite there being no significant difference in tumor size. This contrasts with traditional follow-up using the RECIST, the current basis of cancer treatment effectiveness monitoring, which typically takes months and relies on differences in tumor size.

The original study using this dataset involved the analysis of perfusion parameters [2]. For each of the forty mice, a volume of interest (VOI) was manually created, and then a time–intensity curve was generated that measured how the average signal intensity within the VOI varied with time. After curve fitting was performed to the lognormal distribution, the following perfusion parameters were obtained: peak enhancement (PE), area under the curve (AUC), and time to peak (TTP). The nonparametric paired-samples Wilcoxon rank-sum test was used to assess changes in perfusion parameters. The investigators found that for the responding group, there was a statistically significant difference in the AUC and PE between day 0 and subsequent post-treatment days (p = 0.005), while saline-treated tumors did not show such a difference. Relative changes in the AUC and PE were significantly lower for tumors treated with bevacizumab as opposed to saline on post-treatment days (p < 0.05). Interestingly, there were no differences in the TTP between mice treated with bevacizumab or saline.

However, our method has several advantages. First, it is entirely automated and does not require generating a pre-defined region of interest, which can be time consuming and technically challenging, particularly for three-dimensional data. Given the high volumes of studies that radiologists read and knowing that radiologists do not segment tumors on 3D ultrasound data, such an analysis would be impractical in a busy radiology practice. Moreover, generating a region of interest is a manual step that could introduce human error and inter-observer variability. Like the current study, most studies in the literature have focused on oncologic applications. However, they overwhelmingly used supervised learning, either with traditional machine learning or deep learning. The advantages of our unsupervised learning approach include no requirement for time-consuming manual segmentation, no dependence on a large dataset for training, no concern for overfitting, no feature dependency, and avoidance of the “black box” issue with deep learning algorithms that can make it hard to understand why models make their predictions.

In the literature, the liver has been a common application for machine learning in DCE-US. One study sought to distinguish between benign and malignant focal liver lesions using a convolutional neural network (CNN) based on the ResNet architecture [30]. The algorithm had a receiver operating characteristic area under the curve (ROC AUC) of 0.934 on the test set. Another study also sought to differentiate between benign and malignant focal liver lesions on contrast-enhanced ultrasound, combining features obtained from both spatiotemporal analysis and texture analysis performed at different time points, using several machine learning classifiers on these input features [31]. While most models (logistic regression, support vector machine, random forest, and k-nearest neighbor) performed well, a soft voting classifier combining multiple models obtained the highest balanced accuracy at 0.84. Another study used deep learning to predict microvascular invasion with hepatocellular carcinoma, a key factor related to survival [32]. The best-performing model, ResNet50 with the Bottleneck Attention Module, had an ROC AUC of 0.856. Another study used deep learning combined with radiomics for contrast-enhanced ultrasound to predict progression-free survival after radiofrequency ablation (RFA) and surgical resection (SR); this study sought to optimize the selection between those treatments for patients with early-stage hepatocellular carcinoma [33]. The c-index, which measures performance for survival models, was 0.726 for RFA and 0.741 for SR. Moreover, 17.3% of RFA patients and 27.3% of SR patients would have benefited from switching treatment, on average increasing their 2-year progression-free survival by 12% and 15%, respectively. Overall, machine learning generally performed well for several hepatic applications.

Machine learning applied to contrast-enhanced ultrasound has also been investigated for non-hepatic applications. One study used a deep learning radiomics model based on a ResNet50 backbone to distinguish between pancreatic ductal adenocarcinoma and chronic pancreatitis [34]. On two external validation cohorts, the model demonstrated an ROC AUC of 0.967 and 0.953, respectively. Another pancreatic application sought to predict the efficacy of neoadjuvant chemotherapy for pancreatic ductal adenocarcinoma [35]. Contrast-enhanced ultrasound videos were collected prior to chemotherapy initiation to predict efficacy after chemotherapy. A CNN using videos from both conventional ultrasound and contrast-enhanced ultrasound of the pancreas had an ROC AUC of 0.892, while a second CNN using videos solely of the region of interest with contrast-enhanced ultrasound had an ROC AUC of 0.908. Another clinical application used supervised machine learning with a support vector machine to classify breast tumors as benign or malignant, with an ROC AUC of 0.893 [36]. A prostate cancer application examined molecularly targeted contrast-enhanced ultrasound using a targeted anti-PSMA agent to detect prostate cancer in mouse xenografts [37]. A 3D CNN outperformed other techniques, with an accuracy over 90%. Another prostate cancer application used clinical features, time–intensity curve features, and radiomics features from conventional transrectal ultrasound and contrast-enhanced ultrasound for prostate cancer detection in the peripheral zone [38]. The model combining all these features performed best, with an ROC AUC of 0.89. Overall, the techniques in the literature generally employed traditional machine learning or deep learning using supervised learning, unlike the unsupervised learning approach presented in this paper. In addition, this study focused on early cancer diagnosis by automatically analyzing changes in tumor perfusion.

One study limitation was the sample size of 40, which could be expanded in a future study with a larger sample size. Nevertheless, there was a clear statistical separation of the responding and non-responding groups, which was evident on the first day after treatment. Moreover, obtaining well-controlled data of this variety is technically challenging, time consuming, and expensive. Another potential limitation is that while this technique worked well on this dataset, further research is needed to investigate its scalability to larger multi-institutional datasets or to other forms of cancer. However, this technique was also previously applied to dynamic contrast-enhanced breast MRI [22], which also relied on measuring changes in perfusion. Hence, we hypothesize that our technique could potentially be applied to any study in which there are measurable changes in perfusion, such as when investigating early cancer treatment effectiveness, although additional research is warranted. Another potential future direction involves expanding this study to multiple institutions, noting that external sites would need to follow a similar general protocol with respect to cell lines and treatments. This multi-institutional study may also involve analyzing other algorithms in tandem, such as traditional supervised machine learning or deep learning. An additional potential future direction would be to perform a similar such study using a molecular contrast agent targeting the VEGF receptor, namely BR55 [39]; unlike traditional DCE-US, this agent has been chemically modified to target areas of VEGF expression in vascular endothelial cells, which are upregulated in the setting of angiogenesis, and hence would demonstrate a different pattern of contrast kinetics than untargeted DCE-US. This molecular agent typically sticks to its receptor as opposed to washing out at an earlier time point. Targeted molecular ultrasound is an emerging area.

A natural future direction involves expanding this analysis to human subjects, as there may be some biological differences between human colon cancer and a human colon cancer xenograft, which may be related to differences in the tumor microenvironment. However, since tumor perfusion is not typically assessed this rapidly and this frequently after cancer treatment using contrast-enhanced ultrasound, it would require a prospective clinical study rather than a retrospective study. In fact, this could be performed as an adjunct to an existing clinical study to examine whether changes in perfusion can be detected sooner than changes in tumor size to monitor treatment effectiveness. Early determination of treatment effectiveness can spare a patient costly and ineffective chemotherapy. In agreement with the results of our study, in previous studies, unsupervised curvature learning, along with dimensionality reduction and diffusion maps, was successfully applied to cancer images [13,19,22]. Nonetheless, to the best of our knowledge, we show for the first time that curvature learning can be used in DCE-US images, and it serves as a promising, noninvasive tool to monitor early treatment changes in cancer.

5. Conclusions

Curvature learning was effective for analyzing treatment response in mouse data, and a similar technique could be used for treatment effectiveness monitoring in humans. Unlike other approaches, this could be conducted in a completely automated manner, without requiring manual delineation of a region of interest or requiring a large dataset for training a supervised machine learning algorithm. For humans, patient care would be improved if successful chemotherapy response could be assessed within days rather than waiting 2–3 months via the traditional RECIST, potentially avoiding toxic drug-related side effects or expensive, ineffective therapy.

6. Patents

Michael A. Jacobs and Alireza Akhbardeh have the patent “Multiparametric non-linear dimension reduction methods and systems related thereto”, under US Patent 9,256,966 [40].

Author Contributions

Conceptualization, H.S. and A.A.; methodology, A.A.; software, A.A.; validation, H.S. and A.A.; formal analysis, A.A.; investigation, A.A.; resources, A.A.; data curation, A.A.; writing—original draft preparation, H.S., I.D. and A.A.; writing—review and editing, H.S., I.D. and A.A.; visualization, H.S. and A.A.; supervision, H.S. and A.A.; project administration, H.S. and A.A.; funding acquisition, H.S. and I.D. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Radiological Society of North America, grant number #RF1727 (H.S.), the Stanford Cancer Imaging Training Program, NIH 2T32CA009695-26 (H.S.), and the Swiss Society of Radiology (I.D.).

Institutional Review Board Statement

Data Availability Statement

Data will be made available upon request, and availability is determined by institutional guidelines.

Acknowledgments

The authors thank the late Juergen Willmann, who guided the genesis of this project.

Conflicts of Interest

The authors declare that the research was conducted without any commercial or financial relationships that could be seen as potential conflicts of interest. Ambient Digital LLC, Regionalspital Emmental Burgdorf, Switzerland, The University of Texas Health Science Center at Houston, and the University of Pennsylvania were not involved in the study’s design, data collection, analysis, interpretation, manuscript preparation, or the decision to publish the findings.

References

Eisenhauer, E.A.; Therasse, P.; Bogaerts, J.; Schwartz, L.H.; Sargent, D.; Ford, R.; Dancey, J.; Arbuck, S.; Gwyther, S.; Mooney, M.; et al. New response evaluation criteria in solid tumours: Revised RECIST guideline (version 1.1). Eur. J. Cancer 2009, 45, 228–247. [Google Scholar] [CrossRef] [PubMed]
Zhou, J.; Zhang, H.; Wang, H.; Lutz, A.M.; El Kaffas, A.; Tian, L.; Hristov, D.; Willmann, J.K. Early prediction of tumor response to bevacizumab treatment in murine colon cancer models using three-dimensional dynamic contrast-enhanced ultrasound imaging. Angiogenesis 2017, 20, 547–555. [Google Scholar] [CrossRef]
Zhou, J.; Wang, H.; Zhang, H.; Lutz, A.M.; Tian, L.; Hristov, D.; Willmann, J.K. VEGFR2-Targeted Three-Dimensional Ultrasound Imaging Can Predict Responses to Antiangiogenic Therapy in Preclinical Models of Colon Cancer. Cancer Res. 2016, 76, 4081–4089. [Google Scholar] [CrossRef] [PubMed]
Durot, I.; Wilson, S.R.; Willmann, J.K. Contrast-enhanced ultrasound of malignant liver lesions. Abdom. Radiol. 2018, 43, 819–847. [Google Scholar] [CrossRef]
El Kaffas, A.; Sigrist, R.M.S.; Fisher, G.; Bachawal, S.; Liau, J.; Wang, H.; Karanany, A.; Durot, I.; Rosenberg, J.; Hristov, D.; et al. Quantitative Three-Dimensional Dynamic Contrast-Enhanced Ultrasound Imaging: First-In-Human Pilot Study in Patients with Liver Metastases. Theranostics 2017, 7, 3745–3758. [Google Scholar] [CrossRef]
Hudson, J.M.; Williams, R.; Tremblay-Darveau, C.; Sheeran, P.S.; Milot, L.; Bjarnason, G.A.; Burns, P.N. Dynamic contrast enhanced ultrasound for therapy monitoring. Eur. J. Radiol. 2015, 84, 1650–1657. [Google Scholar] [CrossRef] [PubMed]
Lopes-Coelho, F.; Martins, F.; Pereira, S.A.; Serpa, J. Anti-Angiogenic Therapy: Current Challenges and Future Perspectives. Int. J. Mol. Sci. 2021, 22, 3765. [Google Scholar] [CrossRef]
Hlatky, L.; Hahnfeldt, P.; Folkman, J. Clinical application of antiangiogenic therapy: Microvessel density, what it does and doesn’t tell us. J. Natl. Cancer Inst. 2002, 94, 883–893. [Google Scholar] [CrossRef]
Madsen, H.H.; Rasmussen, F. Contrast-enhanced ultrasound in oncology. Cancer Imaging 2011, 11, S167–S173. [Google Scholar] [CrossRef]
Chang, E.H. An Introduction to Contrast-Enhanced Ultrasound for Nephrologists. Nephron 2018, 138, 176–185. [Google Scholar] [CrossRef]
Malone, C.D.; Fetzer, D.T.; Monsky, W.L.; Itani, M.; Mellnick, V.M.; Velez, P.A.; Middleton, W.D.; Averkiou, M.A.; Ramaswamy, R.S. Contrast-enhanced US for the Interventional Radiologist: Current and Emerging Applications. Radiographics 2020, 40, 562–588. [Google Scholar] [CrossRef] [PubMed]
Mani, S.; Chen, Y.; Li, X.; Arlinghaus, L.; Chakravarthy, A.B.; Abramson, V.; Bhave, S.R.; Levy, M.A.; Xu, H.; Yankeelov, T.E. Machine learning for predicting the response of breast cancer to neoadjuvant chemotherapy. J. Am. Med. Inform. Assoc. 2013, 20, 688–695. [Google Scholar] [CrossRef] [PubMed]
Kourou, K.; Exarchos, T.P.; Exarchos, K.P.; Karamouzis, M.V.; Fotiadis, D.I. Machine learning applications in cancer prognosis and prediction. Comput. Struct. Biotechnol. J. 2015, 13, 8–17. [Google Scholar] [CrossRef]
Erickson, B.J.; Korfiatis, P.; Akkus, Z.; Kline, T.L. Machine Learning for Medical Imaging. Radiographics 2017, 37, 505–515. [Google Scholar] [CrossRef]
Bloniarz, A.; Liu, H.; Zhang, C.H.; Sekhon, J.S.; Yu, B. Lasso adjustments of treatment effect estimates in randomized experiments. Proc. Natl. Acad. Sci. USA 2016, 113, 7383–7390. [Google Scholar] [CrossRef]
Hsu, Y.L.; Huang, P.Y.; Chen, D.T. Sparse principal component analysis in cancer research. Transl. Cancer Res. 2014, 3, 182–190. [Google Scholar] [CrossRef]
Hoyt, K.; Sorace, A.; Saini, R. Quantitative mapping of tumor vascularity using volumetric contrast-enhanced ultrasound. Invest. Radiol. 2012, 47, 167–174. [Google Scholar] [CrossRef]
Varoquaux, G.; Cheplygina, V. Machine learning for medical imaging: Methodological failures and recommendations for the future. NPJ Digit. Med. 2022, 5, 48. [Google Scholar] [CrossRef]
Cruz, J.A.; Wishart, D.S. Applications of machine learning in cancer prediction and prognosis. Cancer Inform. 2007, 2, 59–77. [Google Scholar] [CrossRef] [PubMed]
Lampaskis, M.; Averkiou, M. Investigation of the relationship of nonlinear backscattered ultrasound intensity with microbubble concentration at low MI. Ultrasound Med. Biol. 2010, 36, 306–312. [Google Scholar] [CrossRef]
Akhbardeh, A.; Sagreiya, H.; El Kaffas, A.; Willmann, J.K.; Rubin, D.L. A multi-model framework to estimate perfusion parameters using contrast-enhanced ultrasound imaging. Med. Phys. 2019, 46, 590–600. [Google Scholar] [CrossRef]
Akhbardeh, A.; Jacobs, M.A. Comparative analysis of nonlinear dimensionality reduction techniques for breast MRI segmentation. Med. Phys. 2012, 39, 2275–2289. [Google Scholar] [CrossRef] [PubMed]
Jolliffe, I.T. Principal Component Analysis, 2nd ed.; Springer: New York, NY, USA, 2002; p. 487. [Google Scholar]
Fisher, R.A. The Use of Multiple Measurements in Taxonomic Problems. Ann. Eugen. 1936, 7, 179–188. [Google Scholar] [CrossRef]
Singer, A.; Wu, H.T. Orientability and Diffusion Maps. Appl. Comput. Harmon. Anal. 2011, 31, 44–58. [Google Scholar] [CrossRef] [PubMed]
Coifman, R.R.; Lafon, S.; Lee, A.B.; Maggioni, M.; Nadler, B.; Warner, F.; Zucker, S.W. Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps. Proc. Natl. Acad. Sci. USA 2005, 102, 7426–7431. [Google Scholar] [CrossRef]
Roweis, S.T.; Saul, L.K. Nonlinear dimensionality reduction by locally linear embedding. Science 2000, 290, 2323–2326. [Google Scholar] [CrossRef]
Tenenbaum, J.B.; de Silva, V.; Langford, J.C. A global geometric framework for nonlinear dimensionality reduction. Science 2000, 290, 2319–2323. [Google Scholar] [CrossRef]
Seung, H.S.; Lee, D.D. Cognition. The manifold ways of perception. Science 2000, 290, 2268–2269. [Google Scholar] [CrossRef]
Hu, H.T.; Wang, W.; Chen, L.D.; Ruan, S.M.; Chen, S.L.; Li, X.; Lu, M.D.; Xie, X.Y.; Kuang, M. Artificial intelligence assists identifying malignant versus benign liver lesions using contrast-enhanced ultrasound. J. Gastroenterol. Hepatol. 2021, 36, 2875–2883. [Google Scholar] [CrossRef] [PubMed]
Turco, S.; Tiyarattanachai, T.; Ebrahimkheil, K.; Eisenbrey, J.; Kamaya, A.; Mischi, M.; Lyshchik, A.; Kaffas, A.E. Interpretable Machine Learning for Characterization of Focal Liver Lesions by Contrast-Enhanced Ultrasound. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 2022, 69, 1670–1681. [Google Scholar] [CrossRef]
Qin, X.; Zhu, J.; Tu, Z.; Ma, Q.; Tang, J.; Zhang, C. Contrast-Enhanced Ultrasound with Deep Learning with Attention Mechanisms for Predicting Microvascular Invasion in Single Hepatocellular Carcinoma. Acad. Radiol. 2023, 30 (Suppl. S1), S73–S80. [Google Scholar] [CrossRef] [PubMed]
Liu, F.; Liu, D.; Wang, K.; Xie, X.; Su, L.; Kuang, M.; Huang, G.; Peng, B.; Wang, Y.; Lin, M.; et al. Deep Learning Radiomics Based on Contrast-Enhanced Ultrasound Might Optimize Curative Treatments for Very-Early or Early-Stage Hepatocellular Carcinoma Patients. Liver Cancer 2020, 9, 397–413. [Google Scholar] [CrossRef]
Tong, T.; Gu, J.; Xu, D.; Song, L.; Zhao, Q.; Cheng, F.; Yuan, Z.; Tian, S.; Yang, X.; Tian, J.; et al. Deep learning radiomics based on contrast-enhanced ultrasound images for assisted diagnosis of pancreatic ductal adenocarcinoma and chronic pancreatitis. BMC Med. 2022, 20, 74. [Google Scholar] [CrossRef] [PubMed]
Shao, Y.; Dang, Y.; Cheng, Y.; Gui, Y.; Chen, X.; Chen, T.; Zeng, Y.; Tan, L.; Zhang, J.; Xiao, M.; et al. Predicting the Efficacy of Neoadjuvant Chemotherapy for Pancreatic Cancer Using Deep Learning of Contrast-Enhanced Ultrasound Videos. Diagnostics 2023, 13, 2183. [Google Scholar] [CrossRef]
Kondo, S.; Satoh, M.; Nishida, M.; Sakano, R.; Takagi, K. Ceusia-Breast: Computer-aided diagnosis with contrast enhanced ultrasound image analysis for breast lesions. BMC Med. Imaging 2023, 23, 114. [Google Scholar] [CrossRef] [PubMed]
Feng, Y.; Yang, F.; Zhou, X.; Guo, Y.; Tang, F.; Ren, F.; Guo, J.; Ji, S. A Deep Learning Approach for Targeted Contrast-Enhanced Ultrasound Based Prostate Cancer Detection. IEEE/ACM Trans. Comput. Biol. Bioinform. 2019, 16, 1794–1801. [Google Scholar] [CrossRef]
Sun, Y.; Fang, J.; Shi, Y.; Li, H.; Wang, J.; Xu, J.; Zhang, B.; Liang, L. Machine learning based on radiomics features combing B-mode transrectal ultrasound and contrast-enhanced ultrasound to improve peripheral zone prostate cancer detection. Abdom. Radiol. 2024, 49, 141–150. [Google Scholar] [CrossRef]
Pochon, S.; Tardy, I.; Bussat, P.; Bettinger, T.; Brochot, J.; von Wronski, M.; Passantino, L.; Schneider, M. BR55: A lipopeptide-based VEGFR2-targeted ultrasound contrast agent for molecular imaging of angiogenesis. Invest. Radiol. 2010, 45, 89–95. [Google Scholar] [CrossRef]
Jacobs, M.A.; Akhbardeh, A. Multiparametric Non-Linear Dimension Reduction Methods and Systems Related Thereto. US 9,256,966 B2, 9 February 2016. [Google Scholar]

Figure 1. Three-dimensional dynamic contrast-enhanced ultrasound (DCE-US) imaging protocol for both treatment-sensitive (LS174T) and treatment-resistant (CT26) tumors. Ten days after tumor cell injection, 3D DCE-US scans (arrows) were performed prior to treatment (baseline scan at day 0) and at subsequent days 1, 3, 7, and 10 after treatment using either bevacizumab (treated mice) or saline (control mice). All mice were sacrificed on day 10, and tumors were excised for histologic analysis.

Figure 2. (A) Data processing block diagram to calculate curvature learning (CLS) score, a treatment effectiveness measure. (B) The results generated by the proposed curvature learning algorithm, including the fused heat map (or “embedded image”), the scattergram in multiple color-coded clusters, the three-class scattergram, and finally the percentage of pixels in each of the three classes (blue, yellow, and red). (C) The chart on the left demonstrates a characteristic example of variation in the yellow class on different days (pre-treatment day 0 and post-treatment days 1, 3, 7, and 10). Statistical features from different days were used to calculate intra-day curvature learning scores and the inter-day curvature learning score, resulting in a final curvature learning score representing an overall measure of treatment effectiveness.

Figure 3. Curvature learning algorithm results for a typical treatment-sensitive mouse treated with the chemotherapeutic agent bevacizumab (LSBolusAV) on days 0 (baseline), 3, and 10. This mouse has a cell line that is expected to respond to treatment. In the fused heat map, dark blue represents normal tissue, and the progressive color change from dark blue to cyan, green, yellow, orange, and red represents increasing suspicion for tumor. There is a progressive decrease in color classes more suspicious for tumor (yellow, orange, and red) with time. The scattergrams in the middle two columns demonstrate quantization into three color classes: blue, yellow, and red. In the percentage bar plots on the right, there is a decrease in the more suspicious yellow and red color classes with time.

Figure 4. Curvature learning algorithm results for a typical treatment-resistant, untreated (CTBolusCTRL) mouse on days 0 (baseline), 3, and 10. This mouse is not expected to respond to treatment. Pixels representing normal tissue are dark blue. There is a progressive increase in color classes more suspicious for tumor (yellow and red) with time both in the fused heat map on the left and the percentage bar plots on the right.

Figure 5. Box-and-whisker plot comparing the distribution of curvature learning scores (CLSs) between the four groups of ten mice: treatment-resistant, untreated (CTBolusCTRL); treatment-resistant, treated (CTBolusAV); treatment-sensitive, untreated (LSBolusCTRL); treatment-sensitive, treated (LSBolusAV). The horizontal red lines represent the median, the blue lines indicate the upper and lower quartiles, and the black lines at the ends denote the minimum and maximum values. This figure compares mice pre-treatment and three days after treatment. Only treatment-sensitive mice treated with bevacizumab were expected to respond to treatment, and this group shows a different distribution compared to the other groups.

Figure 6. Performance and capability of the proposed method for early diagnosis using only the first day after treatment: Difference in intra-day curvature learning scores pre-treatment and one day after treatment for the following groups: treatment-resistant, untreated (CTBolusCTRL); treatment-resistant, treated (CTBolusAV); treatment-sensitive, untreated (LSBolusCTRL); treatment-sensitive, treated (LSBolusAV). The horizontal red lines represent the median, the blue lines indicate the upper and lower quartiles, and the black lines at the ends denote the minimum and maximum values. There was a statistically significant difference in the final curvature learning score, encompassing differences in intra-day curvature learning scores pre-treatment and one day after treatment, between the treated, treatment-sensitive group and all other groups (p = 0.0051).

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Sagreiya, H.; Durot, I.; Akhbardeh, A. An Unsupervised Approach for Treatment Effectiveness Monitoring Using Curvature Learning. Computers 2024, 13, 227. https://doi.org/10.3390/computers13090227

AMA Style

Sagreiya H, Durot I, Akhbardeh A. An Unsupervised Approach for Treatment Effectiveness Monitoring Using Curvature Learning. Computers. 2024; 13(9):227. https://doi.org/10.3390/computers13090227

Chicago/Turabian Style

Sagreiya, Hersh, Isabelle Durot, and Alireza Akhbardeh. 2024. "An Unsupervised Approach for Treatment Effectiveness Monitoring Using Curvature Learning" Computers 13, no. 9: 227. https://doi.org/10.3390/computers13090227

APA Style

Sagreiya, H., Durot, I., & Akhbardeh, A. (2024). An Unsupervised Approach for Treatment Effectiveness Monitoring Using Curvature Learning. Computers, 13(9), 227. https://doi.org/10.3390/computers13090227

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu