Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Advertisement
Springer Nature Link
Account
Menu
Find a journal Publish with us Track your research
Search
Cart
  1. Home
  2. Machine Intelligence Research
  3. Article

Pre-training in Medical Data: A Survey

  • Review
  • Open access
  • Published: 21 February 2023
  • Volume 20, pages 147–179, (2023)
  • Cite this article
Download PDF

You have full access to this open access article

Machine Intelligence Research Aims and scope Submit manuscript
Pre-training in Medical Data: A Survey
Download PDF
  • Yixuan Qiu  ORCID: orcid.org/0000-0002-7593-18761,
  • Feng Lin1,
  • Weitong Chen  ORCID: orcid.org/0000-0003-1001-79252 &
  • …
  • Miao Xu1 

  • 2200 Accesses

  • 8 Citations

  • 2 Altmetric

  • Explore all metrics

Abstract

Medical data refers to health-related information associated with regular patient care or as part of a clinical trial program. There are many categories of such data, such as clinical imaging data, bio-signal data, electronic health records (EHR), and multi-modality medical data. With the development of deep neural networks in the last decade, the emerging pre-training paradigm has become dominant in that it has significantly improved machine learning methods′ performance in a data-limited scenario. In recent years, studies of pre-training in the medical domain have achieved significant progress. To summarize these technology advancements, this work provides a comprehensive survey of recent advances for pre-training on several major types of medical data. In this survey, we summarize a large number of related publications and the existing benchmarking in the medical domain. Especially, the survey briefly describes how some pre-training methods are applied to or developed for medical data. From a data-driven perspective, we examine the extensive use of pre-training in many medical scenarios. Moreover, based on the summary of recent pre-training studies, we identify several challenges in this field to provide insights for future studies.

Article PDF

Download to read the full article text

Similar content being viewed by others

Medical Data Analysis Using AutoML Frameworks

Article 06 June 2024

Applications of Deep Learning in Healthcare: A Systematic Analysis

Chapter © 2023

Pretrained transformer framework on pediatric claims data for population specific tasks

Article Open access 07 March 2022

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.
  • Artificial Intelligence
Use our pre-submission checklist

Avoid common mistakes on your manuscript.

References

  1. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, L. D. Jackel. Backpropagation applied to handwritten zip code recognition. Neural Computation, vol. 1, no. 4, pp. 541–551, 1989. DOI: https://doi.org/10.1162/neco.1989.1.4.541.

    Article  Google Scholar 

  2. K. Simonyan, A. Zisserman. Very deep convolutional networks for large-scale image recognition. [Online], Available: https://arxiv.org/abs/1409.1556, 2014.

  3. I. Sutskever, O. Vinyals, Q. V. Le. Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems, ACM, Montreal, Canada, pp.3104–3112, 2014.

    Google Scholar 

  4. J. Chung, C. Gulcehre, K. Cho, Y. Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. [Online], Available: https://arxiv.org/abs/1412.3555, 2014.

  5. S. U. Amin, M. Alsulaiman, G. Muhammad, M. A. Bencherif, M. S. Hossain. Multilevel weighted feature fusion using convolutional neural networks for EEG motor imagery classification. IEEE Access, vol. 7, pp. 18940–18950, 2019. DOI: https://doi.org/10.1109/ACCESS.2019.2895688.

    Article  Google Scholar 

  6. M. Jaderberg, K. Simonyan, A. Zisserman, K. Kavukcuoglu. Spatial transformer networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems, ACM, Montreal, Canada, pp. 2017–2025, 2015.

    Google Scholar 

  7. A. Krizhevsky, I. Sutskever, G. E. Hinton. ImageNet classification with deep convolutional neural networks. Communications of the ACM, vol. 60, no. 6, pp. 84–90, 2017. DOI: https://doi.org/10.1145/306586.

    Article  Google Scholar 

  8. J. Hirschberg, C. D. Manning. Advances in natural language processing. Science, vol. 349, no. 6245, pp. 261–266, 2015. DOI: https://doi.org/10.1126/science.aaa8685.

    Article  MathSciNet  Google Scholar 

  9. G. T. Wang, M. A. Zuluaga, W. Q. Li, R. Pratt, P. A. Patel, M. Aertsen, T. Doel, A. L. David, J. Deprest, S. Ourselin, T. Vercauteren. DeepIGeoS: A deep interactive geodesic framework for medical image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 7, pp. 1559–1572, 2019. DOI: https://doi.org/10.1109/TPAMI.2018.2840695.

    Article  Google Scholar 

  10. S. Minaee, Y. Y. Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz, D. Terzopoulos. Image segmentation using deep learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, pp. 3523–3542, 2022. DOI: https://doi.org/10.1109/TPAMI.2021.3059968.

    Google Scholar 

  11. H. Greenspan, B. Van Ginneken, R. M. Summers. Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique. IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1153–1159, 2016. DOI: https://doi.org/10.1109/TMI.2016.2553401.

    Article  Google Scholar 

  12. Y. D. Wang, W. T. Chen, D. C. Pi, L. Yue. Adversarially regularized medication recommendation model with multi-hop memory network. Knowledge and Information Systems, vol. 63, no. 1, pp. 125–142, 2021. DOI: https://doi.org/10.1007/s10115-020-01513-9.

    Article  Google Scholar 

  13. Y. D. Wang, W. T. Chen, D. C. Pi, L. Yue, S. Wang, M. Xu. Self-supervised adversarial distribution regularization for medication recommendation. In Proceedings of the 30th International Joint Conference on Artificial Intelligence, Montreal, Canada, pp. 3134–3140, 2021. DOI: https://doi.org/10.24963/ijcai.2021/431.

  14. Y. X. Qiu, W. T. Chen, L. Yue, M. Xu, B. F. Zhu. STCT: Spatial-temporal conv-transformer network for cardiac arrhythmias recognition. In Proceedings of the 17th International Conference on Advanced Data Mining and Applications, Springer, Sydney, Australia, pp. 86–100, 2022. DOI: https://doi.org/10.1007/978-3-030-95405-5_7.

    Google Scholar 

  15. V. J. R. Ripoll, A. Wojdel, E. Romero, P. Ramos, J. Brugada. ECG assessment based on neural networks with pretraining. Applied Soft Computing, vol. 49, pp. 399–406, 2016. DOI: https://doi.org/10.1016/j.asoc.2016.08.013.

    Article  Google Scholar 

  16. K. Weimann, T. O. F. Conrad. Transfer learning for ECG classification. Scientific Reports, vol. 11, no. 1, Article number 5251, 2021. DOI: https://doi.org/10.1038/s41598-021-84374-8.

    Google Scholar 

  17. X. S. Wang, Z. Y. Xu, L. Tam, D. Yang, D. G. Xu. Self-supervised image-text pre-training with mixed data in chest X-rays. [Online], Available: https://arxiv.org/abs/2103.16022, 2021.

  18. X. S. Wang, Y. F. Peng, L. Lu, Z. Y. Lu, M. Bagheri, R. M. Summers. ChestX-ray8: Hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, pp.2097-2106, 2017. DOI: https://doi.org/10.1109/CVPR.2017.369.

  19. J. H. Moon, H. Lee, W. Shin, Y. H. Kim, E. Choi. Multi-modal understanding and generation for medical images and text via vision-language pre-training. IEEE Journal of Biomedical and Health Informatics, to be published. DOI: https://doi.org/10.1109/JBHI.2022.3207502.

  20. B. Yan, M. T. Pei. Clinical-BERT: Vision-language pre-training for radiograph diagnosis and reports generation. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 3, pp. 2982–2990, 2022. DOI: https://doi.org/10.1609/aaai.v36i3.20204.

  21. L. Hou, D. Samaras, T. M. Kurc, Y. Gao, J. E. Davis, J. H. Saltz. Patch-based convolutional neural network for whole slide tissue image classification. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 2424–2433, 2016. DOI: https://doi.org/10.1109/CVPR.2016.266.

  22. H. C. Shin, H. R. Roth, M. C. Gao, L. Lu, Z. Y. Xu, I. Nogues, J. H. Yao, D. Mollura, R. M. Summers. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1285–1298, 2016. DOI: https://doi.org/10.1109/TMI.2016.2528162.

    Article  Google Scholar 

  23. T. Würfl, F. C. Ghesu, V. Christlein, A. Maier. Deep learning computed tomography. In Proceedings of the 19th International Conference on Medical Image Computing and Computer-assisted Intervention, Springer, Athens, Greece, pp. 432–440, 2016. DOI: https://doi.org/10.1007/978-3-319-46726-9_50.

    Google Scholar 

  24. B. Ramsundar, P. Eastman, P. Walters, V. Pande. Deep Learning for the Life Sciences: Applying Deep Learning to Genomics, Microscopy, Drug Discovery, and More, Sebastopol, USA: O’Reilly Media, 2019.

    Google Scholar 

  25. X. Han, Z. Y. Zhang, N. Ding, Y. X. Gu, X. Liu, Y. Q. Huo, J. Z. Qiu, Y. Yao, A. Zhang, L. Zhang, W. T. Han, M. L. Huang, Q. Jin, Y. Y. Lan, Y. Liu, Z. Y. Liu, Z. W. Lu, X. P. Qiu, R. H. Song, J. Tang, J. R. Wen, J. H. Yuan, W. X. Zhao, J. Zhu. Pre-trained models: Past, present and future. AI Open, vol. 2, pp. 225–250, 2021. DOI: https://doi.org/10.1016/j.aiopen.2021.08.002.

    Article  Google Scholar 

  26. L. Torrey, J. Shavlik. Transfer learning. Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques, E. S. Olivas, J. D. M. Guerrero, M. Martinez-Sober, J. R. Magdalena-Benedito, A. J. S. López, Eds., Hershey, USA: IGI Global, pp. 242–264, 2010. DOI: https://doi.org/10.4018/978-1-60566-766-9.ch011.

    Chapter  Google Scholar 

  27. S. J. Pan, Q. Yang. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 10, pp. 1345–1359, 2010. DOI: https://doi.org/10.1109/TKDE.2009.191.

    Article  Google Scholar 

  28. V. Jain, E. Learned-Miller. Online domain adaptation of a pre-trained cascade of classifiers. In Proceedings of IEEE Computer Vision and Pattern Recognition, Colorado Springs, USA, pp. 577–584, 2011. DOI: https://doi.org/10.1109/CV-PR.2011.5995317.

  29. A. Newell, J. Deng. How useful is self-supervised pretraining for visual tasks? In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Seattle, USA, pp. 7345–7354, 2020. DOI: https://doi.org/10.1109/CVPR42600.2020.00737.

    Google Scholar 

  30. Y. Z. Yang, Z. Xu. Rethinking the value of labels for improving class-imbalanced learning. In Proceedings of the 34th International Conference on Neural Information Processing Systems, ACM, Vancouver, Canada, pp. 19290–19301, 2020.

    Google Scholar 

  31. H. Liu, J. Z. HaoChen, A. Gaidon, T. Y. Ma. Self-supervised learning is more robust to dataset imbalance. In Proceedings of the 10th International Conference on Learning Representations, 2022.

  32. T. Schlegl, J. Ofner, G. Langs. Unsupervised pre-training across image domains improves lung tissue classification. In Proceedings of the International Workshop on Medical Computer Vision: Algorithms for Big Data, Springer, Cambridge, USA, pp. 82–93, 2014. DOI: https://doi.org/10.1007/978-3-319-13972-2_8.

    Google Scholar 

  33. Y. W. Meng, W. Speier, M. K. Ong, C. W. Arnold. Bidirectional representation learning from transformers using multimodal electronic health record data to predict depression. IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 8, pp. 3121–3129, 2021. DOI: 1109/JBHI.2021.3063721.

    Article  Google Scholar 

  34. S. Azizi, B. Mustafa, F. Ryan, Z. Beaver, J. Freyberg, J. Deaton, A. Loh, A. Karthikesalingam, S. Kornblith, T. Chen, V. Natarajan, M. Norouzi. Big self-supervised models advance medical image classification. In Proceedings of IEEE/CVF International Conference on Computer Vision, IEEE, Montreal, Canada, pp. 3478–3488, 2021. DOI: https://doi.org/10.1109/ICCV48922.2021.00346.

    Google Scholar 

  35. T. Thinsungnoen, K. Kerdprasop, N. Kerdprasop. Deep autoencoder networks optimized with genetic algorithms for efficient ECG clustering. International Journal of Machine Learning and Computing, vol. 8, no. 2, pp. 112–116, 2018. DOI: https://doi.org/10.18178/ijmlc.2018.8.2.672.

    Article  Google Scholar 

  36. V. Cheplygina, M. de Bruijne, J. P. W. Pluim. Not-so-supervised: A survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Medical Image Analysis, vol. 54, pp. 280–296, 2019. DOI: https://doi.org/10.1016/j.media.2019.03.009.

    Article  Google Scholar 

  37. T. D. Pham. A comprehensive study on classffication of COVID-19 on computed tomography with pretrained convolutional neural networks. Scientific Reports, vol. 10, no. 1, Article number 16942, 2020. DOI: https://doi.org/10.1038/s41598-020-74164-z.

    Google Scholar 

  38. P. Y. Chen. Representation learning for electronic health records: A survey. Journal of Physics: Conference Series, vol. 1487, Article number 012015, 2020. DOI: https://doi.org/10.1088/1742-6596/1487/1/012015.

  39. S. Shurrab, R. Duwairi. Self-supervised learning methods and applications in medical imaging analysis: A survey. PeerJ Computer Science, vol. 8, Article number e1045, 2022. DOI: https://doi.org/10.7717/peerj-cs.1045.

  40. A. Ebbehoj, M. Ø. Thunbo, O. E. Andersen, M. V. Glindtvad, A. Hulman. Transfer learning for non-image data in clinical research: A scoping review. PLoS Digital Health, vol. 1, no. 2, Article number e0000014, 2022. DOI: https://doi.org/10.1371/journal.pdig.0000014.

    Google Scholar 

  41. T. J. Pollard, A. E. W. Johnson, J. D. Raffa, L. A. Celi, R. G. Mark, O. Badawi. The eICU collaborative research database, a freely available multi-center database for critical care research. Scientific Data, vol. 5, no. 1, Article number 180178, 2018. DOI: https://doi.org/10.1038/sdata.2018.178.

    Google Scholar 

  42. A. E. W. Johnson, T. J. Pollard, L. Shen, L. W. H. Lehman, M. L. Feng, M. Ghassemi, B. Moody, P. Szolovits, L. Anthony Celi, R. G. Mark. MIMIC-III, a freely accessible critical care database. Scientific Data, vol. 3, no. 1, Article number 160035, 2016. DOI: https://doi.org/10.1038/sdata.2016.35.

    Google Scholar 

  43. A. L. Goldberger, L. A. Amaral, L. Glass, J. M. Hausdorff, P. C. Ivanov, R. G. Mark, J. E. Mietus, G. B. Moody, C. K. Peng, H. E. Stanley. PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation, vol. 101, no. 23, pp. e215–e220, 2000. DOI: https://doi.org/10.1161/01.cir.101.23.e215.

    Article  Google Scholar 

  44. A. E. Kavur, N. S. Gezer, M. Barιş, S. Aslan, P. H. Conze, V. Groza, D. D. Pham, S. Chatterjee, P. Ernst, S. Özkan, B. Baydar, D. Lachinov, S. Han, J. Pauli, F. Isensee, M. Perkonigg, R. Sathish, R. Rajan, D. Sheet, G. Dovletov, O. Speck, A. Nürnberger, K. H. Maier-Hein, G. Bozdağı Akar, G. Ünal, O. Dicle, M. A. Selver. CHAOS challenge-combined (CT-MR) healthy abdominal organ segmentation. Medical Image Analysis, vol. 69, Article number 101950, 2021. DOI: https://doi.org/10.1016/j.media.2020.101950.

  45. A. Sinha, J. Dolz. Multi-scale self-guided attention for medical image segmentation. IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 1, pp. 121–130, 2021. DOI: https://doi.org/10.1109/JBHI.2020.2986926.

    Article  Google Scholar 

  46. H. R. Roth, L. Lu, A. Farag, H. C. Shin, J. M. Liu, E. B. Turkbey, R. M. Summers. DeepOrgan: Multi-level deep convolutional networks for automated pancreas segmentation. In Proceedings of the 18th International Conference on Medical Image Computing and Computer-assisted Intervention, Springer, Munich, Germany, pp. 556–564, 2015. DOI: https://doi.org/10.1007/978-3-319-24553-9_68.

    Google Scholar 

  47. P. Bilic, P. F. Christ, E. Vorontsov, G. Chlebus, H. Chen, Q. Dou, C. W. Fu, X. Han, P. A. Heng, J. Hesser, S. Kadoury, T. K. Konopczynski, M. Le, C. M. Li, X. M. Li, J. Lipková, J. S. Lowengrub, H. Meine, J. H. Moltz, C. Pal, M. Piraud, X. J. Qi, J. Qi, M. Rempfler, K. Roth, A. Schenk, A. Sekuboyina, P. Zhou, C. Hülsemeyer, M. Beetz, F. Ettlinger, F. Grün, G. Kaissis, F. Lohöfer, R. Braren, J. Holch, F. Hofmann, W. H. Sommer, V. Heinemann, C. Jacobs, G. E. H. Mamani, B. van Ginneken, G. Chartrand, A. Tang, M. Drozdzal, A. Ben-Cohen, E. Klang, M. M. Amitai, E. Konen, H. Greenspan, J. Moreau, A. Hostettler, L. Soler, R. Vivanti, A. Szeskin, N. Lev-Cohain, J. Sosna, L. Joskowicz, B. H. Menze. The liver tumor segmentation benchmark (LiTS). [Online], Available: https://arxiv.org/abs/1901.04056, 2019.

  48. X. M. Li, H. Chen, X. J. Qi, Q. Dou, C. W. Fu, P. A. Heng. H-DenseUNet: Hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Transactions on Medical Imaging, vol. 37, no. 12, pp. 2663–2674, 2018. DOI: https://doi.org/10.1109/TMI.2018.2845918.

    Article  Google Scholar 

  49. D. S. Kermany, M. Goldbaum, W. J. Cai, C. C. S. Valentim, H. Y. Liang, S. L. Baxter, A. McKeown, G. Yang, X. K. Wu, F. B. Yan, J. Dong, M. K. Prasadha, J. Pei, M. Y. L. Ting, J. Zhu, C. Li, S. Hewett, J. Dong, I. Ziyar, A. Shi, R. Z. Zhang, L. H. Zheng, R. Hou, W. Shi, X. Fu, Y. O. Duan, V. A. N. Huu, C. Wen, E. D. Zhang, C. L. Zhang, O. L. Li, X. B. Wang, M. A. Singer, X. D. Sun, J. Xu, A. Tafreshi, M. A. Lewis, H. M. Xia, K. Zhang. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell, vol. 172, no. 5, pp. 1122–1131, 2018. DOI: https://doi.org/10.1016/j.cell.2018.02.010.

    Article  Google Scholar 

  50. W. Al-Dhabyani, M. Gomaa, H. Khaled, A. Fahmy. Dataset of breast ultrasound images. Data in Brief, vol. 28, Article number 104863, 2020. DOI: https://doi.org/10.1016/j.dib.2019.104863.

  51. W. K. Moon, Y. W. Lee, H. H. Ke, S. H. Lee, C. S. Huang, R. F. Chang. Computer-aided diagnosis of breast ultrasound images using ensemble learning from convolutional neural networks. Computer Methods and Programs in Biomedicine, vol. 190, Article number 105361, 2020. DOI: https://doi.org/10.1016/j.cmpb.2020.105361.

  52. P. Rajpurkar, J. Irvin, K. Zhu, B. Yang, H. Mehta, T. Duan, D. Ding, A. Bagul, C. Langlotz, K. Shpanskaya, M. P. Lungren, A. Y. Ng. CheXNet: Radiologist-level pneumonia detection on chest X-Rays with deep learning. [Online], Available: https://arxiv.org/abs/1711.05225, 2017.

  53. Z. Wang, Y. X. Yin, J. P. Shi, W. Fang, H. S. Li, X. G. Wang Zoom-in-Net: Deep mining lesions for diabetic retinopathy detection In Proceedings of the 20th International Conference on Medical Image Computing and Computer Assisted Intervention, Springer, Quebec City, Canada, pp. 267–275, 2017. DOI: https://doi.org/10.1007/978-3-319-66179-7_31.

    Google Scholar 

  54. N. C. F. Codella, D. Gutman, M. E. Celebi, B. Helba, M. A. Marchetti, S. W. Dusza, A. Kalloo, K. Liopyris, N. Mishra, H. Kittler, A. Halpern. Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). In Proceedings of 15th IEEE International Symposium on Biomedical Imaging, Washington, USA, pp. 168–172, 2018. DOI: https://doi.org/10.1109/ISBI.2018.8363547.

  55. N. Gessert, M. Nielsen, M. Shaikh, R. Werner, A. Schlaefer. Skin lesion classification using ensembles of multi-resolution efficientNets with meta data. MethodsX, vol. 7, Article number 100864, 2020. DOI: https://doi.org/10.1016/j.mex.2020.100864.

  56. C. Kandoth, M. D. McLellan, F. Vandin, K. Ye, B. F. Niu, C. Lu, M. C. Xie, Q. Y. Zhang, J. F. McMichael, M. A. Wyczalkowski, M. D. M. Leiserson, C. A. Miller, J. S. Welch, M. J. Walter, M. C. Wendl, T. J. Ley, R. K. Wilson, B. J. Raphael, L. Ding. Mutational landscape and significance across 12 major cancer types. Nature, vol. 502, no. 7471, pp. 333–339, 2013. DOI: https://doi.org/10.1038/nature12634.

    Article  Google Scholar 

  57. J. W. Yao, X. L. Zhu, F. Y. Zhu, J. Z. Huang. Deep correlational learning for survival prediction from multi-modality data. In Proceedings of the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Quebec City, Canada, pp. 406–414, 2017. DOI: https://doi.org/10.1007/978-3-319-66185-8_46.

    Google Scholar 

  58. P. Mobadersany, S. Yousefi, M. Amgad, D. A. Gutman, J. S. Barnholtz-Sloan, J. E. Velázquez Vega, D. J. Brat, L. A. D. Cooper. Predicting cancer outcomes from histology and genomics using convolutional networks. Proceedings of the National Academy of Sciences of the United States of America, vol. 115, no. 13, pp. E2970–E2979, 2018. DOI: https://doi.org/10.1073/pnas.1717139115.

    Google Scholar 

  59. National Lung Screening Trial Research Team. The national lung screening trial: Overview and study design. Radiology, vol. 258, no. 1, pp. 243–253, 2011. DOI: https://doi.org/10.1148/radiol.10091808.

    Article  Google Scholar 

  60. J. W. Yao, X. L. Zhu, J. Jonnagaddala, N. Hawkins, J. Z. Huang. Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Medical Image Analysis, vol. 65, Article number 101789, 2020. DOI: https://doi.org/10.1016/j.media.2020.101789.

  61. B. S. Veeling, J. Linmans, J. Winkens, T. Cohen, M. Welling. Rotation equivariant CNNs for digital pathology. In Proceedings of the 21st International Conference on Medical Image Computing and Computer-assisted Intervention, Springer, Granada, Spain, pp. 210–218, 2018. DOI: https://doi.org/10.1007/978-3-030-00934-2_24.

    Google Scholar 

  62. G. B. Moody, R. G. Mark. The impact of the MIT-BIH arrhythmia database. IEEE Engineering in Medicine and Biology Magazine, vol. 20, no. 3, pp. 45–50, 2001. DOI: https://doi.org/10.1109/51.932724.

    Article  Google Scholar 

  63. P. Wagner, N. Strodthoff, R. D. Bousseljot, D. Kreiseler, F. I. Lunze, W. Samek, T. Schaeffter. PTB-XL, a large publicly available electrocardiography dataset. Scientific Data, vol. 7, no. 1, Article number 154, 2020. DOI: https://doi.org/10.1038/s41597-020-0495-6.

    Google Scholar 

  64. G. B. Moody, W. K. Muldrow, R. G. Mark. A noise stress test for arrhythmia detectors. Computers in Cardiology, vol. 11, no. 3, pp. 381–384, 1984.

    Google Scholar 

  65. A. Taddei, G. Distante, M. Emdin, P. Pisani, G. B. Moody, C. Zeelenberg, C. Marchesi. The European ST-T database: Standard for evaluating systems for the analysis of ST-T changes in ambulatory electrocardiography. European Heart Journal, vol. 13, no. 9, pp. 1164–1172, 1992. DOI: https://doi.org/10.1093/oxfordjournals.eurheartj.a060332.

    Article  Google Scholar 

  66. G. D. Clifford, C. Y. Liu, B. Moody, L. W. H. Lehman, I. Silva, Q. Li, A. E. Johnson, R. G. Mark. AF classification from a short single lead ECG recording: The physioNet/computing in cardiology challenge 2017. In Proceedings of Computing in Cardiology, IEEE, Rennes, France, 2017. DOI: https://doi.org/10.22489/CinC.2017.065-469.

    Google Scholar 

  67. F. Andreotti, O. Carr, M. A. F. Pimentel, A. Mahdi, M. De Vos. Comparing feature-based classifiers and convolutional neural networks to detect arrhythmia from short segments of ECG. In Proceedings of Computing in Cardiology, IEEE, Rennes, France, pp. 1–4, 2017. DOI: https://doi.org/10.22489/CinC.2017.360-239.

    Google Scholar 

  68. R. Bousseljot, D. Kreiseler, A. Schnabel. Nutzung der EKG-signaldatenbank cardiodat der PTB über das internet. Biomedizinische Technik, vol. 40, Article number 317, 1995. DOI: https://doi.org/10.1515/bmte.1995.40.s1.317.

  69. L. Sharma, R. Tripathy, S. Dandapat. Multiscale energy and eigenspace approach to detection and localization of myocardial infarction. IEEE Transactions on Biomedical Engineering, vol. 62, no. 7, pp. 1827–1837, 2015.

    Article  Google Scholar 

  70. G. B. Moody, R. G. Mark. Development and evaluation of a 2-lead ECG analysis program. Computers in Cardiology, vol. 1982, pp. 39–44, 1982.

    Google Scholar 

  71. F. F. Liu, C. Y. Liu, L. N. Zhao, X. Y. Zhang, X. L. Wu, X. Y. Xu, Y. L. Liu, C. Y. Ma, S. S. Wei, Z. Q. He, J. Q. Li, E. N. Yin Kwee. An open access database for evaluating the algorithms of electrocardiogram rhythm and morphology abnormality detection. Journal of Medical Imaging and Health Informatics, vol. 8, no. 7, pp. 1368–1373, 2018. DOI: https://doi.org/10.1166/jmihi.2018.2442.

    Article  Google Scholar 

  72. T. M. Chen, C. H. Huang, E. S. C. Shih, Y. F. Hu, M. J. Hwang. Detection and classification of cardiac arrhythmias by a challenge-best deep learning neural network model. iScience, vol. 23, no. 3, Article number 100886, 2020. DOI: https://doi.org/10.1016/j.isci.2020.100886.

    Google Scholar 

  73. J. A. Miranda-Correa, M. K. Abadi, N. Sebe, I. Patras. AMIGOS: A dataset for affect, personality and mood research on individuals and groups. IEEE Transactions on Affective Computing, vol. 12, no. 2, pp. 479–493, 2021. DOI: https://doi.org/10.1109/TAFFC.2018.2884461.

    Article  Google Scholar 

  74. L. Santamaria-Granados, M. Munoz-Organero, G. Ramirez-González, E. Abdulhay, N. Arunkumar. Using deep convolutional neural network for emotion detection on a physiological signals dataset (AMIGOS). IEEE Access, vol. 7, pp. 57–67, 2018. DOI: https://doi.org/10.1109/ACCESS.2018.2883213.

    Article  Google Scholar 

  75. R. Subramanian, J. Wache, M. K. Abadi, R. L. Vieriu, S. Winkler, N. Sebe. ASCERTAIN: Emotion and personality recognition using commercial sensors. IEEE Transactions on Affective Computing, vol. 9, no. 2, pp. 147–160, 2018. DOI: https://doi.org/10.1109/TAFFC.2016.2625250.

    Article  Google Scholar 

  76. L. Zhang, S. Walter, X. Y. Ma, P. Werner, A. Al-Hamadi, H. C. Traue, S. Gruss. “BioVid Emo DB”: A multimodal database for emotion analyses validated by subjective ratings. In Proceedings of IEEE Symposium Series on Computational Intelligence, Athens, Greece, 2016. DOI: https://doi.org/10.1109/SSCI.2016.7849931.

  77. Z. Cheng, L. Shu, J. Y. Xie, C. L. P. Chen. A novel ECG-based real-time detection method of negative emotions in wearable applications. In Proceedings of International Conference on Security, Pattern Analysis, and Cybernetics, IEEE, Shenzhen, China, pp. 296–301, 2017. DOI: https://doi.org/10.1109/SPAC.2017.8304293.

    Google Scholar 

  78. S. Koelstra, C. Muhl, M. Soleymani, J. S. Lee, A. Yazdani, T. Ebrahimi, T. Pun, A. Nijholt, I. Patras. DEAP: A database for emotion analysis; using physiological signals. IEEE Transactions on Affective Computing, vol. 3, no. 1, pp. 18–31, 2012. DOI: https://doi.org/10.1109//T-AFFC.2011.15.

    Article  Google Scholar 

  79. Z. Yin, M. Y. Zhao, Y. X. Wang, J. D. Yang, J. H. Zhang. Recognition of emotions using multimodal physiological signals and an ensemble deep learning model. Computer Methods and Programs in Biomedicine, vol. 140, pp. 93–110, 2017. DOI: https://doi.org/10.1016/j.cmpb.2016.12.005.

    Article  Google Scholar 

  80. S. Katsigiannis, N. Ramzan. DREAMER: A database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices. IEEE Journal of Biomedical and Health Informatics, vol. 22, no. 1, pp. 98–107, 2018. DOI: https://doi.org/10.1109/JBHI.2017.2688239.

    Article  Google Scholar 

  81. T. F. Song, W. M. Zheng, P. Song, Z. Cui. EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Transactions on Affective Computing, vol. 11, no. 3, pp. 532–541, 2020. DOI: https://doi.org/10.1109/TAFFC.2018.2817622.

    Article  Google Scholar 

  82. M. Soleymani, J. Lichtenauer, T. Pun, M. Pantic. A multimodal database for affect recognition and implicit tagging. IEEE Transactions on Affective Computing, vol. 3, no. 1, pp. 42–55, 2012. DOI: https://doi.org/10.1109/T-AFFC.2011.25.

    Article  Google Scholar 

  83. X. B. Li, J. Chen, G. Y. Zhao, M. Pietikäinen. Remote heart rate measurement from face videos under realistic situations. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Columbus, USA, pp. 4264–4271, 2014. DOI: https://doi.org/10.1109/CVPR.2014.543.

  84. T. F. Song, W. M. Zheng, C. Lu, Y. Zong, X. L. Zhang, Z. Cui. MPED: A multi-modal physiological emotion database for discrete emotion recognition. IEEE Access, vol. 7, pp. 12177–12191, 2019. DOI: https://doi.org/10.1109/ACCESS.2019.2891579.

    Article  Google Scholar 

  85. W. L. Zheng, B. L. Lu. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Transactions on Autonomous Mental Development, vol. 7, no. 3, pp. 162–175, 2015. DOI: https://doi.org/10.1109/TAMD.2015.2431497.

    Article  Google Scholar 

  86. I. Obeid, J. Picone. The temple university hospital EEG data corpus. Frontiers in Neuroscience, vol. 10, Article number 196, 2016. DOI: https://doi.org/10.3389/fnins.2016.00196.

  87. G. Schalk, D. J. McFarland, T. Hinterberger, N. Birbaumer, J. R. Wolpaw. BCI2000: A general-purpose brain-computer interface (BCI) system. IEEE Transactions on Biomedical Engineering, vol. 51, no. 6, pp. 1034–1043, 2004. DOI: https://doi.org/10.1109/TBME.2004.827072.

    Article  Google Scholar 

  88. A. Supratak, H. Dong, C. Wu, Y. K. Guo. DeepSleepNet: A model for automatic sleep stage scoring based on raw single-channel EEG. IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 11, pp. 1998–2008, 2017. DOI: https://doi.org/10.1109/TNSRE.2017.2721116.

    Article  Google Scholar 

  89. G. Q. Zhang, L. C. Cui, R. Mueller, S. Q. Tao, M. Kim, M. Rueschman, S. Mariani, D. Mobley, S. Redline. The national sleep research resource: Towards a sleep data commons. Journal of the American Medical Informatics Association, vol. 25, no. 10, pp. 1351–1358, 2018. DOI: https://doi.org/10.1093/jamia/ocy064.

    Article  Google Scholar 

  90. S. F. Quan, B. V. Howard, C. Iber, J. P. Kiley, F. J. Nieto, G. T. O’Connor, D. M. Rapoport, S. Redline, J. Robbins, J. M. Samet, P. W. Wahl. The sleep heart health study: Design, rationale, and methods. Sleep, vol. 20, no. 12, pp. 1077–1085, 1997.

    Google Scholar 

  91. A. Sors, S. Bonnet, S. Mirek, L. Vercueil, J. F. Payen. A convolutional neural network for sleep stage scoring from raw single-channel EEG. Biomedical Signal Processing and Control, vol. 42, pp. 107–114, 2018. DOI: https://doi.org/10.1016/j.bspc.2017.12.001.

    Article  Google Scholar 

  92. Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle. Greedy layer-wise training of deep networks. In Proceedings of the 19th International Conference on Neural Information Processing Systems, ACM, Vancouver, Canada, pp. 153–160, 2006.

    Google Scholar 

  93. M. Ranzato, Y. L. Boureau, Y. LeCun. Sparse feature learning for deep belief networks. In Proceedings of the 20th International Conference on Neural Information Processing Systems, ACM, Vancouver, Canada, pp. 1185–1192, 2007.

    Google Scholar 

  94. K. M. He, R. Girshick, P. Dollár. Rethinking ImageNet pre-training. In Proceedings of IEEE/CVF International Conference on Computer Vision, IEEE, Seoul, Republic of Korea, pp. 4918–4927, 2019. DOI: https://doi.org/10.1109/ICCV.2019.00502.

    Google Scholar 

  95. M. Raghu, C. Y. Zhang, J. Kleinberg, S. Bengio. Transfusion: Understanding transfer learning for medical imaging. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, ACM, Vancouver, Canada, pp. 3347–3357, 2019.

    Google Scholar 

  96. S. Thrun, L. Pratt. Learning to learn: Introduction and overview. Learning to Learn, S. Thrun, L. Pratt, Eds., Boston, USA: Springer, pp. 3–17, 1998. DOI: https://doi.org/10.1007/978-1-4615-5529-2_1.

    Chapter  Google Scholar 

  97. H. Scudder. Probability of error of some adaptive pattern-recognition machines. IEEE Transactions on Information Theory, vol. 11, no. 3, pp. 363–371, 1965. DOI: https://doi.org/10.1109/TIT.1965.1053799.

    Article  MathSciNet  Google Scholar 

  98. K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Vegas, USA, pp. 770–778, 2016. DOI: https://doi.org/10.1109/CV-PR.2016.90.

  99. C. Szegedy, W. Liu, Y. Q. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich. Going deeper with convolutions. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, pp. 1–9, 2015. DOI: https://doi.org/10.1109/CVPR.2015.7298594.

  100. G. Huang, Z. Liu, L. Van Der Maaten, K. Q. Weinberger. Densely connected convolutional networks. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, pp. 2261–2269, 2017. DOI: https://doi.org/10.1109/CVPR.2017.243.

  101. J. Sarzynska-Wawer, A. Wawer, A. Pawlak, J. Szymanowska, I. Stefaniak, M. Jarkiewicz, L. Okruszek. Detecting formal thought disorder by deep contextualized word representations. Psychiatry Research, vol. 304, Article number 114135, 2021. DOI: https://doi.org/10.1016/j.psychres.2021.114135.

  102. Z. Y. Han, B. Z. Wei, Y. J. Zheng, Y. L. Yin, K. J. Li, S. Li. Breast cancer multi-classification from histopathological images with structured deep learning model. Scientific Reports, vol. 7, no. 1, Article number 4172, 2017. DOI: https://doi.org/10.1038/s41598-017-04075-z.

    Google Scholar 

  103. A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, S. Thrun. Dermatologist-level classification of skin cancer with deep neural networks. Nature, vol. 542, no. 7639, pp. 115–118, 2017. DOI: https://doi.org/10.1038/nature21056.

    Article  Google Scholar 

  104. J. De Fauw, J. R. Ledsam, B. Romera-Paredes, S. Nikolov, N. Tomasev, S. Blackwell, H. Askham, X. Glorot, B. O’Donoghue, D. Visentin, G. Van Den Driessche, B. Lakshminarayanan, C. Meyer, F. Mackinder, S. Bouton, K. Ayoub, R. Chopra, D. King, A. Karthikesalingam, C. O. Hughes, R. Raine, J. Hughes, D. A. Sim, C. Egan, A. Tufail, H. Montgomery, D. Hassabis, G. Rees, T. Back, P. T. Khaw, M. Suleyman, J. Cornebise, P. A. Keane, O. Ronneberger. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nature Medicine, vol. 24, no. 9, pp. 1342–1350, 2018. DOI: https://doi.org/10.1038/s41591-018-0107-6.

    Article  Google Scholar 

  105. M. Treder, J. L. Lauermann, N. Eter. Automated detection of exudative age-related macular degeneration in spectral domain optical coherence tomography using deep learning. Graefe’s Archive for Clinical and Experimental Ophthalmology, vol. 256, no. 2, pp. 259–265, 2018. DOI: https://doi.org/10.1007/s00417-017-3850-3.

    Article  Google Scholar 

  106. I. D. Apostolopoulos, T. A. Mpesiana. Covid-19: Automatic detection from X-ray images utilizing transfer learning with convolutional neural networks. Physical and Engineering Sciences in Medicine, vol. 43, no. 2, pp. 635–640, 2020. DOI: https://doi.org/10.1007/s13246-020-00865-4.

    Article  Google Scholar 

  107. M. M. Al Rahhal, Y. Bazi, M. Al Zuair, E. Othman, B. BenJdira. Convolutional neural networks for electrocardiogram classification. Journal of Medical and Biological Engineering, vol. 38, no. 6, pp. 1014–1025, 2018. DOI: https://doi.org/10.1007/s40846-018-0389-7.

    Article  Google Scholar 

  108. F. Demir, A. Sengur, V. Bajaj. Convolutional neural networks based efficient approach for classification of lung diseases. Health Information Science and Systems, vol. 8, no. 1, Article number 4, 2020. DOI: https://doi.org/10.1007/s13755-019-0091-3.

    Google Scholar 

  109. H. T. Shi, H. R. Wang, C. J. Qin, L. Q. Zhao, C. L. Liu. An incremental learning system for atrial fibrillation detection based on transfer learning and active learning. Computer Methods and Programs in Biomedicine, vol. 187, Article number 105219, 2020. DOI: https://doi.org/10.1016/j.cmpb.2019.105219.

  110. A. Shyam, V. Ravichandran, S. P. Preejith, J. Joseph, M. Sivaprakasam. PPGnet: Deep network for device independent heart rate estimation from photoplethysmogram. In Proceedings of the 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, Berlin, Germany, pp. 1899–1902, 2019. DOI: https://doi.org/10.1109/EMBC.2019.8856989.

    Google Scholar 

  111. Y. K. Li, S. Rao, J. R. A. Solares, A. Hassaine, R. Ramakrishnan, D. Canoy, Y. J. Zhu, K. Rahimi, G. Salimi-Khorshidi. BEHRT: Transformer for electronic health records. Scientific Reports, vol. 10, no. 1, Article number 7155, 2020. DOI: https://doi.org/10.1038/s41598-020-62922-y.

    Google Scholar 

  112. C. Matsoukas, J. F. Haslum, M. Sorkhei, M. Söderberg, K. Smith. What makes transfer learning work for medical images: Feature reuse & other factors. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, New Orleans, USA, pp. 9215–9224, 2022. DOI: https://doi.org/10.1109/CVPR52688.2022.00901.

    Google Scholar 

  113. T. Chen, S. Kornblith, M. Norouzi, G. Hinton. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, pp. 1597–1607, 2020.

  114. J. B. Grill, F. Strub, F. Altché, C. Tallec, P. H. Richemond, E. Buchatskaya, C. Doersch, B. Avila Pires, Z. D. Guo, M. Gheshlaghi Azar, B. Piot, K. Kavukcuoglu, R. Munos, M. Valko. Bootstrap your own latent a new approach to self-supervised learning. In Proceedings of the 34th International Conference on Neural Information Processing Systems, ACM, Vancouver, Canada, pp. 21271–21284, 2020.

    Google Scholar 

  115. M. Ishan, L. V. D. Maaten. Self-supervised learning of pretext-invariant representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6707–6717. 2020. DOI: https://doi.org/10.1109/cvpr42600.2020.00674.

  116. M. Caron, P. Bojanowski, J. Mairal, A. Joulin. Unsupervised pre-training of image features on non-curated data. In Proceedings of IEEE/CVF International Conference on Computer Vision, IEEE, Seoul, Republic of Korea, pp. 2959–2968, 2019. DOI: https://doi.org/10.1109/ICCV.2019.00305.

    Google Scholar 

  117. T. Chen, S. Kornblith, K. Swersky, M. Norouzi, G. E. Hinton. Big self-supervised models are strong semi-supervised learners. In Proceedings of the 34th Conference on Neural Information Processing Systems, Vancouver, Canada, pp. 22243–22255, 2020.

  118. A. Van Den Oord, Y. Z. Li, O. Vinyals. Representation learning with contrastive predictive coding. [Online], Available: https://arxiv.org/abs/1807.03748, 2018.

  119. K. M. He, H. Q. Fan, Y. X. Wu, S. N. Xie, R. Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Seattle, USA, pp. 9729–9738, 2020. DOI: https://doi.org/10.1109/CVPR42600.2020.00975.

    Google Scholar 

  120. M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, A. Joulin. Unsupervised learning of visual features by contrasting cluster assignments. In Proceedings of the 34th International Conference on Neural Information Processing Systems, ACM, Vancouver, Canada, pp. 9912–9924, 2020.

    Google Scholar 

  121. K. M. He, X. L. Chen, S. N. Xie, Y. H. Li, P. Dollár, R. Girshick. Masked autoencoders are scalable vision learners. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, New Orleans, USA, pp. 15979–15988, 2022. DOI: https://doi.org/10.1109/CV-PR52688.2022.01553.

    Google Scholar 

  122. X. L. Chen, K. M. He. Exploring simple Siamese representation learning. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Nashville, USA, pp. 15745–15753, 2021. DOI: https://doi.org/10.1109/CVPR46437.2021.01549.

    Google Scholar 

  123. O. J. Hénaff. Data-efficient image recognition with contrastive predictive coding. In Proceedings of the 37th International Conference on Machine Learning, pp. 4182–4192, 2020.

  124. X. L. Chen, H. Q. Fan, R. Girshick, K. M. He. Improved baselines with momentum contrastive learning. [Online], Available: https://arxiv.org/abs/2003.04297, 2020.

  125. X. L. Chen, S. N. Xie, K. M. He. An empirical study of training self-supervised vision transformers. In Proceedings of IEEE/CVF International Conference on Computer Vision, IEEE, Montreal, Canada, pp. 9620–9629, 2021. DOI: https://doi.org/10.1109/ICCV48922.2021.00950.

    Google Scholar 

  126. L. Rasmy, Y. Xiang, Z. Q. Xie, C. Tao, D. G. Zhi. Med-BERT: Pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. NPJ Digital Medicine, vol. 4, no. 1, Article number 86, 2021. DOI: https://doi.org/10.1038/s41746-021-00455-y.

    Google Scholar 

  127. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, ACM, Long Beach, USA, pp. 6000–6010, 2017.

    Google Scholar 

  128. J. Devlin, M. W. Chang, K. Lee, K. Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, USA, pp. 4171–4186, 2019. DOI: https://doi.org/10.18653/v1/N19-1423.

  129. S. M. Anwar, M. Majid, A. Qayyum, M. Awais, M. Alnowami, M. K. Khan. Medical image analysis using convolutional neural networks: A review. Journal of Medical Systems, vol. 42, no. 11, Article number 226, 2018. DOI: https://doi.org/10.1007/s10916-018-1088-1.

    Google Scholar 

  130. R. Paul, S. H. Hawkins, L. O. Hall, D. B. Goldgof, R. J. Gillies. Combining deep neural network and traditional image features to improve survival prediction accuracy for lung cancer patients from diagnostic CT. In Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, Budapest, Hungary, pp. 2570–2575, 2016. DOI: https://doi.org/10.1109/SMC.2016.7844626.

  131. M. W. Ren, N. Dey, M. A. Styner, K. Botteron, G. Gerig. Local spatiotemporal representation learning for longitudinally-consistent neuroimage analysis. [Online], Available: https://arxiv.org/abs/2206.04281, 2022.

  132. A. Bhandary, G. A. Prabhu, V. Rajinikanth, K. P. Thanaraj, S. C. Satapathy, D. E. Robbins, C. Shasky, Y. D. Zhang, J. M. R. Tavares, N. S. M. Raja. Deep-learning framework to detect lung abnormality-A study with chest X-ray and lung CT scan images. Pattern Recognition Letters, vol. 129, pp. 271–278, 2020. DOI: https://doi.org/10.1016/j.patrec.2019.11.013.

    Article  Google Scholar 

  133. D. S. Reddy, R. Bharath, P. Rajalakshmi. A novel computer-aided diagnosis framework using deep learning for classification of fatty liver disease in ultrasound imaging. In Proceedings of the 20th IEEE International Conference on E-health Networking, Applications and Services (Healthcom), Ostrava, Czech Republic, pp. 1–5, 2018. DOI: https://doi.org/10.1109/HealthCom.2018.8531118.

  134. C. Z. Wu, J. Sun, J. Wang, L. F. Xu, S. Zhan. Encoding-decoding network with pyramid self-attention module for retinal vessel segmentation. International Journal of Automation and Computing, vol. 18, no. 6, pp. 973–980, 2021. DOI: https://doi.org/10.1007/s11633-020-1277-0.

    Article  Google Scholar 

  135. J. Ker, L. P. Wang, J. Rao, T. Lim. Deep learning applications in medical image analysis. IEEE Access, vol. 6, pp. 9375–9389, 2017. DOI: https://doi.org/10.1109/ACCESS.2017.2788044.

    Article  Google Scholar 

  136. A. Fernandez-Quilez. Deep Learning for an Improved Diagnostic Pathway of Prostate Cancer in a Small Multi-Parametric Magnetic Resonance Data Regime, Ph.D. dissertation, University of Stavanger, Stavanger, Norway, 2022.

    Google Scholar 

  137. K. B. Ahmed, L. O. Hall, D. B. Goldgof, R. H. Liu, R. A. Gatenby. Fine-tuning convolutional deep features for MRI based brain tumor classification. In Proceedings of SPIE 10134, Medical Imaging 2017: Computer-Aided Diagnosis, Orlando, USA, Article number 101342E, 2017. DOI: https://doi.org/10.1117/12.2253982.

  138. R. M. Prakash, R. S. S. Kumari. Classification of MR brain images for detection of tumor with transfer learning from pre-trained CNN models. In Proceedings of the 2019 International Conference on Wireless Communications Signal Processing and Networking, IEEE, Chennai, India, pp. 508–511, 2019. DOI: https://doi.org/10.1109/WiSPNET45539.2019.9032811.

    Google Scholar 

  139. H. A. Khan, W. Jue, M. Mushtaq, M. U. Mushtaq. Brain tumor classification in MRI image using convolutional neural network. Mathematical Biosciences and Engineering, vol. 17, no. 5, pp. 6203–6216, 2020. DOI: https://doi.org/10.3934/mbe.2020328.

    Article  MathSciNet  Google Scholar 

  140. M. Sajjad, S. Khan, K. Muhammad, W. Q. Wu, A. Ullah, S. W. Baik. Multi-grade brain tumor classification using deep CNN with extensive data augmentation. Journal of Computational Science, vol. 30, pp. 174–182, 2019. DOI: https://doi.org/10.1016/j.jocs.2018.12.003.

    Article  Google Scholar 

  141. S. Deepak, P. M. Ameer. Brain tumor classification using deep CNN features via transfer learning. Computers in Biology and Medicine, vol. 111, Article number 103345, 2019. DOI: https://doi.org/10.1016/j.compbiomed.2019.103345.

  142. N. Noreen, S. Palaniappan, A. Qayyum, I. Ahmad, M. Imran, M. Shoaib. A deep learning model based on concatenation approach for the diagnosis of brain tumor. IEEE Access, vol. 8, pp. 55135–55144, 2020. DOI: https://doi.org/10.1109/ACCESS.2020.2978629.

    Article  Google Scholar 

  143. J. Cheng. Brain tumor dataset. Figshare, [Online], Available: https://doi.org/10.6084/m9.figshare.1512427.v5, 2017.

  144. Z. N. K. Swati, Q. H. Zhao, M. Kabir, F. Ali, Z. Ali, S. Ahmed, J. F. Lu. Brain tumor classification for MR images using transfer learning and fine-tuning. Computerized Medical Imaging and Graphics, vol. 75, pp. 34–46, 2019. DOI: https://doi.org/10.1016/j.compmedimag.2019.05.001.

    Article  Google Scholar 

  145. F. J. Díaz-Pernas, M. Martínez-Zarzuela, M. Antón-Rodríguez, D. González-Ortega. A deep learning approach for brain tumor classification and segmentation using a multiscale convolutional neural network. Healthcare, vol. 9, no. 2, Article number 153, 2021. DOI: https://doi.org/10.3390/healthcare9020153.

    Google Scholar 

  146. S. D. Wang, L. Y. Dong, X. Wang, X. G. Wang. Classification of pathological types of lung cancer from CT images by deep residual neural networks with transfer learning strategy. Open Medicine, vol. 15, no. 1, pp. 190–197, 2020. DOI: https://doi.org/10.1515/med-2020-0028.

    Article  Google Scholar 

  147. P. Marentakis, P. Karaiskos, V. Kouloulias, N. Kelekis, S. Argentos, N. Oikonomopoulos, C. Loukas. Lung cancer histology classification from CT images based on radiomics and deep learning models. Medical & Biological Engineering & Computing, vol. 59, no. 1, pp. 215–226, 2021. DOI: https://doi.org/10.1007/s11517-020-02302-w.

    Article  Google Scholar 

  148. H. Kutlu, E. Avcı. A novel method for classifying liver and brain tumors using convolutional neural networks, discrete wavelet transform and long short-term memory networks. Sensors, vol. 19, no. 9, Article number 1992, 2019. DOI: https://doi.org/10.3390/s19091992.

    Google Scholar 

  149. M. Byra, G. Styczynski, C. Szmigielski, P. Kalinowski, Ł. Michałowski, R. Palusekiewicz, B. Ziarkiewicz-Wróblewska, K. Zieniewicz, P. Sobieraj, A. Nowicki. Transfer learning with deep convolutional neural network for liver steatosis assessment in ultrasound images. International Journal of Computer Assisted Radiology and Surgery, vol. 13, no. 12, pp. 1895–1903, 2018. DOI: https://doi.org/10.1007/s11548-018-1843-2.

    Article  Google Scholar 

  150. M. Alkhaleefah, S. C. Ma, Y. L. Chang, B. Huang, P. K. Chittem, V. P. Achhannagari. Double-shot transfer learning for breast cancer classification from X-ray images. Applied Sciences, vol. 10, no. 11, Article number 3999, 2020. DOI: https://doi.org/10.3390/app10113999.

  151. S. G. Armato III, G. McLennan, L. Bidaut, M. F. McNitt-Gray, C. R. Meyer, A. P. Reeves, B. Zhao, D. R. Aberle, C. I. Henschke, E. A. Hoffman, E. A. Kazerooni, H. Macmahon, E. J. R. Van Beek, D. Yankelevitz, A. M. Biancardi, P. H. Bland, M. S. Brown, R. M. Engelmann, G. E. Laderach, D. Max, R. C. Pais, D. P. Y. Qing, R. Y. Roberts, A. R. Smith, A. Starkey, P. Batra, P. Caligiuri, A. Farooqi, G. W. Gladish, C. M. Jude, R. F. Munden, I. Petkovska, L. E. Quint, L. H. Schwartz, B. Sundaram, L. E. Dodd, C. Fenimore, D. Gur, N. Petrick, J. Freymann, J. Kirby, B. Hughes, A. Vande Casteele, S. Gupte, M. Sallam, M. D. Heath, M. H. Kuhn, E. Dharaiya, R. Burns, D. S. Fryd, M. Salganicoff, V. Anand, U. Shreter, S. Vastagh, B. Y. Croft, L. P. Clarke. The lung image database consortium (LIDC) and image database resource initiative (IDRI): A completed reference database of long nodules on CT scans. Medical Physics, vol. 38, no. 2, pp. 915–931, 2011. DOI: https://doi.org/10.1118/1.3528204.

    Article  Google Scholar 

  152. S. G. Armato III, G. McLennan, L. Bidaut, M. F. McNitt-Gray, C. R. Meyer, A. P. Reeves, B. Zhao, D. R. Aberle, C. I. Henschke, E. A. Hoffman, E. A. Kazerooni, H. MacMahon, E. J. R. Van Beek, D. Yankelevitz, A. M. Biancardi, P. H. Bland, M. S. Brown, R. M. Engelmann, G. E. Laderach, D. Max, R. C. Pais, D. P. Y. Qing, R. Y. Roberts, A. R. Smith, A. Starkey, P. Batra, P. Caligiuri, A. Farooqi, G. W. Gladish, C. M. Jude, R. F. Munden, I. Petkovska, L. E. Quint, L. H. Schwartz, B. Sundaram, L. E. Dodd, C. Fenimore, D. Gur, N. Petrick, J. Freymann, J. Kirby, B. Hughes, A. V. Casteele, S. Gupte, M. Sallam, M. D. Heath, M. H. Kuhn, E. Dharaiya, R. Burns, D. S., Fryd, M. Salganicoff, V. Anand, U. Shreter, S. Vastagh, B. Y. Croft, Clarke, L. P. Data From LIDC-ID-RI [Data set]. The Cancer Imaging Archive. [Online], Available: https://doi.org/10.7937/K9/TCIA.2015.LO9QL9SX, 2015.

  153. P. F. Christ, F. Ettlinger, F. Grün, M. E. A. Elshaera, J. Lipkova, S. Schlecht, F. Ahmaddy, S. Tatavarty, M. Bickel, P. Bilic, M. Rempfler, F. Hofmann, M. D. Anastasi, S. A. Ahmadi, G. Kaissis, J. Holch, W. Sommer, R. Braren, V. Heinemann, B. Menze. Automatic liver and tumor segmentation of CT and MRI volumes using cascaded fully convolutional neural networks. [Online], Available: https://arxiv.org/abs/1702.05970, 2017.

  154. O. Ronneberger, P. Fischer, T. Brox. U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of the 18th International Conference on Medical Image Computing and Computer-assisted Intervention, Springer, Munich, Germany, pp. 234–241, 2015. DOI: https://doi.org/10.1007/978-3-319-24574-4_28.

    Google Scholar 

  155. P. H. Conze, A. E. Kavur, E. Cornec-Le Gall, N. S. Gezer, Y. Le Meur, M. A. Selver, F. Rousseau. Abdominal multi-organ segmentation with cascaded convolutional and adversarial deep networks. Artificial Intelligence in Medicine, vol. 117, Article number 102109, 2021. DOI: https://doi.org/10.1016/j.artmed.2021.102109.

  156. M. J. Li, W. J. Cai, K. Verspoor, S. R. Pan, X. D. Liang, X. J. Chang. Cross-modal clinical graph transformer for ophthalmic report generation. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, New Orleans, USA, pp. 20656–20665, 2022. DOI: https://doi.org/10.1109/CVPR52688.2022.02000.

    Google Scholar 

  157. W. Gómez-Flores, W. C. de Albuquerque Pereira. A comparative study of pre-trained convolutional neural networks for semantic segmentation of breast tumors in ultrasound. Computers in Biology and Medicine, vol. 126, Article number 104036, 2020. DOI: https://doi.org/10.1016/j.compbiomed.2020.104036.

  158. H. Piotrzkowska-Wróblewska, K. Dobruch-Sobczak, M. Byra, A. Nowicki. Open access database of raw ultrasonic signals acquired from malignant and benign breast lesions. Medical Physics, vol. 44, no. 11, pp. 6105–6109, 2017. DOI: https://doi.org/10.1002/mp.12538.

    Article  Google Scholar 

  159. A. Hijab, M. A. Rushdi, M. M. Gomaa, A. Eldeib. Breast cancer classification in ultrasound images using transfer learning. In Proceedings of the 5th International Conference on Advances in Biomedical Engineering, IEEE, Tripoli, Lebanon, 2019. DOI: https://doi.org/10.1109/ICABME47164.2019.8940291.

    Book  Google Scholar 

  160. G. Ayana, K. Dese, S. W. Choe. Transfer learning in breast cancer diagnoses via ultrasound imaging Cancers, vol 13, no 4, Article number 738, 2021 DOI: https://doi.org/10.3390/cancers13040738.

    Google Scholar 

  161. G. Ayana, J. Park, J. W. Jeong, S. W. Choe. A novel multistage transfer learning for ultrasound breast cancer image classification. Diagnostics, vol. 12, no. 1, Article number 135, 2022. DOI: https://doi.org/10.3390/diagnostics12010135.

    Google Scholar 

  162. S. Sudharson, P. Kokil. An ensemble of deep neural networks for kidney ultrasound image classification. Computer Methods and Programs in Biomedicine, vol. 197, Article number 105709, 2020. DOI: https://doi.org/10.1016/j.cmpb.2020.105709.

  163. W. J. Bai, C. Chen, G. Tarroni, J. M. Duan, F. Guitton, S. E. Petersen, Y. K. Guo, P. M. Matthews, D. Rueckert. Self-supervised learning for cardiac MR image segmentation by anatomical position prediction. In Proceedings of the 22nd International Conference on Medical Image Computing and Computer Assisted Intervention, Springer, Shenzhen, China, pp. 541–549, 2019. DOI: https://doi.org/10.1007/978-3-030-32245-8_60.

    Google Scholar 

  164. Y. X. Li, J. W. Chen, X. P. Xie, K. Ma, Y. F. Zheng. Self-loop uncertainty: A novel pseudo-label for semi-supervised medical image segmentation. In Proceedings of the 23rd International Conference on Medical Image Computing and Computer Assisted Intervention, Springer, Lima, Peru, pp. 614–623, 2020. DOI: 01007/978-3-030-59710-8_60.

    Google Scholar 

  165. C. Doersch, A. Gupta, A. A. Efros. Unsupervised visual representation learning by context prediction. In Proceedings of IEEE International Conference on Computer Vision, Santiago, Chile, pp. 1422–1430, 2015. DOI: https://doi.org/10.1109/ICCV.2015.167.

  166. D. Pathak, P. Krähenbühl, J. Donahue, T. Darrell, A. A. Efros. Context encoders: Feature learning by inpainting. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 2536–2544, 2016. DOI: https://doi.org/10.1109/CVPR.2016.278.

  167. L. Chen, P. Bentley, K. Mori, K. Misawa, M. Fujiwara, D. Rueckert. Self-supervised learning for medical image analysis using image context restoration. Medical Image Analysis, vol. 58, Article number 101539, 2019. DOI: https://doi.org/10.1016/j.media.2019.101539.

  168. X. L. Zhu, J. W. Yao, J. Z. Huang. Deep convolutional neural network for survival analysis with pathological images. In Proceedings of IEEE International Conference on Bioinformatics and Biomedicine, Shenzhen, China, pp. 544–547, 2016. DOI: https://doi.org/10.1109/BIBM.2016.7822579.

  169. X. L. Zhu, J. W. Yao, F. Y. Zhu, J. Z. Huang. WSISA: Making survival prediction from whole slide histopathological images. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, pp. 7234–7242, 2017. DOI: https://doi.org/10.1109/CVPR.2017.725.

  170. K. A. Tran, O. Kondrashova, A. Bradley, E. D. Williams, J. V. Pearson, N. Waddell. Deep learning in cancer diagnosis, prognosis and treatment selection. Genome Medicine, vol. 13, no. 1, Article number 152, 2021. DOI: https://doi.org/10.1186/S13073-021-00968-X.

    Google Scholar 

  171. Y. Li, L. Wang, J. Wang, J. P. Ye, C. K. Reddy. Transfer learning for survival analysis via efficient L2, 1-norm regularized cox regression. In Proceedings of the 16th IEEE International Conference on Data Mining, IEEE, Barcelona, Spain, pp. 231–240, 2016. DOI: https://doi.org/10.1109/ICDM.2016.0034.

    Google Scholar 

  172. R. R. Agravat, M. S. Raval. Brain tumor segmentation and survival prediction. In Proceedings of the 5th International Workshop on Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Springer, Shenzhen, China, pp. 338–348, 2019. DOI: https://doi.org/10.1007/978-3-030-46640-4_32.

    Google Scholar 

  173. A. A. A. Setio, A. Traverso, T. De Bel, M. S. N. Berens, C. Van Den Bogaard, P. Cerello, H. Chen, Q. Dou, M. E. Fantacci, B. Geurts, R. Van Den Gugten, P. A. Heng, B. Jansen, M. M. J. De Kaste, V. Kotov, J. Y. H. Lin, J. T. M. C. Manders, A. Sóñora-Mengana, J. C. García-Naranjo, E. Papavasileiou, M. Prokop, M. Saletta, C. M. Schaefer-Prokop, E. T. Scholten, L. Scholten, M. M. Snoeren, E. L. Torres, J. Vandemeulebroucke, N. Walasek, G. C. A. Zuidhof, B. Van Ginneken, C. Jacobs. Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: The LUNA16 challenge. Medical Image Analysis, vol. 42, pp. 1–13, 2017. DOI: https://doi.org/10.1016/j.media.2017.06.015.

    Article  Google Scholar 

  174. R. J. Chen, C. K. Chen, Y. C. Li, T. Y. Chen, A. D. Trister, R. G. Krishnan, F. Mahmood. Scaling vision transformers to gigapixel images via hierarchical self-supervised learning. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, New Orleans, USA, pp. 16123–16134, 2022. DOI: https://doi.org/10.1109/CVPR52688.2022.01567.

    Google Scholar 

  175. Y. W. Xu, A. Hosny, R. Zeleznik, C. Parmar, T. Coroller, I. Franco, R. H. Mak, H. J. W. L. Aerts. Deep learning predicts lung cancer treatment response from serial medical imaging. Clinical Cancer Research, vol. 25, no. 11, pp. 3266–3275, 2019. DOI: https://doi.org/10.1158/1078-0432.CCR-18-2495.

    Article  Google Scholar 

  176. T. D. Pham. Time-frequency time-space long short-term memory networks for image classification of histopathological tissue. Scientific Reports, vol. 11, no. 1, Article number 13703, 2021. DOI: https://doi.org/10.1038/s41598-021-93160-5.

    Google Scholar 

  177. A. Konwer, X. Xu, J. Bae, C. Chen, P. Prasanna. Temporal context matters: Enhancing single image prediction with disease progression representations. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, New Orleans, USA, pp. 18802–18813, 2022. DOI: https://doi.org/10.1109/CVPR52688.2022.01826.

    Google Scholar 

  178. R. Q. Gao, Y. K. Huo, S. X. Bao, Y. C. Tang, S. L. Antic, E. S. Epstein, A. B. Balar, S. Deppen, A. B. Paulson, K. L. Sandler, P. P. Massion, B. A. Landman. Distanced LSTM: Time-distanced gates in long short-term memory models for lung cancer detection. In Proceedings of the 10th International Workshop on Machine Learning in Medical Imaging, Springer, Shenzhen, China, pp. 310–318, 2019. DOI: https://doi.org/10.1007/978-3-030-32692-0_36.

    Google Scholar 

  179. J. Ouyang, Q. Y. Zhao, E. Adeli, E. V. Sullivan, A. Pfefferbaum, G. Zaharchuk, K. M. Pohl. Self-supervised longitudinal neighbourhood embedding. In Proceedings of the 24th International Conference on Medical Image Computing and Computer Assisted Intervention, Springer, Strasbourg, France, pp. 80–89, 2021. DOI: https://doi.org/10.1007/978-3-030-87196-3_8.

    Google Scholar 

  180. K. Antczak. Deep recurrent neural networks for ECG signal denoising. [Online], Available: https://arxiv.org/abs/1807.11551, 2018.

  181. K. Antczak. A generative adversarial approach to ECG synthesis and denoising. [Online], Available: https://arxiv.org/abs/2009.02700, 2020.

  182. R. Rodrigues, P. Couto. Semi-supervised learning for ECG classification. In Proceedings of Computing in Cardiology, IEEE, Brno, Czech Republic, 2021. DOI: https://doi.org/10.23919/CinC53138.2021.9662693.

    Google Scholar 

  183. J. H. Jang, T. Y. Kim, D. Yoon. Effectiveness of transfer learning for deep learning-based electrocardiogram analysis. Healthcare Informatics Research, vol. 27, no. 1, pp. 19–28, 2021. DOI: https://doi.org/10.4258/hir.2021.27.1.19.

    Article  Google Scholar 

  184. M. T. Almalchy, S. M. S. ALGayar, N. Popescu. Atrial fibrillation automatic diagnosis based on ECG signal using pretrained deep convolution neural network and SVM multiclass model. In Proceedings of the 13th International Conference on Communications, IEEE, Bucharest, Romania, pp. 197–202, 2020. DOI: https://doi.org/10.1109/COMM48946.2020.9141994.

    Google Scholar 

  185. A. Qayyum, F. Mériaudeau, G. C. Y. Chan. Classification of atrial fibrillation with pre-trained convolutional neural network models. In Proceedings of IEEE/EMBS Conference on Biomedical Engineering and Sciences, IEEE, Sarawak, Malaysia, pp. 594–599, 2018. DOI: https://doi.org/10.1109/IECBES.2018.8626624.

    Google Scholar 

  186. D. Kiyasseh, T. T. Zhu, D. A. Clifton. CLOCS: Contrastive learning of cardiac signals across space, time, and patients. In Proceedings of the 38th International Conference on Machine Learning, pp. 5606–5615, 2021.

  187. D. Gedon, A. H. Ribeiro, N. Wahlström, T. B. Schön. First steps towards self-supervised pretraining of the 12-lead ECG. In Proceedings of Computing in Cardiology, IEEE, Brno, Czech Republic, 2021. DOI: https://doi.org/10.23919/CinC53138.2021.9662748.

    Book  Google Scholar 

  188. T. Mehari, N. Strodthoff. Self-supervised representation learning from 12-lead ECG data. Computers in Biology and Medicine, vol. 141, Article number 105114, 2022. DOI: https://doi.org/10.1016/j.compbiomed.2021.105114.

  189. H. Liu, Z. B. Zhao, Q. She. Self-supervised ECG pre-training. Biomedical Signal Processing and Control, vol. 70, Article number 103010, 2021. DOI: https://doi.org/10.1016/j.bspc.2021.103010.

  190. J. Y. Cheng, H. Goh, K. Dogrusoz, O. Tuzel, E. Azemi. Subject-aware contrastive learning for biosignals. [Online], Available: https://arxiv.org/abs/2007.04871, 2020.

  191. X. Zhang, Z. Y. Zhao, T. Tsiligkaridis, M. Zitnik. Self-supervised contrastive pre-training for time series via time-frequency consistency. [Online], Available: https://arxiv.org/abs2206.08496, 2022.

  192. H. Chen, G. J. Wang, G. D. Zhang, P. Zhang, H. Z. Yang. CLECG: A novel contrastive learning framework for electrocardiogram arrhythmia classification. IEEE Signal Processing Letters, vol. 28, pp. 1993–1997, 2021. DOI: https://doi.org/10.1109/LSP.2021.3114119.

    Article  Google Scholar 

  193. K. Radhika, V. R. M. Oruganti. Transfer learning for subject-independent stress detection using physiological signals. In Proceedings of the 17th IEEE India Council International Conference, New Delhi, India, 2020. DOI: https://doi.org/10.1109/INDICON49873.2020.9342505.

  194. P. Sarkar, S. Lobmaier, B. Fabre, D. González, A. Mueller, M. G. Frasch, M. C. Antonelli, A. Etemad. Detection of maternal and fetal stress from the electrocardiogram with self-supervised representation learning. Scientific Reports, vol. 11, no. 1, Article number 24146, 2021. DOI: https://doi.org/10.1038/S41598-021-03376-8.

    Google Scholar 

  195. P. Sarkar, A. Etemad. Self-supervised ECG representation learning for emotion recognition. IEEE Transactions on Affective Computing, vol. 13, no. 3, pp. 1541–1554, 2022. DOI: https://doi.org/10.1109/TAFFC.2020.3014842.

    Article  Google Scholar 

  196. P. J. Aston, J. V. Lyle, E. Bonet-Luz, C. L. Huang, Y. M. Zhang, K. Jeevaratnam, M. Nandi. Deep learning applied to attractor images derived from ECG signals for detection of genetic mutation. In Proceedings of Computing in Cardiology, IEEE, Singapore, pp. 1–4, 2019. DOI: https://doi.org/10.22489/CinC.2019.097.

    Google Scholar 

  197. Y. Cimtay, E. Ekmekcioglu. Investigating the use of pre-trained convolutional neural network on cross-subject and cross-dataset EEG emotion recognition. Sensors, vol. 20, no. 7, Article number 2034, 2020. DOI: https://doi.org/10.3390/s20072034.

    Google Scholar 

  198. S. Bagherzadeh, K. Maghooli, A. Shalbaf, A. Maghsoudi. Recognition of emotional states using frequency effective connectivity maps through transfer learning approach from electroencephalogram signals. Biomedical Signal Processing and Control, vol. 75, Article number 103544, 2022. DOI: https://doi.org/10.1016/j.bspc.2022.103544.

  199. M. N. Mohsenvand, M. R. Izadi, P. Maes. Contrastive representation learning for electroencephalogram classification. In Proceedings of the Machine Learning for Health, pp. 238–253, 2020.

  200. S. Raghu, N. Sriraam, Y. Temel, S. V. Rao, P. L. Kubben. EEG based multi-class seizure type classification using convolutional neural network and transfer learning. PMLR Neural Networks, vol. 124, pp. 202–212, 2020. DOI: https://doi.org/10.1016/j.neunet.2020.01.017.

    Article  Google Scholar 

  201. H. S. Nogay, H. Adeli. Detection of epileptic seizure using pretrained deep convolutional neural network and transfer learning. European Neurology, vol. 83, no. 6, pp. 602–614, 2020. DOI: https://doi.org/10.1159/000512985.

    Article  Google Scholar 

  202. S. Y. Tang, J. Dunnmon, K. K. Saab, X. Zhang, Q. Y. Huang, F. Dubost, D. Rubin, C. Lee-Messer. Self-supervised graph neural networks for improved electroencephalographic seizure analysis. In Proceedings of the 10th International Conference on Learning Representations, 2022.

  203. J. J. Xu, Y. J. Zheng, Y. F. Mao, R. X. Wang, W. S. Zheng. Anomaly detection on electroencephalography with self-supervised learning. In Proceedings of IEEE International Conference on Bioinformatics and Biomedicine, Seoul, Republic of Korea, pp. 363–368, 2020. DOI: https://doi.org/10.1109/BIBM49941.2020.9313163.

  204. H. Banville, O. Chehab, A. Hyvärinen, D. A. Engemann, A. Gramfort. Uncovering the structure of clinical EEG signals with self-supervised learning. Journal of Neural Engineering, vol. 18, no. 4, Article number 046020, 2021. DOI: https://doi.org/10.1088/1741-2552/abca18.

    Google Scholar 

  205. M. T. Sadiq, M. Z. Aziz, A. Almogren, A. Yousaf, S. Siuly, A. U. Rehman. Exploiting pretrained CNN models for the development of an EEG-based robust BCI framework. Computers in Biology and Medicine, vol. 143, Article number 105242, 2022. DOI: https://doi.org/10.1016/j.compbiomed.2022.105242.

  206. Y. H. Ou, S. Q. Sun, H. T. Gan, R. Zhou, Z. Yang. An improved self-supervised learning for EEG classification. Mathematical Biosciences and Engineering, vol. 19, no. 7, pp. 6907–6922, 2022. DOI: https://doi.org/10.3934/mbe.2022325.

    Article  Google Scholar 

  207. H. Phan, O. Y. Chén, P. Koch, Z. Q. Lu, I. McLoughlin, A. Mertins, M. De Vos. Towards more accurate automatic sleep staging via deep transfer learning. IEEE Transactions on Biomedical Engineering, vol. 68, no. 6, pp. 1787–1798, 2021. DOI: https://doi.org/10.1109/TBME.2020.3020381.

    Article  Google Scholar 

  208. X. Jiang, J. H. Zhao, B. Du, Z. Y. Yuan. Self-supervised contrastive learning for EEG-based sleep staging. In Proceedings of International Joint Conference on Neural Networks, IEEE, Shenzhen, China, 2021. DOI: https://doi.org/10.1109/IJCNN52387.2021.9533305.

    Book  Google Scholar 

  209. N. Wagh, J. H. Wei, S. Rawal, B. M. Berry, L. Barnard, B. Brinkmann, G. Worrell, D. Jones, Y. Varatharajah. Domain-guided self-supervision of EEG data improves downstream classification performance and generalizability. In Proceedings of Machine Learning for Health, pp. 130–142, 2021.

  210. R. W. Picard, E. Vyzas, J. Healey. Toward machine emotional intelligence: Analysis of affective physiological state. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 10, pp. 1175–1191, 2001. DOI: https://doi.org/10.1109/34.954607.

    Article  Google Scholar 

  211. D. R. Simkin, R. W. Thatcher, J. Lubar. Quantitative EEG and neurofeedback in children and adolescents: Anxiety disorders, depressive disorders, comorbid addiction and attention-deficit/hyperactivity disorder, and brain injury. Child and Adolescent Psychiatric Clinics of North America, vol. 23, no. 3, pp. 427–464, 2014. DOI: https://doi.org/10.1016/j.chc.2014.03.001.

    Article  Google Scholar 

  212. G. Z. Zhao, Y. Ge, B. Y. Shen, X. J. Wei, H. Wang. Emotion analysis for personality inference from EEG signals. IEEE Transactions on Affective Computing, vol. 9, no. 3, pp. 362–371, 2017. DOI: https://doi.org/10.1109/TAFFC.2017.2786207.

    Article  Google Scholar 

  213. N. Lu, T. F. Li, X. D. Ren, H. Y. Miao. A deep learning scheme for motor imagery classification based on restricted boltzmann machines. IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 6, pp. 566–576, 2017. DOI: https://doi.org/10.1109/TNSRE.2016.2601240.

    Article  Google Scholar 

  214. M. McDermott, B. Nestor, E. Kim, W. C. Zhang, A. Goldenberg, P. Szolovits, M. Ghassemi. A comprehensive EHR timeseries pre-training benchmark. In Proceedings of the Conference on Health, Inference, and Learning, ACM, pp. 257–278, 2021. DOI: https://doi.org/10.1145/3450439.3451877.

  215. H. Chen, S. M. Lundberg, G. Erion, J. H. Kim, S. I. Lee. Forecasting adverse surgical events using self-supervised transfer learning for physiological signals. Digital Medicine, vol. 4, no. 1, Article number 167, 2021. DOI: https://doi.org/10.1038/s41746-021-00536-y.

    Google Scholar 

  216. X. Xu, X. Xu, Y. Y. Sun, X. S. Liu, X. Li, G. T. Xie, F. Wang. Predictive modeling of clinical events with mutual enhancement between longitudinal patient records and medical knowledge graph. In Proceedings of IEEE International Conference on Data Mining, Auckland, New Zealand, pp. 777–786, 2021. DOI: https://doi.org/10.1109/ICDM51629.2021.00089.

  217. Y. Xue, N. Du, A. Mottram, M. Seneviratne, A. M. Dai. Learning to select best forecast tasks for clinical outcome prediction. In Proceedings of the 34th Conference on Neural Information Processing Systems, Vancouver, Canada, pp. 15031–15041, 2020.

  218. S. Tipirneni, C. K. Reddy. Self-supervised transformer for sparse and irregularly sampled multivariate clinical time-series. ACM Transactions on Knowledge Discovery from Data, vol. 1, no. 1, Article number 105, 2022. DOI: https://doi.org/10.1145/3516367.

    Google Scholar 

  219. H. X. Ren, J. Y. Wang, W. X. Zhao, N. Wu. RAPT: Pre-training of time-aware transformer for learning robust healthcare representation. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, Singapore, pp. 3503–3511, 2021. DOI: https://doi.org/10.1145/3447548.3467069.

  220. B. van Aken, J. M. Papaioannou, M. Mayrdorfer, K. Budde, F. Gers, A. Löser. Clinical outcome prediction from admission notes using self-supervised knowledge integration. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pp.881–893, 2021. DOI: https://doi.org/10.18653/v1/2021.eacl-main.75.

  221. C. Lu, C. K. Reddy, Y. Ning. Self-supervised graph learning with hyperbolic embedding for temporal health event prediction. IEEE Transactions on Cybernetics, to be published. DOI: https://doi.org/10.1109/TCYB.2021.3109881.

  222. K. Hur, J. Lee, J. Oh, W. Price, Y. Kim, E. Choi. Unifying heterogeneous electronic health records systems via text-based code embedding. In Proceedings of Conference on Health, Inference, and Learning, pp.183–203, 2022.

  223. S. Biswal, C. Xiao, L. M. Glass, E. Milkovits, J. M. Sun. Doctor2Vec: Dynamic doctor representation learning for clinical trial recruitment. Proceedings of AAAI Conference on Artificial Intelligence, vol. 34, no. 1, pp. 557–564, 2020. DOI: https://doi.org/10.1609/aaai.v34i01.5394.

    Article  Google Scholar 

  224. Y. P. Chen, Y. H. Lo, F. P. Lai, C. H. Huang. Disease concept-embedding based on the self-supervised method for medical information extraction from electronic health records and disease retrieval: Algorithm development and validation study. Journal of Medical Internet Research, vol. 23, no. 1, Article number e25113, 2021. DOI: https://doi.org/10.2196/25113.

    Google Scholar 

  225. E. Lehman, S. Jain, K. Pichotta, Y. Goldberg, B. C. Wallace. Does BERT pretrained on clinical notes reveal sensitive data? In Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 946–959, 2021. DOI: https://doi.org/10.18653/v1/2021.naacl-main.73.

  226. X. Y. Zhang, C. Xiao, L. M. Glass, J. M. Sun. DeepEnroll: Patient-trial matching with deep embedding and entailment prediction. In Proceedings of The Web Conference, ACM, Taipei, China, pp. 1029–1037, 2020. DOI: https://doi.org/10.1145/3366423.3380181.

    Google Scholar 

  227. H. D. Hlynsson, S. Ellertsson, J. F. Daðason, E. L. Sigurdsson, H. Loftsson. Semi-self-supervised automated ICD coding. [Online], Available: https://arxiv.org/abs/2205.10088, 2022.

  228. Z. Zhang, J. S. Liu, N. Razavian. BERT-XML: Large scale automated ICD coding using BERT pretraining. In Proceedings of the 3rd Clinical Natural Language Processing Workshop, pp. 24–34, 2020. DOI: https://doi.org/10.18653/v1/2020.clinicalnlp-1.3.

  229. Y. Q. Su, Y. L. Shi, W. Lee, L. Cheng, H. M. Guo. TAHDNet: Time-aware hierarchical dependency network for medication recommendation. Journal of Biomedical Informatics, vol. 129, Article number 104069, 2022. DOI: https://doi.org/10.1016/j.jbi.2022.104069.

  230. J. Y. Shang, T. F. Ma, C. Xiao, J. M. Sun. Pre-training of graph augmented transformers for medication recommendation. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, pp. 5953–5959, 2019. DOI: https://doi.org/10.24963/ijcai.2019/825.

  231. Z. Sun, S. L. Peng, Y. N. Yang, X. Q. Wang, F. Li. A general fine-tuned transfer learning model for predicting clinical task acrossing diverse EHRs datasets. In Proceedings of IEEE International Conference on Bioinformatics and Biomedicine, San Diego, USA, pp. 490–495, 2019. DOI: https://doi.org/10.1109/BIBM47256.2019.8983098.

  232. L. T. Ma, X. Y. Ma, J. Y. Gao, X. F. Jiao, Z. H. Yu, C. H. Zhang, W. J. Ruan, Y. S. Wang, W. Tang, J. T. Wang. Distilling knowledge from publicly available online EMR data to emerging epidemic for prognosis. In Proceedings of Web Conference, ACM, Ljubljana, Slovenia, pp. 3558–3568, 2021. DOI: https://doi.org/10.1145/3442381.3449855.

    Google Scholar 

  233. H. Quan, V. Sundararajan, P. Halfon, A. Fong, B. Burnand, J. C. Luthi, L. D. Saunders, C. A. Beck, T. E. Feasby, W. A. Ghali. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Medical Care, vol. 43, no. 11, pp. 1130–1139, 2005. DOI: https://doi.org/10.1097/01.mlr.0000182534.19832.83.

    Article  Google Scholar 

  234. Y. Y. Zhang, X. Wu, Q. Fang, S. S. Qian, C. S. Xu. Knowledge-enhanced attributed multi-task learning for medicine recommendation. ACM Transactions on Inormation Systems, to be published. DOI: https://doi.org/10.1145/3527662.

  235. Y. K. Li, H. Y. Wang, Y. Luo. A comparison of pre-trained vision-and-language models or multimodal representation learning across medical images and reports. In Proceedings of IEEE International Conference on Bioinformatics and Biomedicine, Seoul, Republic of Korea, pp. 1999–2004, 2020. DOI: https://doi.org/10.1109/BIBM49941.2020.9313289.

  236. L. H. Li, M. Yatskar, D. Yin, C. J. Hsieh, K. W. Chang. VisualBERT: A simple and performant baseline or vision and language. [Online], Available: https://arxiv.org/abs/1908.03557, 2019.

  237. Y. C. Chen, L. J. Li, L. C. Yu, A. El Kholy, F. Ahmed, Z. Gan, Y. Cheng, J. J. Liu. UNITER: UNiversal Image-TExt representation learning. [Online], Available: https://arxiv.org/abs/1909.11740, 2019.

  238. H. Tan, M. Bansal. LXMERT: Learning cross-modality encoder representations from transformers. In Proceedings of Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, pp. 5100–5111, 2019. DOI: https://doi.org/10.18653/v1/D19-1514.

  239. Z. C. Huang, Z. Y. Zeng, B. Liu, D. M. Fu, J. L. Fu. Pixel-BERT: Aligning image pixels with text by deep multi-modal transformers. [Online], Available: https://arxiv.org/abs/2004.00849, 2020.

  240. Y. Khare, V. Bagal, M. Mathew, A. Devi, U. D. Priyakumar, C. V. Jawahar. MMBERT: Multimodal BERT pre-training for improved medical VQA. In Proceedings of the 18th IEEE International Symposium on Biomedical Imaging, Nice, France, pp. 1033–1036, 2021. DOI: https://doi.org/10.1109/ISBI48211.2021.9434063.

  241. N. Rieke, J. Hancox, W. Q. Li, F. Milletarì, H. R. Roth, S. Albarqouni, S. Bakas, M. N. Galtier, B. A. Landman, K. Maier-Hein, S. Ourselin, M. Sheller, R. M. Summers, A. Trask, D. G. Xu, M. Baust, M. J. Cardoso. The future of digital health with federated learning. Digital Medicine, vol. 3, no. 1, Article number 119, 2020. DOI: https://doi.org/10.1038/s41746-020-00323-1.

    Google Scholar 

  242. M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, L. Zhang. Deep learning with differential privacy. In Proceedings of ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria, pp. 308–318, 2016. DOI: https://doi.org/10.1145/2976749.2978318.

Download references

Acknowledgements

This work was supported by 2021 UQ School of Information Technology and Electrical Engineering (ITEE) Research Support Funding, Cyber Research Seed Funding (No. 2021-R3), the University of Adelaide (No. 1531570) and New Staff Research Start-up Funds (No. NS-2102). Open Access funding enabled and organized by CAUL and its Member Institutions.

Author information

Authors and Affiliations

  1. The University of Queensland, Brisbane, 4072, Australia

    Yixuan Qiu, Feng Lin & Miao Xu

  2. The University of Adelaide, Adelaide, 5005, Australia

    Weitong Chen

Authors
  1. Yixuan Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feng Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weitong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Miao Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weitong Chen.

Additional information

Yixuan Qiu received the M. Sc. degree in electrical engineering from The University of Queensland, Australia in 2020. Currently, he is a Ph. D. degree candidate in data science at School of information Technology and Electrical Engineering, The University of Queensland, Australia.

His research interests include medical data analytic, self-supervised learning and federated learning.

Feng Lin received the M. Sc. degree from The University of Queensland, Australia in 2022. He is currently working at Wipro, Australia.

His research interests include weakly-supervised learning, data mining and deep learning.

Weitong Chen received the Ph. D. degree in computer science from The University of Queensland, Australia in 2020. He is currently a lecturer at The University of Adelaide, Australia.

His research interests include machine learning and its application to medical domains.

Miao Xu received the Ph. D. degree in machine learning from Nanjing University, China in 2017. She is a lecturer at The University of Queensland (UQ), Australia. Before joining UQ, she was a postdoctoral researcher at RIKEN, Japan.

Her research interests include weakly supervised learning and its application to the medical and cyber-security domains.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qiu, Y., Lin, F., Chen, W. et al. Pre-training in Medical Data: A Survey. Mach. Intell. Res. 20, 147–179 (2023). https://doi.org/10.1007/s11633-022-1382-8

Download citation

  • Received: 30 June 2022

  • Accepted: 18 October 2022

  • Published: 21 February 2023

  • Issue Date: April 2023

  • DOI: https://doi.org/10.1007/s11633-022-1382-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Keywords

  • Medical data
  • pre-training
  • transfer learning
  • self-supervised learning
  • medical image data
  • electrocardiograms (ECG) data
Use our pre-submission checklist

Avoid common mistakes on your manuscript.

Advertisement

Search

Navigation

  • Find a journal
  • Publish with us
  • Track your research

Discover content

  • Journals A-Z
  • Books A-Z

Publish with us

  • Journal finder
  • Publish your research
  • Open access publishing

Products and services

  • Our products
  • Librarians
  • Societies
  • Partners and advertisers

Our imprints

  • Springer
  • Nature Portfolio
  • BMC
  • Palgrave Macmillan
  • Apress
  • Your US state privacy rights
  • Accessibility statement
  • Terms and conditions
  • Privacy policy
  • Help and support
  • Cancel contracts here

Not affiliated

Springer Nature

© 2024 Springer Nature