Abstract
Recently, functional magnetic resonance imaging (fMRI) has been employed to classify brain disorders such as Alzheimer’s disease and autism spectrum disorder. Deep learning models have made significant progress in interpreting complex neural data in the context of the evolving field of fMRI analysis. In this study, we face the substantial challenge of simultaneously processing connections between brain regions and contextual representations on different time scales. This study proposes STCTb, a spatio-temporal collaborative Transformer block structure for fMRI time series. STCTb uniquely integrates multi-scale spatiotemporal information processing and enables the collaboration of temporal and spatial features through an innovative architecture, aiming to improve the sensitivity and specificity of the model in recognizing brain activity signals. While inheriting the cascade structure of the Swin Transformer, the mechanism proposes a two-branch block structure, which achieves rich and efficient global information integration at both temporal and spatial scales. Extensive experiments on the publicly available fMRI datasets ADNI and ABIDE-I have shown that the model using STCTb exhibits significantly superior performance compared to existing methods, and this innovative approach provides valuable insights for the further development of deep learning in the field of functional data analysis of brain diseases.
Supported by Chongqing Natural Science Foundation (CSTB2022NSCQ-MSX1415) and the Ministry of Education Humanities and Social Sciences Fund (23YJAZH129).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Gupta, G.L., Samant, N.P.: Current druggable targets for therapeutic control of Alzheimer’s disease. Contemp. Clin. Trials 109, 106549 (2021)
Yahata, N., et al.: A small number of abnormal brain connections predicts adult autism spectrum disorder. Nat. Commun. 7(1), 11254 (2016)
Matthews, P.M., Jezzard, P.: Functional magnetic resonance imaging. J. Neurol. Neurosurg. Psychiatr. 75(1), 6–12 (2004)
Hillman, E.M.: Coupling mechanism and significance of the bold signal: a status report. Annu. Rev. Neurosci. 37, 161–181 (2014)
Heinsfeld, A.S., Franco, A.R., Craddock, R.C., Buchweitz, A., Meneguzzi, F.: Identification of autism spectrum disorder using deep learning and the abide dataset. NeuroImage: Clin. 17, 16–23 (2018)
Xia, L., et al.: A nested parallel multiscale convolution for cerebrovascular segmentation. Med. Phys. 48(12), 7971–7983 (2021)
Li, X., et al.: BrainGNN: interpretable brain graph neural network for fMRI analysis. Med. Image Anal. 74, 102233 (2021)
Malkiel, I., Rosenman, G., Wolf, L., Hendler, T.: Pre-training and fine-tuning transformers for fMRI prediction tasks 105. arXiv:2112.05761 (2021)
Bedel, H.A., Sivgin, I., Dalmaz, O., Dar, S.U., Çukur, T.: BolT: fused window transformers for fMRI time series analysis. Med. Image Anal. 88, 102841 (2023)
Qin, Z., Liu, Z., Zhu, P.: Aiding Alzheimer’s disease diagnosis using graph convolutional networks based on RS-fMRI data. In: 2022 15th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pp. 1–7. IEEE (2022)
Alorf, A., Khan, M.U.G.: Multi-label classification of Alzheimer’s disease stages from resting-state fMRI-based correlation connectivity data and deep learning. Comput. Biol. Med. 151, 106240 (2022)
Cui, W., et al.: Personalized functional connectivity based spatio-temporal aggregated attention network for mci identification. IEEE Trans. Neural Syst. Rehabil. Eng. (2023)
Zhang, H., et al.: Classification of brain disorders in RS-fMRI via local-to-global graph neural networks. IEEE Trans. Med. Imaging 42(2), 444–455 (2022)
Kim, B.H., Ye, J.C., Kim, J.J.: Learning dynamic graph representation of brain connectome with spatio-temporal attention. Adv. Neural. Inf. Process. Syst. 34, 4314–4327 (2021)
Su, L., Wang, L., Shen, H., Feng, G., Hu, D.: Discriminative analysis of non-linear brain connectivity in schizophrenia: an fMRI study. Front. Hum. Neurosci. 7, 702 (2013)
Zhao, C., Li, H., Jiao, Z., Du, T., Fan, Y.: A 3D convolutional encapsulated long short-term memory (3DConv-LSTM) model for denoising fMRI data. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part VII 23, pp. 479–488. Springer (2020)
Li, W., Lin, X., Chen, X.: Detecting Alzheimer’s disease based on 4D fMRI: an exploration under deep learning framework. Neurocomputing 388, 280–287 (2020)
Dvornek, N.C., Li, X., Zhuang, J., Duncan, J.S.: Jointly discriminative and generative recurrent neural networks for learning from fMRI. In: Machine Learning in Medical Imaging: 10th International Workshop, MLMI 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 13, 2019, Proceedings, vol. 10, pp. 382–390. Springer (2019)
Guo, X., Tinaz, S., Dvornek, N.C.: Characterization of early stage Parkinson’s disease from resting-state fMRI data using a long short-term memory network. Front. Neuroimaging 1, 952084 (2022)
Nguyen, S., Ng, B., Kaplan, A.D., Ray, P.: Attend and decode: 4D fMRI task state decoding using attention models. In: Machine Learning for Health, pp. 267–279. PMLR (2020)
Zhang, J., Zhou, L., Wang, L., Liu, M., Shen, D.: Diffusion kernel attention network for brain disorder classification. IEEE Trans. Med. Imaging 41(10), 2814–2827 (2022)
Dosovitskiy, A., et al.: An image is worth 16 \(\times \) 16 words: transformers for image recognition at scale. arXiv:2010.11929 (2020)
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016)
Yang, L., Zhang, R.Y., Li, L., Xie, X.: Simam: a simple, parameter-free attention module for convolutional neural networks. In: International Conference on Machine Learning, pp. 11863–11874. PMLR (2021)
Woo, S., Park, J., Lee, J.Y., Kweon, I.S.: CBAM: convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)
Lee, H., Kim, H.E., Nam, H.: SRM: a style-based recalibration module for convolutional neural networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1854–1862 (2019)
Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021)
Yang, J., et al.: Focal self-attention for local-global interactions in vision transformers. arXiv:2107.00641 (2021)
Ashburner, J.: A fast diffeomorphic image registration algorithm. Neuroimage 38(1), 95–113 (2007)
Craddock, C., et al.: Towards automated analysis of connectomes: the configurable pipeline for the analysis of connectomes (C-PAC). Front. Neuroinform. 42(10.3389) (2013)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 (2014)
Tzourio-Mazoyer, N., et al.: Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage 15(1), 273–289 (2002)
Abraham, A., et al.: Deriving reproducible biomarkers from multi-site resting-state data: an autism-based example. Neuroimage 147, 736–745 (2017)
Kam, T.E., Zhang, H., Jiao, Z., Shen, D.: Deep learning of static and dynamic brain functional networks for early mci detection. IEEE Trans. Med. Imaging 39(2), 478–487 (2019)
Sarraf, S., Sarraf, A., DeSouza, D.D., Anderson, J.A., Kabia, M., Initiative, A.D.N.: OViTAD: optimized vision transformer to predict various stages of Alzheimer’s disease using resting-state fMRI and structural MRI data. Brain Sci. 13(2), 260 (2023)
Kawahara, J., et al.: BrainNetCNN: convolutional neural networks for brain networks; towards predicting neurodevelopment. Neuroimage 146, 1038–1049 (2017)
Dvornek, N.C., Ventola, P., Pelphrey, K.A., Duncan, J.S.: Identifying autism from resting-state fMRI using long short-term memory networks. In: Machine Learning in Medical Imaging: 8th International Workshop, MLMI 2017, Held in Conjunction with MICCAI 2017, Quebec City, QC, Canada, September 10, 2017, Proceedings 8, pp. 362–370. Springer (2017)
Román, G., Pascual, B.: Contribution of neuroimaging to the diagnosis of Alzheimer’s disease and vascular dementia. Arch. Med. Res. 43(8), 671–676 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yan, Y., Shan, K., Li, W. (2025). STCTb: A Spatio-Temporal Collaborative Transformer Block for Brain Diseases Classification Using fMRI Time Series. In: Lin, Z., et al. Pattern Recognition and Computer Vision. PRCV 2024. Lecture Notes in Computer Science, vol 15045. Springer, Singapore. https://doi.org/10.1007/978-981-97-8499-8_6
Download citation
DOI: https://doi.org/10.1007/978-981-97-8499-8_6
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-8498-1
Online ISBN: 978-981-97-8499-8
eBook Packages: Computer ScienceComputer Science (R0)