Abstract
Benchmarking brain–computer interface (BCI) applications, considering all available smart training environment (STE) criteria, is a challenging task due to the following issues: inconsistent weights, static ranks and ranking stability measurements. Therefore, this study aims to develop a dynamic decision-making framework for benchmarking BCI applications based on STE criteria through three integrated phases. In the first phase, the adaptivity of the decision matrix is identified concerning two dimensions: 27 BCI applications as alternatives and 10 STE criteria. In the second phase, the consistency of weights is evaluated and constructed to each STE criterion via the fuzzy-weighted zero-inconsistency (FWZIC) method and the VIekriterijumsko KOmpromisno Rangiranje (VIKOR) method for benchmarking the BCI applications. In the third phase, four sensitive scenarios are developed for measuring the consistency of the STE criteria’s weights and the ranking performance of the BCI applications. The experimental result shows that the ‘ease of use’ STE criterion obtains a high-affected weight with a value of 0.13, while other criteria, augmented reality, hybrid and desktop use (stationary), obtain less weight with a 0.075 value. Additionally, BCI applications A5 and A6 are robust and stable among the others based on the consistency of weights concerning the four scenarios, and they are further candidates to be deployed in real-life applications. The overall ranking results are stable and less affected when applied to the four sensitive scenarios due to the robustness of the integrated FWZIC-VIKOR method of the proposed dynamic framework. The outcome of this framework is objectively validated in terms of five groups, and the ranking results are reliable and the closest to the decision-makers' viewpoints. The proposed framework considers a good solution for choosing a dependable application to support the user and community of BCI systems with a stable STE environment.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Availability of data and materials
The datasets supporting the conclusions of this article are included within the article (and its presented in Table 1).
References
Abdulkader SN et al (2015) Brain computer interfacing: Applications and challenges. Egypt Inf J 16:213–230
Aggarwal S, Chugh N (2022) Review of machine learning techniques for EEG based brain computer interface. Arch Comput Methods Eng 29(5):1–20
Kohli V et al (2022) A review on virtual reality and augmented reality use-cases of brain computer interface based applications for smart cities. Microprocess Microsyst 88:104392
Al-Canaan A et al (2022) BCI-control and monitoring system for smart home automation using wavelet classifiers. IET Signal Proc 16:141–156
Sharma R et al (2022) Motor imagery classification in brain-machine interface with machine learning algorithms: classical approach to multi-layer perceptron model. Biomed Signal Process Control 71:103101
Khan MM et al (2021) Research and development of a brain-controlled wheelchair for paralyzed patients. Intell Autom Soft Comput 30:49–64
Petoku E and Capi G (2021) Object Movement Motor Imagery for EEG based BCI System using Convolutional Neural Networks. In: 2021 9th International Winter Conference on Brain-Computer Interface (BCI), pp. 1–5
Martínez-Cagigal V et al (2021) Brain–computer interfaces based on code-modulated visual evoked potentials (c-VEP): a literature review. J Neural Eng 18(6):061002
Brusini L et al (2021) A systematic review on motor-imagery brain-connectivity-based computer interfaces. IEEE Trans Human-Machine Syst 51(6):725–733
Han D-ID et al (2022) Virtual reality consumer experience escapes: preparing for the metaverse. Virtual Reality 26(4):1–16
Scurati GW et al (2021) Exploring the use of virtual reality to support environmentally sustainable behavior: a framework to design experiences. Sustainability 13:943
Schroeter R (2022) Inception of Perception—Augmented Reality in Virtual Reality: Prototyping Human–Machine Interfaces for Automated Driving. In: User Experience Design in the Era of Automated Driving, ed: Springer, pp. 477–503.
räker J et al. (2022) Conceptualizing Interactions of Augmented Reality Solutions. In: HICSS, pp. 1–10
Chen H et al (2021) Development of BIM, IoT and AR/VR technologies for fire safety and upskilling. Autom Constr 125:103631
Hu H et al (2022) Navigation in virtual and real environment using brain computer interface: a progress report. Virtual Real Intell Hardware 4:89–114
Yue L et al (2021) Exploring BCI control in smart environments: intention recognition via EEG representation enhancement learning. ACM Trans Knowl Discov Data (TKDD) 15:1–20
Fang B et al (2022) Brain-computer interface integrated with augmented reality for human-robot interaction. IEEE Trans Cognit Dev Syst. https://doi.org/10.1109/TCDS.2022.3194603
Al-Qaysi Z et al (2021) Systematic review of training environments with motor imagery brain–computer interface: Coherent taxonomy, open issues and recommendation pathway solution. Heal Technol 11:783–801
Longo BB et al. (2014) Using brain-computer interface to control an avatar in a virtual reality environment. In: 5th ISSNIP-IEEE Biosignals and Biorobotics Conference (2014): Biosignals and Robotics for Better and Safer Living (BRC), pp. 1-4
Choi J and Jo S (2020) Application of Hybrid Brain-Computer Interface with Augmented Reality on Quadcopter Control. In: 2020 8th International Winter Conference on Brain-Computer Interface (BCI), pp. 1–5
Vourvopoulos A, Bermúdez i Badia S (2016) Motor priming in virtual reality can augment motor-imagery training efficacy in restorative brain-computer interaction: a within-subject analysis. J Neuroeng Rehabil 13:1–14
Liang S et al (2016) Improving the discrimination of hand motor imagery via virtual reality based visual guidance. Comput Methods Programs Biomed 132:63–74
Chin ZY et al. (2010) Online performance evaluation of motor imagery BCI with augmented-reality virtual hand feedback. In: 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, pp. 3341-3344
Velasco-Álvarez F et al (2013) Audio-cued motor imagery-based brain–computer interface: navigation through virtual and real environments. Neurocomputing 121:89–98
Vourvopoulos A et al (2019) Efficacy and brain imaging correlates of an immersive motor imagery BCI-driven VR system for upper limb motor rehabilitation: a clinical case report. Front Hum Neurosci 13:244
Al-Qaysi Z et al (2022) A systematic rank of smart training environment applications with motor imagery brain-computer interface. Multimed Tools Appl 82(12):1–23
Vasiljevic GAM, de Miranda LC (2020) Brain–computer interface games based on consumer-grade EEG Devices: a systematic literature review. Int J Human-Comput Inter 36:105–142
Salih MM et al (2022) A new extension of fuzzy decision by opinion score method based on Fermatean fuzzy: a benchmarking COVID-19 machine learning methods. J Intell Fuzzy Syst 43:1–11
Zaidan A et al (2018) A review on smartphone skin cancer diagnosis apps in evaluation and benchmarking: coherent taxonomy, open issues and recommendation pathway solution. Heal Technol 8:223–238
Mohsin AH et al (2019) Based medical systems for patient’s authentication: towards a new verification secure framework using CIA standard. J Med Syst 43:1–34
Alsalem M et al (2019) Multiclass benchmarking framework for automated acute Leukaemia detection and classification based on BWM and group-VIKOR. J Med Syst 43:1–32
Alnoor A et al (2022) Toward a sustainable transportation industry: Oil company benchmarking based on the extension of linear diophantine fuzzy rough sets and multicriteria decision-making methods. IEEE Trans Fuzzy Syst 31(2):449–459
Alamleh A et al (2022) Multi-attribute decision-making for intrusion detection systems: a systematic review. Int J Inform Technol Decision Making 22:1–48
Al-Qaysi Z et al (2018) A review of disability EEG based wheelchair control system: coherent taxonomy, open challenges and recommendations. Comput Methods Programs Biomed 164:221–237
Albahri AS et al (2020) Multi-biological laboratory examination framework for the prioritization of patients with COVID-19 based on integrated AHP and group VIKOR methods. Int J Inf Technol Decis Mak 19:1247–1269
Siuly S et al (2019) Exploring Hermite transformation in brain signal analysis for the detection of epileptic seizure. IET Sci Meas Technol 13:35–41
Cage E, Troxell-Whitman Z (2019) Understanding the reasons, contexts and costs of camouflaging for autistic adults. J Autism Dev Disord 49:1899–1911
Nguyen CT et al (2016) Sociodemographic disparities in intervention service utilization in families of children with autism spectrum disorder. J Autism Dev Disord 46:3729–3738
DiGuiseppi CG et al (2016) Demographic profile of families and children in the study to explore early development (SEED): Case-control study of autism spectrum disorder. Disabil Health J 9:544–551
Khanna R et al (2011) Assessment of health-related quality of life among primary caregivers of children with autism spectrum disorders. J Autism Dev Disord 41:1214–1227
Mohammed R et al (2022) Determining importance of many-objective optimisation competitive algorithms evaluation criteria based on a novel fuzzy-weighted zero-inconsistency method. Int J Inf Technol Decis Mak 21:195–241
Zughoul O et al (2021) Novel triplex procedure for ranking the ability of software engineering students based on two levels of AHP and group TOPSIS techniques. Int J Inf Technol Decis Mak 20:67–135
Khatari M et al (2021) Multidimensional benchmarking framework for AQMs of network congestion control based on AHP and Group-TOPSIS. Int J Inf Technol Decis Mak 20:1409–1446
Csató L (2018) Characterization of an inconsistency ranking for pairwise comparison matrices. Ann Oper Res 261:155–165
Bozóki S et al (2016) An application of incomplete pairwise comparison matrices for ranking top tennis players. Eur J Oper Res 248:211–218
Albahri AS et al (2021) Detection-based prioritisation: Framework of multi-laboratory characteristics for asymptomatic COVID-19 carriers based on integrated Entropy–TOPSIS methods. Artif Intell Med 111:101983
Mohammed TJ et al (2022) Improving the prioritization procedure of patients with COVID-19 in hospitals based on decision-making techniques: a systematic review. Iraqi J Comput Inf 48:6–12
Baqer NS et al (2022) Indoor air quality pollutants predicting approach using unified labelling process-based multi-criteria decision making and machine learning techniques. Telecommun Syst 81:591–613
Krishnan E et al (2021) Interval type 2 trapezoidal-fuzzy weighted with zero inconsistency combined with VIKOR for evaluating smart e-tourism applications. Int J Intell Syst 36:4723–4774
Albahri O et al (2022) Novel dynamic fuzzy decision-making framework for COVID-19 vaccine dose recipients. J Adv Res 37:147–168
Alsalem M et al (2021) Based on T-spherical fuzzy environment: a combination of FWZIC and FDOSM for prioritising COVID-19 vaccine dose recipients. J Infect Public Health 14:1513–1559
Albahri AS et al (2022) Integration of fuzzy-weighted zero-inconsistency and fuzzy decision by opinion score methods under a q-rung orthopair environment: a distribution case study of COVID-19 vaccine doses. Comput Standards Int 80:103572
Alamoodi A et al (2022) Based on neutrosophic fuzzy environment: a new development of FWZIC and FDOSM for benchmarking smart e-tourism applications. Complex Intell Syst 8(4):1–25
Alamoodi A et al (2022) New extension of fuzzy-weighted zero-inconsistency and fuzzy decision by opinion score method based on cubic pythagorean fuzzy environment: a benchmarking case study of sign language recognition systems. Int J Fuzzy Syst 24(4):1–18
Joudar SS et al (2022) Intelligent triage method for early diagnosis autism spectrum disorder (ASD) based on integrated fuzzy multi-criteria decision-making methods. Inf Med Unlock 36:101131
Alqaysi M et al (2022) Hybrid diagnosis models for autism patients based on medical and sociodemographic features using machine learning and multicriteria decision-making (MCDM) techniques: an evaluation and benchmarking framework. Comput Math Methods Med 2022:19
Alamoodi A et al (2022) Hospital selection framework for remote MCD patients based on fuzzy q-rung orthopair environment. Neural Comput Appl 35:1–12
Alsalem M et al (2022) Rescuing emergency cases of COVID-19 patients: An intelligent real-time MSC transfusion framework based on multicriteria decision-making methods. Appl Intell 52:1–25
Albahri O et al (2022) Combination of fuzzy-weighted zero-inconsistency and fuzzy decision by opinion score methods in pythagorean m-polar fuzzy environment: a case study of sing language recognition systems. Int J Inf Technol Decision Making 22:1–29
Al-Samarraay MS et al (2022) A new extension of FDOSM based on Pythagorean fuzzy environment for evaluating and benchmarking sign language recognition systems. Neural Comput Appl 34:4937–4955
Mohammed K et al (2020) Novel technique for reorganisation of opinion order to interval levels for solving several instances representing prioritisation in patients with multiple chronic diseases. Comput Methods Programs Biomed 185:105151
Albahri OS et al (2021) Multidimensional benchmarking of the active queue management methods of network congestion control based on extension of fuzzy decision by opinion score method. Int J Intell Syst 36:796–831
Al-Samarraay MS et al (2022) Extension of interval-valued Pythagorean FDOSM for evaluating and benchmarking real-time SLRSs based on multidimensional criteria of hand gesture recognition and sensor glove perspectives. Appl Soft Comput 116:108284
Mohammed K et al (2020) A uniform intelligent prioritisation for solving diverse and big data generated from multiple chronic diseases patients based on hybrid decision-making and voting method. IEEE Access 8:91521–91530
Almahdi E et al (2019) Mobile-based patient monitoring systems: a prioritisation framework using multi-criteria decision-making techniques. J Med Syst 43:1–19
Albahri OS et al (2022) New mHealth hospital selection framework supporting decentralised telemedicine architecture for outpatient cardiovascular disease-based integrated techniques: Haversine-GPS and AHP-VIKOR. J Ambient Intell Humaniz Comput 13:219–239
Kim D-H et al (2021) Introduction of brain computer interface to neurologists. Annals of Clinical Neurophysiol 23:92–98
Palumbo A et al (2021) Biopotential signal monitoring systems in rehabilitation: a review. Sensors 21:7172
Mudgal SK et al (2020) Brain computer interface advancement in neurosciences: applications and issues. Interdiscip Neurosurg 20:100694
Vourvopoulos A, i Badia SB (2016) Motor priming in virtual reality can augment motor-imagery training efficacy in restorative brain-computer interaction: a within-subject analysis. J Neuroeng Rehabil 13:1–14
Aamer A et al. (2019) BCI Integrated with VR for Rehabilitation. In: 2019 31st International Conference on Microelectronics (ICM), pp 166–169
Alchalabi B, Faubert J (2019) A comparison between BCI simulation and neurofeedback for forward/backward navigation in virtual reality. Comput Intell Neurosci. https://doi.org/10.1155/2019/2503431
Škola F, Liarokapis F (2018) Embodied VR environment facilitates motor imagery brain–computer interface training. Comput Graph 75:59–71
Kwon BH et al. (2020) A Novel Framework for Visual Motion Imagery Classification Using 3D Virtual BCI Platform. In: 2020 8th International Winter Conference on Brain-Computer Interface (BCI), pp 1–5
Liu X et al. (2017) Performance evaluation of walking imagery training based on virtual environment in brain-computer interfaces. In: 2017 IEEE International Symposium on Multimedia (ISM), pp 25-30
Afdideh F et al. (2012) Development of a MATLAB-based toolbox for brain computer interface applications in virtual reality. In: 20th Iranian Conference on Electrical Engineering (ICEE2012), pp 1579–1583
Yeh S-C et al (2018) A multiplayer online car racing virtual-reality game based on internet of brains. J Syst Architect 89:30–40
i Badia SB et al (2012) Using a hybrid brain computer interface and virtual reality system to monitor and promote cortical reorganization through motor activity and motor imagery training. IEEE Trans Neur Syst Rehabil Eng 21:174–181
Huang D et al (2012) Electroencephalography (EEG)-based brain–computer interface (BCI): A 2-D virtual wheelchair control based on event-related desynchronization/synchronization and state control. IEEE Trans Neural Syst Rehabil Eng 20:379–388
Yang F et al. (2010) An adaptive BCI system for virtual navigation. In: The 2nd International Conference on Information Science and Engineering, pp 64–68
Achanccaray D et al. (2018) Immersive virtual reality feedback in a brain computer interface for upper limb rehabilitation. In: 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp 1006–1010
Chin ZY et al. (2013) Navigation in a virtual environment using multiclass motor imagery Brain-Computer Interface. In: 2013 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB), pp 152–157
Cantillo-Negrete J et al (2019) Robotic orthosis compared to virtual hand for brain-computer interface feedback. Biocybernetics Biomed Eng 39:263–272
Ren S et al (2020) Enhanced motor imagery based brain-computer interface via FES and VR for lower limbs. IEEE Trans Neural Syst Rehabil Eng 28:1846–1855
Xia B et al. (2010) The training strategy in brain-computer interface. In: 2010 Sixth International Conference on Natural Computation, pp 2190-2193
Song M, Kim J (2019) A paradigm to enhance motor imagery using rubber hand illusion induced by visuo-tactile stimulus. IEEE Trans Neural Syst Rehabil Eng 27:477–486
Škola F et al (2019) Progressive training for motor imagery brain-computer interfaces using gamification and virtual reality embodiment. Front Hum Neurosci 13:329
Liang S et al. (2014) Effective user training for motor imagery based brain computer interface with object-directed 3D visual display. In: 2014 7th International Conference on Biomedical Engineering and Informatics, pp 297–301
Lotte F et al. (1999) Electroencephalography (EEG)‐based brain–computer interfaces. Wiley Encyclopedia of Electrical and Electronics Engineering, pp 1–20
Albahri O et al (2021) New mHealth hospital selection framework supporting decentralised telemedicine architecture for outpatient cardiovascular disease-based integrated techniques: Haversine-GPS and AHP-VIKOR. J Ambient Intell Humani Comput 13:1–21
Alsalem MA et al (2021) Rise of multiattribute decision-making in combating COVID-19: a systematic review of the state-of-the-art literature. Int J Intell Syst 37:3514–3624
Alsalem M et al (2022) Multi-criteria decision-making for coronavirus disease 2019 applications: a theoretical analysis review. Artif Intell Rev 55:1–84
Kalid N et al (2018) Based on real time remote health monitoring systems: a new approach for prioritization “large scales data” patients with chronic heart diseases using body sensors and communication technology. J Med Syst 42:69
Abdulkareem KH et al (2020) A novel multi-perspective benchmarking framework for selecting image dehazing intelligent algorithms based on BWM and group VIKOR techniques. Int J Inf Technol Decision Making 19:1–49
Albahri OS et al (2021) Multidimensional benchmarking of the active queue management methods of network congestion control based on extension of fuzzy decision by opinion score method. Int J Intell Syst 36(2):796–831
Alamleh A et al (2022) Federated learning for IoMT applications: a standardisation and benchmarking framework of intrusion detection systems. IEEE J Biomed Health Inf 27:878–887
Abdulkareem KH et al (2020) A new standardisation and selection framework for real-time image dehazing algorithms from multi-foggy scenes based on fuzzy Delphi and hybrid multi-criteria decision analysis methods. Neural Comput Appl 33:1029–1054
Mohammed K et al (2020) Novel technique for reorganisation of opinion order to interval levels for solving several instances representing prioritisation in patients with multiple chronic diseases. Comput Methods Programs Biomed 185:105151
Abdulkareem KH (2020) A novel multi-perspective benchmarking framework for selecting image dehazing intelligent algorithms based on bwm and group vikor techniques. Int J Inf Technol Decision Making 19(03):909–957
Funding
A.S. Albahri would like to acknowledge the support of Technical College, Imam Ja’afar Al-Sadiq University, Baghdad, Iraq.
Author information
Authors and Affiliations
Contributions
The authors contributed equally to this work.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no competing interests.
Ethics approval
Not applicable.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Al-qaysi, Z.T., Albahri, A.S., Ahmed, M.A. et al. Dynamic decision-making framework for benchmarking brain–computer interface applications: a fuzzy-weighted zero-inconsistency method for consistent weights and VIKOR for stable rank. Neural Comput & Applic 36, 10355–10378 (2024). https://doi.org/10.1007/s00521-024-09605-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-024-09605-1