CN107168346A - A kind of asynchronous system brain control UAS based on wearable display - Google Patents
A kind of asynchronous system brain control UAS based on wearable display Download PDFInfo
- Publication number
- CN107168346A CN107168346A CN201710298938.XA CN201710298938A CN107168346A CN 107168346 A CN107168346 A CN 107168346A CN 201710298938 A CN201710298938 A CN 201710298938A CN 107168346 A CN107168346 A CN 107168346A
- Authority
- CN
- China
- Prior art keywords
- wearable display
- visual
- unit
- state
- eeg
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000004556 brain Anatomy 0.000 title claims abstract description 18
- 230000000638 stimulation Effects 0.000 claims abstract description 75
- 230000000007 visual effect Effects 0.000 claims abstract description 75
- 230000000763 evoking effect Effects 0.000 claims abstract description 31
- 238000000034 method Methods 0.000 claims description 31
- 238000012545 processing Methods 0.000 claims description 29
- 230000005540 biological transmission Effects 0.000 claims description 14
- 238000010219 correlation analysis Methods 0.000 claims description 7
- 238000001514 detection method Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 6
- 230000004936 stimulating effect Effects 0.000 claims 1
- 238000011022 operating instruction Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 11
- 238000002474 experimental method Methods 0.000 description 8
- 108010001267 Protein Subunits Proteins 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 241000251730 Chondrichthyes Species 0.000 description 2
- 230000002411 adverse Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 210000003710 cerebral cortex Anatomy 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- VZCCETWTMQHEPK-QNEBEIHSSA-N gamma-linolenic acid Chemical compound CCCCC\C=C/C\C=C/C\C=C/CCCCC(O)=O VZCCETWTMQHEPK-QNEBEIHSSA-N 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 206010033799 Paralysis Diseases 0.000 description 1
- 102000002067 Protein Subunits Human genes 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 210000000869 occipital lobe Anatomy 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 210000004761 scalp Anatomy 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 235000013311 vegetables Nutrition 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/08—Control of attitude, i.e. control of roll, pitch, or yaw
- G05D1/0808—Control of attitude, i.e. control of roll, pitch, or yaw specially adapted for aircraft
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Automation & Control Theory (AREA)
- Theoretical Computer Science (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Neurology (AREA)
- Neurosurgery (AREA)
- Dermatology (AREA)
- Human Computer Interaction (AREA)
- Health & Medical Sciences (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
本发明公开了一种基于可穿戴显示器的异步式脑控无人机系统,使用可穿戴显示器呈现一组以不同频率闪烁的视觉刺激单元,分别对应无人机的一组不同的操作指令,使用者注视不同的视觉刺激单元,由此产生的脑电信号被处理并识别出被使用者注视的刺激单元及其对应的操作指令,识别结果被发送往无人机端执行,无人机的实时景象画面被传送回地面,与视觉刺激单元叠加,显示在可穿戴显示器,给使用者提供第一人称视角反馈可以提高稳态视觉诱发电位脑机接口系统的便携性。本发明还充分利用空闲态解码,可以很大程度地降低使用脑机接口控制飞行的操作负担和疲劳感受,给使用者更自由的飞行体验。
The invention discloses an asynchronous brain-controlled unmanned aerial vehicle system based on a wearable display. The wearable display is used to present a group of visual stimulation units flashing at different frequencies, corresponding to a set of different operating instructions of the unmanned aerial vehicle. The user looks at different visual stimulation units, and the resulting EEG signals are processed to identify the stimulation unit and the corresponding operation instructions that the user is looking at, and the recognition results are sent to the drone for execution. The scene picture is sent back to the ground, superimposed with the visual stimulation unit, and displayed on the wearable display. Providing the user with first-person perspective feedback can improve the portability of the steady-state visual evoked potential brain-computer interface system. The present invention also makes full use of the idle state decoding, which can greatly reduce the operation burden and fatigue feeling of using the brain-computer interface to control the flight, and give the user a more free flying experience.
Description
技术领域technical field
本发明属于生物医学工程及自动控制交叉技术领域,具体地,涉及一种稳态视觉诱发电位呈现方式和基于此的脑机接口异步式控制无人机飞行系统,尤其是一种基于可穿戴显示器的异步式脑控无人机系统。The invention belongs to the technical fields of biomedical engineering and automatic control, in particular, it relates to a steady-state visual evoked potential presentation method and a brain-computer interface asynchronous control drone flight system based on it, especially a wearable display-based Asynchronous brain-controlled UAV system.
背景技术Background technique
脑机接口是一种不依赖外部肌肉即可实现人与外界交互的技术。可以帮助不便行动、瘫痪、中风患者重新建立与外部世界的联系,表达自己的意愿,辅助日常生活,异步式脑机接口令系统跟随患者的意愿表达,而不是患者跟随系统的命令,具有很大的使用价值和发展前景。常用的脑电模式有运动想象、稳态视觉诱发电位、P300等。Brain-computer interface is a technology that can realize the interaction between human and the outside world without relying on external muscles. It can help patients with limited mobility, paralysis, and stroke to re-establish contact with the outside world, express their wishes, and assist in daily life. The asynchronous brain-computer interface allows the system to follow the patient's wishes, rather than the patient following the system's orders, which has great potential use value and development prospects. Commonly used EEG patterns include motor imagery, steady-state visual evoked potentials, and P300.
稳态视觉诱发电位是当人眼接受一定频率的视觉刺激时,大脑枕叶区被诱发产生与刺激频率相同或成倍频关系的响应信号,该信号即可被识别并转换成相应的控制指令用来控制外部设备。基于稳态视觉诱发电位的脑机接口系统由于其稳定性、极少的训练、可提供的控制指令多以及只需简单的数据处理方法即可实现较高的信息传输率等优点被广泛研究,现有的基于稳态视觉诱发电位的脑机接口系统通常采用LED板/LCD/CRT屏幕作为视觉刺激的呈现手段,造成整体系统显得十分臃肿,不便于携带,给使用者的操作感受也不够沉浸,对系统的实用性造成了不良影响。Steady-state visual evoked potential is when the human eye receives a certain frequency of visual stimulation, the occipital lobe of the brain is induced to produce a response signal that is the same as the frequency of the stimulation or multiplied by the frequency, and the signal can be recognized and converted into a corresponding control command Used to control external devices. Brain-computer interface systems based on steady-state visual evoked potentials have been extensively studied due to their advantages of stability, minimal training, many control commands that can be provided, and high information transmission rates that can be achieved with simple data processing methods. Existing brain-computer interface systems based on steady-state visual evoked potentials usually use LED boards/LCD/CRT screens as visual stimulus presentation means, resulting in the overall system being very bloated, not easy to carry, and not immersive enough for users to operate , which adversely affects the practicability of the system.
无人机作为当下的一个研究热点,以及在实用性上巨大的潜力被广泛研究,尤其当无人机搭载一些外部设备例如机械臂、摄像头、各类操纵装置等,它将为行动不便的人群带来很大的帮助,满足行动不便人群的许多远程操作的生活需求,例如出门探路,买菜,运东西。但现有的无人机一般采用遥控器或飞行地面软件的方式,对手脚行动不便的人来说并不易用,而他们往往比健全人有更迫切的需求。As a current research hotspot, UAVs have been extensively studied for their huge potential in practicality, especially when UAVs are equipped with some external equipment such as robotic arms, cameras, various manipulation devices, etc. It brings a lot of help and meets the life needs of many remote operations of people with disabilities, such as going out to find the way, shopping for vegetables, and transporting things. However, existing drones generally use remote controls or flight ground software, which are not easy to use for people with handicapped hands and feet, and they often have more urgent needs than able-bodied people.
东南大学的“基于稳态视觉诱发电位的无人机操控方法”以及中国计量学院的“一种基于稳态视觉诱发电位的飞行器控制装置”分别提出了使用稳态视觉诱发电位控制无人机飞行的方法和装置,但其中视觉刺激单元仍位于使用者正前方一定距离处,影响了使用者的参与感;另外,使用者在控制某一方向飞行过程中,需要持续注视该方块,由于稳态视觉诱发刺激单元本身会给使用者带来疲劳感,长时间注视将给使用者带来很强的闪烁感和疲劳感,降低了系统的易用性和使用者对系统的认可度,久而久之,使用者将越来越不愿意使用系统。The "UAV control method based on steady-state visual evoked potential" of Southeast University and "an aircraft control device based on steady-state visual method and device, but the visual stimulation unit is still located at a certain distance directly in front of the user, which affects the user's sense of participation; in addition, the user needs to keep looking at the block when flying in a certain direction, due to the steady state The visual evoked stimulation unit itself will bring fatigue to the user, and long-term staring will bring a strong sense of flickering and fatigue to the user, which reduces the ease of use of the system and the user's recognition of the system. Over time, Users will become less and less willing to use the system.
发明内容Contents of the invention
针对现有技术中存在的上述不足,本发明旨在提供一种基于可穿戴显示器的异步式稳态视觉诱发电位脑控无人机系统,以有效提高系统的便携性,以及降低使用者的疲劳感和操作负担,以最终提高系统的实用性。In view of the above-mentioned deficiencies in the prior art, the present invention aims to provide an asynchronous steady-state visual evoked potential brain-controlled drone system based on a wearable display, so as to effectively improve the portability of the system and reduce user fatigue sense and operational burden to ultimately improve the usability of the system.
本发明具体采用以下技术方案解决上述技术问题。The present invention specifically adopts the following technical solutions to solve the above-mentioned technical problems.
一种基于可穿戴显示器的异步式脑控无人机系统,包括可穿戴显示器以及搭载于可穿戴显示器上的稳态视觉诱发刺激单元、脑电信号采集单元以及脑电信号处理单元;其中:An asynchronous brain-controlled unmanned aerial vehicle system based on a wearable display, including a wearable display and a steady-state visually evoked stimulation unit mounted on the wearable display, an EEG signal acquisition unit, and an EEG signal processing unit; wherein:
所述稳态视觉诱发刺激单元通过视觉刺激诱发生成脑电信号;The steady-state visual evoking stimulation unit induces and generates EEG signals through visual stimulation;
所述脑电信号采集单元用于采集脑电信号并发送至脑电信号处理单元进行实时处理设别,并将识别结果发送至无人机。The EEG signal acquisition unit is used to collect EEG signals and send them to the EEG signal processing unit for real-time processing and identification, and send the recognition results to the UAV.
优选地,所述稳态视觉诱发刺激单元包括多个具有不同闪烁频率的视觉刺激子单元。Preferably, the steady state visual evoking stimulation unit includes a plurality of visual stimulation subunits with different flicker frequencies.
优选地,多个视觉刺激子单元分别为:向前子单元、向上子单元、向下子单元、左转子单元和右转子单元,相应地,每一个视觉刺激子单元分别对应于向前、向上、向下、左转、右转控制指令。Preferably, the plurality of visual stimulation subunits are: forward subunit, upward subunit, downward subunit, left rotor unit and right rotor unit, correspondingly, each visual stimulation subunit corresponds to forward, Up, down, left turn, right turn control commands.
优选地,向后控制指令通过连续的向前、向上、向下、左转和/或右转控制指令实现。Preferably, the backward control command is realized by successive forward, upward, downward, left turn and/or right turn control commands.
优选地,所述向前子单元、左转子单元、右转子单元、向上子单元和向下子单元的闪烁频率分别为10.7Hz、9.37Hz、8.33Hz、12.5Hz和7.5Hz。Preferably, the flickering frequencies of the forward subunit, the left rotor unit, the right rotor unit, the up subunit and the down subunit are 10.7Hz, 9.37Hz, 8.33Hz, 12.5Hz and 7.5Hz respectively.
优选地,所述脑电信号采集单元包括便携式脑电采集设备和无线传输模块;其中:Preferably, the EEG signal acquisition unit includes a portable EEG acquisition device and a wireless transmission module; wherein:
所述便携式脑电采集设备为能够采集大脑皮层枕区产生的脑电信号的任意通道数的脑电采集设备,用于采集脑电信号;The portable EEG acquisition device is an EEG acquisition device capable of collecting any number of channels of the EEG signals generated in the occipital region of the cerebral cortex, and is used to collect EEG signals;
所述无线传输模块用于将采集到的脑电信号发送至脑电信号处理单元。The wireless transmission module is used to send the collected EEG signal to the EEG signal processing unit.
优选地,所述脑电信号处理单元包括分类处理模块和分类识别模块;其中:Preferably, the EEG signal processing unit includes a classification processing module and a classification recognition module; wherein:
所述分类处理模块用于对接收到的脑电信号进行特征提取及分类;The classification processing module is used for feature extraction and classification of the received EEG signals;
所述分类识别模块用于对分类后的脑电信号进行识别,得到的识别结果发送至无人机。The classification and identification module is used to identify the classified EEG signals, and the obtained identification results are sent to the drone.
优选地,所述分类处理模块采用典型相关分析方法进行特征提取及分类。Preferably, the classification processing module uses a canonical correlation analysis method for feature extraction and classification.
优选地,分类处理模块对脑电信号分类为:空闲态和工作态;其中:Preferably, the classification processing module classifies the EEG signals into: idle state and working state; wherein:
当使用者注视稳态视觉诱发刺激单元,其脑电信号被分类为工作态;When the user gazes at the steady-state visual evoking stimulation unit, the EEG signal is classified as a working state;
当使用者从稳态视觉诱发刺激单元移开目光,其脑电信号将被分类为空闲态。When the user looks away from the steady-state visual evoking stimulation unit, the EEG signal will be classified as an idle state.
优选地,所述分类识别模块采用基于阈值检测的空闲态识别方法对分类后的脑电信号进行识别,当最大相关系数与次大相关系数之比小于等于设定阈值时,判断为空闲态,否则为工作态;工作态下最大相关系数对应的闪烁频率所对应的飞行方向作为识别结果输出至无人机,空闲态下发送给无人机的识别结果仍被翻译为最近一次工作态下的飞行方向。Preferably, the classification and identification module adopts an idle state identification method based on threshold detection to identify the classified EEG signals, and when the ratio of the largest correlation coefficient to the next largest correlation coefficient is less than or equal to the set threshold, it is judged as an idle state, Otherwise, it is in the working state; the flight direction corresponding to the flicker frequency corresponding to the maximum correlation coefficient in the working state is output to the UAV as the recognition result, and the recognition result sent to the UAV in the idle state is still translated into the latest working state. flight direction.
优选地,所述基于可穿戴显示器的异步式脑控无人机系统,还包括实时景象采集及传送单元,所述实时景象采集及传送单元搭载于无人机,用于采集无人机的实时影像画面并传送至可穿戴显示器;所述实时影像画面与稳态视觉诱发刺激单元叠加,显示于可穿戴显示器。Preferably, the wearable display-based asynchronous brain-controlled UAV system also includes a real-time scene acquisition and transmission unit, which is mounted on the UAV to collect real-time images of the UAV. The image frame is transmitted to the wearable display; the real-time image frame is superimposed with the steady-state visual evoking stimulation unit and displayed on the wearable display.
优选地,所述实时景象采集及传送单元包括摄像头和视频采集卡;其中:Preferably, the real-time scene collection and transmission unit includes a camera and a video capture card; wherein:
所述摄像头用于无人机飞行视频画面采集;The camera is used for the acquisition of unmanned aerial vehicle flight video images;
所述视频采集卡用于将采集到的无人机飞行视频画面无线传输至可穿戴显示器。The video capture card is used to wirelessly transmit the captured drone flight video images to the wearable display.
与现有技术相比,本发明具有以下有益效果:Compared with the prior art, the present invention has the following beneficial effects:
1、传统的基于稳态视觉诱发电位的脑机接口系统通常采用LED板/LCD/CRT屏幕作为视觉刺激的呈现手段,造成整体系统显得十分臃肿,不便于携带,给使用者的操作感受也不够沉浸,对系统的实用性造成了不良影响。本发明提出的显示手段,将刺激单元和视觉反馈画面显示在可穿戴的显示器上,一组具有不同频率闪烁的视觉刺激单元外部环绕反馈画面,不遮挡反馈画面视野。使用者佩戴该显示器,即可沉浸式地操纵无人机,了解当前飞行环境。1. Traditional brain-computer interface systems based on steady-state visual evoked potentials usually use LED boards/LCD/CRT screens as visual stimulus presentation methods, resulting in a bloated overall system that is not easy to carry and does not give users enough operating experience Immersion, which adversely affects the usability of the system. The display means proposed by the present invention displays the stimulation unit and the visual feedback screen on a wearable display, and a group of visual stimulation units with different flickering frequencies surround the feedback screen without blocking the visual field of the feedback screen. Wearing the display, the user can control the drone immersively and understand the current flight environment.
2、传统的脑机接口控制无人机系统中,使用者在控制某一方向飞行过程中,需要持续注视该方块,由于稳态视觉诱发刺激单元本身会给使用者带来疲劳感,长时间注视将给使用者带来很强的闪烁感和疲劳感,降低了系统的易用性和使用者对系统的认可度,久而久之,使用者将越来越不愿意使用系统。本发明提出的操控范式中,使用者只需在切换飞行方向时注视视觉刺激单元,无需持续注视视觉刺激单元即可保持无人机飞行状态,降低使用者的操作负担和疲劳感,而且使用者还可持续保持对无人机飞行状态的掌握,不会因为持续注视刺激单元而无暇关注飞行场景反馈,而且在注视刺激单元时也能更加专注,不需要持续分散注意力到飞行场景中,有利于系统整体性能的提升。2. In the traditional brain-computer interface control UAV system, the user needs to keep watching the box while controlling the flight in a certain direction. Since the steady-state visual evoked stimulation unit itself will bring fatigue to the user, it will take a long time Staring will bring a strong sense of flickering and fatigue to the user, reducing the ease of use of the system and the user's recognition of the system. As time passes, the user will become less and less willing to use the system. In the control paradigm proposed by the present invention, the user only needs to watch the visual stimulation unit when switching the flight direction, and can maintain the flying state of the drone without continuously watching the visual stimulation unit, which reduces the user's operating burden and fatigue, and the user It can also maintain a continuous grasp of the flight status of the drone, and will not have time to pay attention to the feedback of the flight scene because of the continuous gazing at the stimulation unit, and can also be more focused when gazing at the stimulation unit, without the need to continue to distract attention in the flight scene. It is beneficial to improve the overall performance of the system.
3、传统的无人机操控方法,需要双手的操作,本发明提出的操控方法,只需要使用者进行空间注意力上的切换,为伤残人士提供了控制无人机的方法,可以帮助他们的生活,改善生活质量。3. The traditional UAV control method requires the operation of both hands. The control method proposed by the present invention only requires the user to switch the spatial attention, and provides a method for the disabled to control the UAV, which can help them life and improve the quality of life.
附图说明Description of drawings
通过阅读参照以下附图对非限制性实施例所作的详细描述,本发明的其它特征、目的和优点将会变得更明显:Other characteristics, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:
图1为本发明优选实施例的系统示意图;Fig. 1 is a system schematic diagram of a preferred embodiment of the present invention;
图2为稳态视觉诱发刺激单元与实时场景反馈画面在可穿戴式显示器的显示示意图;Fig. 2 is the display schematic diagram of the steady-state visual evoked stimulation unit and the real-time scene feedback screen on the wearable display;
图3为便携式脑电采集设备中的电极在头皮上的放置位置图;Fig. 3 is the placement position figure of the electrode in the portable EEG acquisition device on the scalp;
图4为异步控制范式原理流程图。Figure 4 is a flow chart of the principle of the asynchronous control paradigm.
具体实施方式detailed description
下面对本发明的实施例作详细说明:本实施例在以本发明技术方案为前提下进行实施,给出了详细的实施方式和具体的操作过程。应当指出的是,对本领域的普通技术人员来说,在不脱离本发明构思的前提下,还可以做出若干变形和改进,这些都属于本发明的保护范围。The following is a detailed description of the embodiments of the present invention: this embodiment is implemented on the premise of the technical solution of the present invention, and provides detailed implementation methods and specific operation processes. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all belong to the protection scope of the present invention.
实施例Example
本实施例提供的基于可穿戴显示器的异步式脑控无人机系统,采用稳态视觉诱发刺激单元(电位),使用可穿戴显示器呈现一组以不同频率闪烁的视觉刺激子单元,分别对应无人机的一组不同的操作指令,使用者注视不同的视觉刺激子单元,由此产生的脑电信号被处理并识别出被使用者注视的刺激子单元及其对应的操作指令,识别结果被发送往无人机端执行,无人机的实时景象画面被传送回地面,与稳态视觉诱发刺激单元叠加,显示在可穿戴显示器,给使用者提供第一人称视角反馈;还通过设计一种异步式控制飞行的范式用以降低使用脑机接口控制飞行的操作负担和疲劳感受。The asynchronous brain-controlled unmanned aerial vehicle system based on the wearable display provided in this embodiment adopts a steady-state visually evoked stimulation unit (potential), and uses a wearable display to present a group of visual stimulation subunits flashing at different frequencies, respectively corresponding to no A set of different operation instructions of the man-machine, the user looks at different visual stimulation sub-units, the resulting EEG signals are processed and the stimulation sub-units and their corresponding operation instructions that the user is looking at are identified, and the recognition results are Send it to the drone for execution, and the real-time scene of the drone is sent back to the ground, superimposed with the steady-state visual evoked stimulation unit, displayed on the wearable display, and provides the user with first-person perspective feedback; also by designing an asynchronous The paradigm of controlled flight is used to reduce the operational burden and fatigue of using brain-computer interface to control flight.
进一步地,所述稳态视觉诱发刺激单元和视觉反馈画面显示在可穿戴的显示器上,所述的一组具有不同频率闪烁的视觉刺激子单元外部环绕反馈画面以不影响使用者对实时飞行场景和状态的掌握,使用者佩戴该显示器,即可沉浸式地操纵无人机,了解当前飞行环境。Further, the steady-state visual-evoking stimulation unit and the visual feedback picture are displayed on a wearable display, and the feedback picture is surrounded by a group of visual stimulation subunits with different flickering frequencies so as not to affect the user's understanding of the real-time flight scene. Wearing the display, the user can control the drone immersively and understand the current flight environment.
进一步地,为保证较高的识别率和稳定性,以及保证使用者能够对视觉反馈画面有完整的观看体验,本实施例中,视觉刺激子单元有五种:向前子单元、向上子单元、向下子单元、左转子单元、右转子单元,对应于五种常用控制指令:向前控制指令、向上控制指令、向下控制指令、左转控制指令、右转控制指令,能够实现对无人机三维空间飞行的控制。如向后,可以通过连续的五种控制指令实线,例如左转两次实现。Further, in order to ensure a high recognition rate and stability, and to ensure that users can have a complete viewing experience on the visual feedback screen, in this embodiment, there are five types of visual stimulation subunits: forward subunit, upward subunit , down subunit, left rotor unit, and right rotor unit, corresponding to five commonly used control commands: forward control command, upward control command, downward control command, left turn control command, right turn control command, which can realize Control of UAV flight in 3D space. As backward, it can be realized through continuous five control command lines, such as turning left twice.
进一步地,为提高无人机系统的便携性,脑电信号采集单元采用便携式的采集设备(便携式脑电采集设备)。Further, in order to improve the portability of the UAV system, the EEG signal acquisition unit adopts a portable acquisition device (portable EEG acquisition device).
优选地,脑电信号处理单元在进行分类时,采用典型相关分析方法进行特征提取及分类,对空闲态与工作态进行识别时,采用非常实用的基于阈值检测的空闲态识别方法,当最大相关系数与次大相关系数之比小于等于某阈值时,判为空闲态,否则为工作态,工作态下最大相关系数对应的刺激频率所对应的飞行方向为分类结果。Preferably, when classifying, the EEG signal processing unit adopts a typical correlation analysis method for feature extraction and classification, and when identifying the idle state and the working state, adopts a very practical idle state identification method based on threshold detection. When the maximum correlation When the ratio of the coefficient to the next largest correlation coefficient is less than or equal to a certain threshold, it is judged as an idle state, otherwise it is a working state, and the flight direction corresponding to the stimulation frequency corresponding to the largest correlation coefficient in the working state is the classification result.
进一步地,为降低使用者的操作负担和疲劳感受,所述无人机系统中,使用者只需在切换飞行方向时注视视觉刺激单元,无需持续注视视觉刺激单元即可保持无人机飞行状态。(1)使用者注视某刺激单元并被识别后,飞机朝此方向飞行;(2)使用者即可从刺激单元移开目光,转向视觉反馈画面,把握飞行状态,该过程中无人机将持续保持向此方向飞行;(3)之后,使用者可根据意愿结合飞行状态注视其他刺激单元,被识别后即可移开目光,重复上述(1)(2)过程。同时,使用者不被局限于此过程,使用者想实现方向之间快速切换时,仍可省略(2)过程。所述异步式控制范式实现方法是,使用者注视刺激单元,其脑电信号被分类为工作状态下该单元对应的飞行方向;使用者从某个视觉刺激单元移开目光转向视觉反馈画面后,其脑电信号将被分类为空闲状态,但发送给飞机端的命令仍被翻译为最近一次工作状态下的飞行方向。等到下次脑电信号被解算为其他飞行方向,发送给飞机的指令变为其他飞行方向。Further, in order to reduce the operating burden and fatigue experience of the user, in the UAV system, the user only needs to watch the visual stimulation unit when switching the flight direction, and can maintain the flying state of the UAV without continuously watching the visual stimulation unit. . (1) After the user looks at a certain stimulation unit and is recognized, the aircraft will fly in this direction; (2) The user can look away from the stimulation unit and turn to the visual feedback screen to grasp the flight status. Continue to fly in this direction; (3) Afterwards, the user can look at other stimulation units according to their wishes in combination with the flying state. After being recognized, the user can look away and repeat the above (1) (2) process. At the same time, the user is not limited to this process. When the user wants to quickly switch between directions, the process (2) can still be omitted. The implementation method of the asynchronous control paradigm is that the user looks at the stimulation unit, and its EEG signal is classified into the flight direction corresponding to the unit in the working state; Its EEG signals will be classified as idle, but the commands sent to the aircraft will still be translated into the flight direction in the last working state. Wait until the next time the EEG signal is resolved into other flight directions, and the instructions sent to the aircraft will change to other flight directions.
进一步地,无人机飞行视频画面由搭载在无人机上的摄像头采集,经由视频采集卡无线传输回可穿戴显示器。Furthermore, the drone flight video images are collected by the camera mounted on the drone, and wirelessly transmitted back to the wearable display via the video capture card.
以下将参照附图更充分地描述本发明,在附图中示出了各种实施情况。The present invention will be described more fully hereinafter with reference to the accompanying drawings, in which various embodiments are shown.
针对现有稳态视觉诱发电位控制无人机所存在的各种问题,如图1所示,本发明的解决思路是使用可穿戴式的显示器以及设计新的控制范式控制无人机飞行,具体为:使用可穿戴显示器呈现一组以不同频率闪烁的视觉刺激子单元,分别对应无人机的一组不同的操作指令,使用者注视不同的视觉刺激单元,由此产生的脑电信号被处理并识别出被使用者注视的刺激单元及其对应的操作指令,识别结果被发送往无人机端执行,无人机的实时景象画面被传送回地面,与视觉刺激子单元叠加,显示在可穿戴显示器,给使用者提供第一人称视角反馈;同时还设计了一种异步式控制飞行的范式,充分利用空闲态解码,用以降低使用脑机接口控制飞行的操作负担和疲劳感。Aiming at the various problems existing in the control of unmanned aerial vehicles with steady-state visual evoked potentials, as shown in Figure 1, the solution of the present invention is to use wearable displays and design new control paradigms to control the flight of unmanned aerial vehicles. It is: using a wearable display to present a group of visual stimulation sub-units that flash at different frequencies, corresponding to a set of different operating instructions of the drone, the user looks at different visual stimulation units, and the resulting EEG signals are processed And identify the stimulation unit that is watched by the user and its corresponding operation command, the recognition result is sent to the drone for execution, the real-time scene of the drone is sent back to the ground, superimposed with the visual stimulation sub-unit, and displayed on the The wearable display provides the user with first-person perspective feedback; at the same time, an asynchronous control flight paradigm is designed to make full use of idle state decoding to reduce the operational burden and fatigue of using the brain-computer interface to control flight.
本发明的实施例及其具体过程如下:Embodiments of the present invention and concrete processes thereof are as follows:
如图2所示,本实施例采用的可穿戴式显示器可以为Fat Shark FSV1074 HD2,它的眼镜镜片是两个800×600的LCD屏幕,允许在眼镜中开发视觉画面,也可以接收外部设备传来的图像,支持与计算机HDMI线连接投影。本实施例使用方式可以是利用OPEN GL库绘制视觉刺激单元,并将无人机传回来的画面一起绘制在可穿戴显示器中。As shown in Figure 2, the wearable display used in this embodiment can be Fat Shark FSV1074 HD2, and its spectacle lenses are two 800×600 LCD screens, which allow the development of visual images in the glasses, and can also receive data from external devices. To the image, support to connect with the computer HDMI cable to project. The use method of this embodiment can be to use the OPEN GL library to draw the visual stimulation unit, and draw the pictures sent back by the drone together on the wearable display.
如图2所示,本实施例中,五个视觉刺激单元从外周环绕实时景象交互画面,五个视觉刺激频率(闪烁频率)可以分别是10.7Hz,9.37Hz,8.33Hz,12.5Hz,7.5Hz,对应向前、左转、右转、向上、向下五个飞行方向,操作人员可以根据自己的意图选择注视对应的刺激单元。As shown in Figure 2, in this embodiment, five visual stimulation units surround the real-time scene interaction screen from the periphery, and the five visual stimulation frequencies (flicker frequencies) can be 10.7Hz, 9.37Hz, 8.33Hz, 12.5Hz, 7.5Hz respectively , corresponding to the five flight directions of forward, left, right, up, and down, and the operator can choose to look at the corresponding stimulation unit according to his own intention.
本发明使用能够采集大脑皮层枕区产生的信号的任意通道数的脑电采集设备。本实施例可以采用便携式采集设备Emotiv Epoc,它是一款14通道+2参考通道的无线EEG采集器,采用无线蓝牙技术而且续航时长达12h,可充电,盐水或清水润湿的传感器,无需黏性的导电膏。如图3所示,实施例中电极主要选取P3,P4,01,02位置。The present invention uses an EEG collection device capable of collecting signals generated in the occipital region of the cerebral cortex with any number of channels. This embodiment can use the portable acquisition device Emotiv Epoc, which is a wireless EEG acquisition device with 14 channels + 2 reference channels. It adopts wireless Bluetooth technology and has a battery life of up to 12 hours. conductive paste. As shown in Fig. 3, the electrodes in the embodiment mainly select the positions of P3, P4, 01, and 02.
本实施例中,数据传输上,控制指令到飞机端可以采用串口+XBee无线数传方式通信,实时景象画面回到可穿戴显示器采用无线图传模块+UVC视频采集卡方式通信。In this embodiment, in terms of data transmission, the control command can be communicated to the aircraft by serial port + XBee wireless data transmission, and the real-time scene picture can be returned to the wearable display by wireless image transmission module + UVC video acquisition card.
本实施例中,脑电信号处理模块可以是一个用Visual C++编写的Win32应用程序,具有数据接收,保存,在线处理,指令发送功能。在处理脑电信号时,先进行5-64Hz带通滤波,然后以1s窗长,0.125s滑动窗长的解算速度对脑电信号进行典型相关分析,从而完成特征提取,其中在典型相关分析算法中,参考信号是由刺激频率所对应的基波和谐波组成,每一个刺激频率与脑电信号通过相关分析都会输出最大相关系数,本实施例共有五个刺激频率(闪烁频率),故产生五个最大相关系数。如图4所示,首先有空闲态与工作态的一次判别,在五个最大相关系数中,最大的一个和次大的一个的比值如果小于某阈值,则视为空闲态,空闲态下,输出指令被映射为最近一次工作状态的指令,则可以实现保持原来方向继续飞行。如判别为工作态,则输出五个最大相关系数中最大的一个对应的飞行指令。In this embodiment, the EEG signal processing module can be a Win32 application program written in Visual C++, which has the functions of data receiving, saving, online processing, and instruction sending. When processing EEG signals, first perform 5-64Hz band-pass filtering, and then perform canonical correlation analysis on the EEG signals at a calculation speed of 1s window length and 0.125s sliding window length, so as to complete feature extraction, among which canonical correlation analysis In the algorithm, the reference signal is composed of the fundamental wave and harmonics corresponding to the stimulation frequency, and each stimulation frequency and the EEG signal will output the maximum correlation coefficient through correlation analysis. There are five stimulation frequencies (flicker frequencies) in this embodiment, so Produces the five largest correlation coefficients. As shown in Figure 4, first there is a discrimination between the idle state and the working state. Among the five largest correlation coefficients, if the ratio of the largest one to the second largest one is less than a certain threshold, it is regarded as the idle state. In the idle state, The output command is mapped to the command of the latest working state, so that the flight can be continued in the original direction. If it is judged to be in the working state, then output the flight command corresponding to the largest one of the five largest correlation coefficients.
本实施例中,为保证无人机飞行的稳定性,当脑电信号处理模块检测到工作态指令切换时,无人机将在指令刚刚切换的1s内以较慢的速度沿着该方向飞行,而忽视这1s内传过来的其他指令,1s后,无论使用者以空闲态操控无人机继续朝此方向更快飞行或注视其他刺激切换飞行方向,这1s的缓冲时间有助于无人机在类别之间切换时保持稳定。In this embodiment, in order to ensure the flight stability of the UAV, when the EEG signal processing module detects that the working state instruction is switched, the UAV will fly along this direction at a slower speed within 1 second after the instruction has just been switched. , while ignoring other instructions transmitted within this 1s. After 1s, whether the user controls the UAV to continue to fly faster in this direction in an idle state or watches other stimuli to switch the flight direction, this 1s buffer time will help the drone The machine remains stable when switching between categories.
本实施例中,可以采用ARDrone无人机,飞机端接收到控制指令,根据控制指令自动调整姿态实现对应方向的飞行。同时,机上搭载的摄像头通过无线图传模块将实时画面传送回可穿戴显示器,可穿戴显示屏收到画面并绘制。In this embodiment, the ARDrone unmanned aerial vehicle can be used, and the aircraft end receives the control command, and automatically adjusts the attitude according to the control command to realize the flight in the corresponding direction. At the same time, the camera equipped on the aircraft transmits the real-time images back to the wearable display through the wireless image transmission module, and the wearable display receives the images and draws them.
本实施例中,利用该系统对无人机进行操控的具体实施过程如下:In this embodiment, the specific implementation process of using the system to control the drone is as follows:
(1)选取五名身体健康的受试人员,标号S1到S5。(1) Select five healthy subjects, labeled S1 to S5.
(2)预实验:每个受试者共接受1次预实验,每次实验中,受试者先佩戴脑电采集设备Emotiv Epoc和可穿戴显示器Fat shark FSV1074 HD2,然后根据提示依次注视不同的视觉刺激方块5s,每个方块被随机提示,另外,还会被提示空闲态,此时受试者应注视画面中央,此实验中反馈画面用一张静态图像代替,每次实验共240s。(2) Pre-experiment: Each subject received a pre-experiment in total. In each experiment, the subject first wore the EEG acquisition device Emotiv Epoc and the wearable display Fat shark FSV1074 HD2, and then stared at different images in turn according to the prompts. The visual stimulation block is 5s, and each block is randomly prompted. In addition, it will be prompted to be idle. At this time, the subject should focus on the center of the screen. In this experiment, the feedback screen is replaced by a static image, and each experiment lasts 240s.
(3)预实验时,脑电信号处理模块即Win32应用程序将本次实验数据记录保存下来,实验结束后,该模块将保存的数据进行滤波和离线分类,并自动搜寻对每位受试者最合适的空闲态检测阈值。(3) During the pre-experiment, the EEG signal processing module, that is, the Win32 application program, records and saves the experimental data. After the experiment, the module filters and classifies the saved data offline, and automatically searches for each subject. The most appropriate idle state detection threshold.
(4)无人机操控实验:受试者根据自己的意愿操控无人机飞行,此时脑电信号处理模块根据(3)中得到的最佳阈值在线处理分类受试者脑电信号,并输出控制指令到飞机端,飞机端传回实时反馈画面给受试者。受试者可先注视向前方块,则飞机开始向前缓慢飞行1s,受试者若想:a.继续保持向前飞行,则可不注视任何刺激,观察反馈画面,无人机将以较快速度保持前进;b.转换飞行方向,如右转,则注视右转方块,飞机将以较慢速度右转1s;c.以此类推。(4) UAV control experiment: The subject controls the UAV to fly according to his own wishes. At this time, the EEG signal processing module processes and classifies the EEG signal of the subject online according to the optimal threshold obtained in (3), and Output control commands to the aircraft end, and the aircraft end sends back real-time feedback screens to the subjects. The subject can first stare at the forward square, and then the aircraft starts to fly forward slowly for 1 second. If the subject wants to: a. Continue to fly forward, he can not watch any stimulus and observe the feedback screen, and the drone will fly at a faster speed. Keep the speed forward; b. Change the flight direction, if you turn right, look at the right turn square, and the aircraft will turn right at a slower speed for 1 second; c. And so on.
在本实施例中:In this example:
本实施例提供的是一种基于可穿戴显示器的异步式稳态视觉诱发电位控制无人机系统,包括:一组稳态视觉诱发刺激单元与第一人称视角反馈画面呈现在可穿戴的显示器上,使用者佩戴该显示器并选择注视某一飞行方向对应的视觉刺激单元即可诱发出相应的脑电信号,该信号被信号处理模块处理并识别,识别结果被传送至无人机,无人机即可按照使用者的意愿朝向对应的飞行方向飞行,无人机的实时景象画面被传送回地面,与视觉刺激单元叠加,显示在可穿戴显示器。还有一种充分利用空闲态解码的异步式控制飞行的范式。This embodiment provides an asynchronous steady-state visual evoked potential control drone system based on a wearable display, including: a set of steady-state visual evoked stimulation units and a first-person perspective feedback screen are presented on the wearable display, The user wears the display and chooses to look at the visual stimulation unit corresponding to a certain flight direction to induce the corresponding EEG signal, which is processed and recognized by the signal processing module, and the recognition result is sent to the drone, and the drone immediately It can fly in the corresponding flight direction according to the user's wishes, and the real-time scene of the drone is transmitted back to the ground, superimposed with the visual stimulation unit, and displayed on the wearable display. There is also an asynchronous control flight paradigm that takes advantage of idle state decoding.
稳态视觉诱发刺激单元和视觉反馈画面显示在可穿戴的显示器上,使用者佩戴该显示器,即可沉浸式地操纵无人机,了解当前飞行环境。The steady-state visual evoking stimulation unit and visual feedback images are displayed on the wearable display, and the user can control the drone immersively and understand the current flight environment by wearing the display.
所述的一组具有不同频率闪烁的视觉刺激子单元外部环绕反馈画面,不同的视觉刺激子单元:向前子单元、向上子单元、向下子单元、左转子单元、右转子单元的箭头分别对应无人机向前、向上、向下、左转、右转飞行方向。The group of visual stimulation subunits with different flickering frequencies are surrounded by external feedback images, different visual stimulation subunits: arrows of forward subunit, upward subunit, downward subunit, left rotor unit and right rotor unit Corresponding to the flight directions of the UAV forward, upward, downward, left turn, and right turn respectively.
所述脑电信号被便携式采集电极(便携式采集设备)获取并无线传输给信号处理模块处理,所述脑电信号处理过程主要包括:(1)带通滤波;(2)基于典型相关分析的分类方法;以及(3)基于阈值检测的空闲态识别方法,当最大相关系数与次大相关系数之比小于等于某阈值时,判为空闲态,否则为工作态,工作态下最大相关系数对应的刺激频率所对应的飞行方向为分类结果,该结果被无线传送至无人机。The EEG signal is acquired by a portable acquisition electrode (portable acquisition device) and wirelessly transmitted to the signal processing module for processing. The EEG signal processing process mainly includes: (1) band-pass filtering; (2) classification based on canonical correlation analysis method; and (3) an idle state identification method based on threshold detection, when the ratio of the maximum correlation coefficient to the next largest correlation coefficient is less than or equal to a certain threshold, it is judged as an idle state, otherwise it is a working state, and the maximum correlation coefficient corresponding to the working state The flight direction corresponding to the stimulus frequency is the classification result, which is wirelessly transmitted to the drone.
所述无人机系统中,使用者只需在切换飞行方向时注视视觉刺激单元,无需持续注视视觉刺激单元即可保持无人机飞行状态。(1)使用者注视某刺激单元并被识别后,飞机朝此方向飞行;(2)使用者即可从刺激单元移开目光,转向视觉反馈画面,把握飞行状态,该过程中无人机将持续保持向此方向飞行;(3)之后,使用者可根据意愿结合飞行状态注视其他刺激单元,被识别后即可移开目光,重复上述(1)(2)过程。同时,使用者不被局限于此过程,使用者想实现方向之间快速切换时,仍可省略(2)过程。In the unmanned aerial vehicle system, the user only needs to watch the visual stimulation unit when switching the flight direction, and can maintain the flying state of the drone without continuously watching the visual stimulation unit. (1) After the user looks at a certain stimulation unit and is recognized, the aircraft will fly in this direction; (2) The user can look away from the stimulation unit and turn to the visual feedback screen to grasp the flight status. Continue to fly in this direction; (3) Afterwards, the user can look at other stimulation units according to their wishes in combination with the flying state. After being recognized, the user can look away and repeat the above (1) (2) process. At the same time, the user is not limited to this process. When the user wants to quickly switch between directions, the process (2) can still be omitted.
所述异步式控制范式实现方法是如上所述空闲态检测方法,使用者注视刺激单元,其脑电信号被分类为工作态下该单元对应的飞行方向;使用者从某个视觉刺激单元移开目光转向视觉反馈画面后,其脑电信号将被分类为空闲态,但发送给飞机端的命令仍被翻译为最近一次工作态下的飞行方向。等到下次脑电信号被解算为其他飞行方向,发送给飞机的指令变为其他飞行方向。The implementation method of the asynchronous control paradigm is the above-mentioned idle state detection method, the user looks at the stimulation unit, and its EEG signal is classified as the flight direction corresponding to the unit in the working state; the user moves away from a certain visual stimulation unit After the eyes turn to the visual feedback screen, the EEG signal will be classified as an idle state, but the commands sent to the aircraft end will still be translated into the flight direction in the last working state. Wait until the next time the EEG signal is resolved into other flight directions, and the instructions sent to the aircraft will change to other flight directions.
所述无人机飞行视频画面由搭载在无人机上的摄像头采集,经由视频采集卡无线传输回可穿戴显示器。The drone flight video picture is collected by a camera mounted on the drone, and wirelessly transmitted back to the wearable display via a video capture card.
本实施例给使用者提供第一人称视角反馈可以提高稳态视觉诱发电位脑机接口系统的便携性,还充分利用空闲态解码,可以很大程度地降低使用脑机接口控制飞行的操作负担和疲劳感受,给使用者更自由的飞行体验。This embodiment provides users with first-person perspective feedback, which can improve the portability of the steady-state visual evoked potential brain-computer interface system. It also makes full use of idle state decoding, which can greatly reduce the operational burden and fatigue of using the brain-computer interface to control flight. Feel and give users a more free flying experience.
以上对本发明的具体实施例进行了描述。需要理解的是,本发明并不局限于上述特定实施方式,本领域技术人员可以在权利要求的范围内做出各种变形或修改,这并不影响本发明的实质内容。Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention.
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710298938.XA CN107168346A (en) | 2017-04-28 | 2017-04-28 | A kind of asynchronous system brain control UAS based on wearable display |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710298938.XA CN107168346A (en) | 2017-04-28 | 2017-04-28 | A kind of asynchronous system brain control UAS based on wearable display |
Publications (1)
Publication Number | Publication Date |
---|---|
CN107168346A true CN107168346A (en) | 2017-09-15 |
Family
ID=59812424
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710298938.XA Pending CN107168346A (en) | 2017-04-28 | 2017-04-28 | A kind of asynchronous system brain control UAS based on wearable display |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN107168346A (en) |
Cited By (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108196566A (en) * | 2018-03-16 | 2018-06-22 | 西安科技大学 | A kind of small drone cloud brain control system and its method |
CN108415565A (en) * | 2018-02-25 | 2018-08-17 | 西北工业大学 | The machine integrated intelligent control method of unmanned plane brain and technology |
CN108897418A (en) * | 2018-05-15 | 2018-11-27 | 天津大学 | A kind of wearable brain-machine interface arrangement, man-machine interactive system and method |
CN109009887A (en) * | 2018-07-17 | 2018-12-18 | 东北大学 | A kind of man-machine interactive navigation system and method based on brain-computer interface |
CN109521784A (en) * | 2018-12-13 | 2019-03-26 | 华南农业大学 | A kind of wearable upper limb ectoskeleton unmanned aerial vehicle control system of tactilely-perceptible formula and method |
CN110502101A (en) * | 2019-05-29 | 2019-11-26 | 中国人民解放军军事科学院军事医学研究院 | Virtual reality interaction method and device based on EEG signal acquisition |
CN110597201A (en) * | 2019-09-05 | 2019-12-20 | 佛山市云米电器科技有限公司 | Intelligent household system controlled by AR equipment and control method thereof |
CN110658810A (en) * | 2019-08-27 | 2020-01-07 | 北京航空航天大学 | An unmanned weapon control system for individual combat based on SSVEP brain-computer interface |
CN111487988A (en) * | 2020-03-03 | 2020-08-04 | 天津大学 | Brain-controlled unmanned aerial vehicle method based on steady-state visual evoked potential brain-computer interface |
CN112034979A (en) * | 2020-07-31 | 2020-12-04 | 西安交通大学 | A wearable flight sensory feedback system based on force feedback |
CN112114662A (en) * | 2020-08-03 | 2020-12-22 | 西安交通大学 | Reality-augmented self-adaptive dynamic multi-scene evoked brain control method |
CN113009931A (en) * | 2021-03-08 | 2021-06-22 | 北京邮电大学 | Man-machine and unmanned-machine mixed formation cooperative control device and method |
WO2023173323A1 (en) * | 2022-03-16 | 2023-09-21 | 深圳市大疆创新科技有限公司 | Power consumption control method, apparatus and system for unmanned aerial vehicle, and storage medium |
CN117908674A (en) * | 2024-01-11 | 2024-04-19 | 北京理工大学 | Brain-computer interface and brain-computer control method for brain-controlled air-ground collaborative unmanned system |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103955269A (en) * | 2014-04-09 | 2014-07-30 | 天津大学 | Intelligent glass brain-computer interface method based on virtual real environment |
CN105204517A (en) * | 2015-09-24 | 2015-12-30 | 杨珊珊 | Personal service method and system for small and mini-type unmanned aerial vehicles |
CN106371451A (en) * | 2016-11-07 | 2017-02-01 | 东南大学 | Unmanned aerial vehicle manipulation method and device based on steady state visual evoked potential |
WO2017020763A1 (en) * | 2015-07-31 | 2017-02-09 | 广州极飞科技有限公司 | Unmanned aerial vehicle |
-
2017
- 2017-04-28 CN CN201710298938.XA patent/CN107168346A/en active Pending
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103955269A (en) * | 2014-04-09 | 2014-07-30 | 天津大学 | Intelligent glass brain-computer interface method based on virtual real environment |
WO2017020763A1 (en) * | 2015-07-31 | 2017-02-09 | 广州极飞科技有限公司 | Unmanned aerial vehicle |
CN105204517A (en) * | 2015-09-24 | 2015-12-30 | 杨珊珊 | Personal service method and system for small and mini-type unmanned aerial vehicles |
CN106371451A (en) * | 2016-11-07 | 2017-02-01 | 东南大学 | Unmanned aerial vehicle manipulation method and device based on steady state visual evoked potential |
Non-Patent Citations (1)
Title |
---|
王行愚 等: "脑控:基于脑-机接口的人机融合控制", 《自动化学报》 * |
Cited By (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108415565A (en) * | 2018-02-25 | 2018-08-17 | 西北工业大学 | The machine integrated intelligent control method of unmanned plane brain and technology |
CN108196566A (en) * | 2018-03-16 | 2018-06-22 | 西安科技大学 | A kind of small drone cloud brain control system and its method |
CN108897418A (en) * | 2018-05-15 | 2018-11-27 | 天津大学 | A kind of wearable brain-machine interface arrangement, man-machine interactive system and method |
CN109009887A (en) * | 2018-07-17 | 2018-12-18 | 东北大学 | A kind of man-machine interactive navigation system and method based on brain-computer interface |
CN109521784B (en) * | 2018-12-13 | 2021-05-11 | 华南农业大学 | Touch sensing type wearable upper limb exoskeleton unmanned aerial vehicle control system and method |
CN109521784A (en) * | 2018-12-13 | 2019-03-26 | 华南农业大学 | A kind of wearable upper limb ectoskeleton unmanned aerial vehicle control system of tactilely-perceptible formula and method |
CN110502101A (en) * | 2019-05-29 | 2019-11-26 | 中国人民解放军军事科学院军事医学研究院 | Virtual reality interaction method and device based on EEG signal acquisition |
CN110658810A (en) * | 2019-08-27 | 2020-01-07 | 北京航空航天大学 | An unmanned weapon control system for individual combat based on SSVEP brain-computer interface |
CN110597201A (en) * | 2019-09-05 | 2019-12-20 | 佛山市云米电器科技有限公司 | Intelligent household system controlled by AR equipment and control method thereof |
CN111487988A (en) * | 2020-03-03 | 2020-08-04 | 天津大学 | Brain-controlled unmanned aerial vehicle method based on steady-state visual evoked potential brain-computer interface |
CN111487988B (en) * | 2020-03-03 | 2022-04-15 | 天津大学 | Brain-controlled unmanned aerial vehicle method based on steady-state visual evoked potential brain-computer interface |
CN112034979A (en) * | 2020-07-31 | 2020-12-04 | 西安交通大学 | A wearable flight sensory feedback system based on force feedback |
CN112114662A (en) * | 2020-08-03 | 2020-12-22 | 西安交通大学 | Reality-augmented self-adaptive dynamic multi-scene evoked brain control method |
CN113009931A (en) * | 2021-03-08 | 2021-06-22 | 北京邮电大学 | Man-machine and unmanned-machine mixed formation cooperative control device and method |
CN113009931B (en) * | 2021-03-08 | 2022-11-08 | 北京邮电大学 | A collaborative control device and method for a mixed formation of manned aircraft and unmanned aerial vehicles |
WO2023173323A1 (en) * | 2022-03-16 | 2023-09-21 | 深圳市大疆创新科技有限公司 | Power consumption control method, apparatus and system for unmanned aerial vehicle, and storage medium |
CN117908674A (en) * | 2024-01-11 | 2024-04-19 | 北京理工大学 | Brain-computer interface and brain-computer control method for brain-controlled air-ground collaborative unmanned system |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN107168346A (en) | A kind of asynchronous system brain control UAS based on wearable display | |
CN108762303A (en) | A kind of portable brain control UAV system and control method based on Mental imagery | |
CN103885445B (en) | The brain control method of a kind of brain control animal robot system and animal robot | |
CN106292705B (en) | Multi-rotor UAV teleoperating system and operation method based on Bluetooth EEG headset | |
CN104799984B (en) | Assistive system for the disabled based on brain-controlled moving eyes and its control method | |
CN106371451A (en) | Unmanned aerial vehicle manipulation method and device based on steady state visual evoked potential | |
CN111487988B (en) | Brain-controlled unmanned aerial vehicle method based on steady-state visual evoked potential brain-computer interface | |
CN104391569A (en) | Brain-machine interface system based on cognition and emotional state multi-mode perception | |
CN112114662A (en) | Reality-augmented self-adaptive dynamic multi-scene evoked brain control method | |
CN110727353A (en) | Method and device for controlling control components defined based on two-dimensional intent | |
CN111728608A (en) | EEG signal analysis method, device, medium and equipment based on augmented reality | |
CN108897418A (en) | A kind of wearable brain-machine interface arrangement, man-machine interactive system and method | |
CN107510562A (en) | Nursing bed system based on brain-computer interface | |
CN103294192A (en) | LED lamp switch control device and control method thereof based on motor imagery | |
CN109009887A (en) | A kind of man-machine interactive navigation system and method based on brain-computer interface | |
Liu et al. | An SSVEP-BCI in augmented reality | |
CN111399652A (en) | Multi-robot Hybrid System Based on Hierarchical SSVEP and Visual Aid | |
CN110716578A (en) | Aircraft control system based on hybrid brain-computer interface and control method thereof | |
CN116382481A (en) | A kind of human-computer interaction glasses based on brain-computer interface technology | |
CN110658810A (en) | An unmanned weapon control system for individual combat based on SSVEP brain-computer interface | |
CN204759349U (en) | Aircraft controlling means based on stable state vision evoked potential | |
CN110688013A (en) | A kind of English keyboard spelling system and spelling method based on SSVEP | |
CN113101021B (en) | Mechanical arm control method based on MI-SSVEP hybrid brain-computer interface | |
CN108319367B (en) | Brain-computer interface method based on motion initiation evoked potential | |
Simoens et al. | Vision: smart home control with head-mounted sensors for vision and brain activity |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170915 |
|
RJ01 | Rejection of invention patent application after publication |