CN108415560B - Electronic device, operation control method and related product - Google Patents
Electronic device, operation control method and related product Download PDFInfo
- Publication number
- CN108415560B CN108415560B CN201810140828.5A CN201810140828A CN108415560B CN 108415560 B CN108415560 B CN 108415560B CN 201810140828 A CN201810140828 A CN 201810140828A CN 108415560 B CN108415560 B CN 108415560B
- Authority
- CN
- China
- Prior art keywords
- operation instruction
- electromyogram
- brain wave
- electroencephalogram
- electronic device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 210000004556 brain Anatomy 0.000 claims abstract description 160
- 230000015654 memory Effects 0.000 claims description 29
- 238000012545 processing Methods 0.000 claims description 19
- 238000013507 mapping Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 14
- 238000010606 normalization Methods 0.000 claims description 12
- 238000004891 communication Methods 0.000 claims description 8
- 230000009471 action Effects 0.000 abstract description 4
- 230000000875 corresponding effect Effects 0.000 description 37
- 230000003183 myoelectrical effect Effects 0.000 description 23
- 238000010586 diagram Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 10
- 210000001508 eye Anatomy 0.000 description 9
- 230000008569 process Effects 0.000 description 8
- 230000002349 favourable effect Effects 0.000 description 7
- 210000005252 bulbus oculi Anatomy 0.000 description 5
- 230000001276 controlling effect Effects 0.000 description 5
- 210000003205 muscle Anatomy 0.000 description 5
- 230000003993 interaction Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000002650 habitual effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 210000003710 cerebral cortex Anatomy 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000004070 electrodeposition Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 241000203475 Neopanax arboreus Species 0.000 description 1
- 230000036982 action potential Effects 0.000 description 1
- 238000010009 beating Methods 0.000 description 1
- 230000007177 brain activity Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000000537 electroencephalography Methods 0.000 description 1
- 238000002567 electromyography Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 210000001087 myotubule Anatomy 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 230000008035 nerve activity Effects 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 210000004761 scalp Anatomy 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Dermatology (AREA)
- Neurosurgery (AREA)
- Neurology (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- User Interface Of Digital Computer (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The embodiment of the application discloses an electronic device, an operation control method and a related product, comprising the following steps: collecting brain wave signals of a user through the brain wave sensor; acquiring user information of a preset type, wherein the user information of the preset type comprises gesture information and electromyographic signals; and determining a target operation instruction according to the brain wave signal and the preset type of user information, and controlling the electronic device through the target operation instruction. According to the embodiment of the application, the control of the electronic device through the combined action of the brain waves and the user information is realized, and the convenience, accuracy and intelligence of the operation control of the electronic device are improved.
Description
Technical Field
The present application relates to the field of mobile terminal technologies, and in particular, to an electronic device, an operation control method, and a related product.
Background
With the widespread application of mobile terminals (such as smart phones), the applications that the mobile terminals can support are increasing, the functions are becoming more and more powerful, and smart phones are developing towards diversification and personalization, becoming indispensable electronic appliances in user life.
At present, a user generally performs multiple instruction interactions with a mobile terminal through finger touch, voice input, gesture control and other forms, and the mobile terminal can perform execution control on multiple functional events required by the user according to the instruction interactions.
Disclosure of Invention
The embodiment of the application provides an electronic device, an operation control method and a related product, so that the electronic device is controlled under the combined action of brain waves and user information, and convenience, accuracy and intelligence of operation control of the electronic device are improved.
In a first aspect, embodiments of the present application provide an electronic device, including a processor, a brain wave sensor connected to the processor, and a plurality of sensors other than the brain wave sensor, wherein,
the brain wave sensor is used for collecting brain wave signals of a user;
the sensors are used for acquiring preset types of user information, and the preset types of user information comprise gesture information and electromyographic signals;
the processor is used for determining a target operation instruction according to the brain wave signal and the preset type of user information, and controlling the electronic device through the target operation instruction.
In a second aspect, an embodiment of the present application provides an operation control method applied to an electronic device including a brain wave sensor, the method including:
collecting brain wave signals of a user through the brain wave sensor;
acquiring user information of a preset type, wherein the user information of the preset type comprises gesture information and electromyographic signals;
and determining a target operation instruction according to the brain wave signal and the preset type of user information, and controlling the electronic device through the target operation instruction.
In a third aspect, the present invention provides an operation control device, which is applied to an electronic device including a brain wave sensor, the operation control device including an acquisition unit, and a determination unit, wherein,
the acquisition unit is used for acquiring brain wave signals of a user through the brain wave sensor;
the acquisition unit is used for acquiring preset types of user information, and the preset types of user information comprise gesture information and electromyographic signals;
the determining unit is used for determining a target operation instruction according to the brain wave signals acquired by the acquiring unit and the preset type of user information acquired by the acquiring unit, and controlling the electronic device through the target operation instruction.
In a fourth aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the steps of any of the methods in the second aspect of the embodiment of the present application.
In a fifth aspect, the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program for electronic data exchange, where the computer program makes a computer perform part or all of the steps described in any one of the methods in the second aspect of the present application.
In a sixth aspect, the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform some or all of the steps described in any one of the methods of the second aspect of the present application. The computer program product may be a software installation package.
It can be seen that, in the embodiment of the application, an electronic device firstly acquires brain wave signals of a user through a brain wave sensor, secondly acquires preset types of user information, and finally determines a target operation instruction according to the brain wave signals and the preset types of user information and controls the electronic device through the target operation instruction. Therefore, the electronic device can realize automatic operation control on the electronic device by collecting the brain waves of the user and acquiring the finger information and the myoelectric signals of the user, and is favorable for improving the intelligence of the electronic device, and the brain waves and the myoelectric signals are the same type of data and can be simultaneously acquired by the same equipment, so that the electronic device is controlled by the brain waves and the myoelectric signals, the convenience of control is improved, the hardware cost is low, the control speed is improved, in addition, the gesture information can be accurately expressed by the user, but not the heart rate, the eye spirit and other incompletely controllable data, therefore, the gesture information can accurately express the requirements of the user, and the accuracy of the electronic device is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1A is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure;
fig. 1B is a schematic structural diagram of a brain wave sensor signal receiver according to an embodiment of the present application;
fig. 1C is a schematic structural diagram of a chip-type signal collector of a brain wave sensor according to an embodiment of the present disclosure;
fig. 1D is a schematic structural diagram of another electronic device provided in the embodiment of the present application;
fig. 2A is a schematic flow chart of an operation control method according to an embodiment of the present application;
FIG. 2B is an exemplary diagram of a waveform of an electroencephalogram provided by an embodiment of the present application;
FIG. 2C is a diagram illustrating an example of an electromyogram waveform provided by an embodiment of the present application;
FIG. 2D is a flowchart illustrating a process of forming a target amplitude spectrogram by electroencephalography and electromyography according to an embodiment of the present disclosure;
FIG. 3 is a schematic flow chart diagram illustrating another method for controlling operations provided by an embodiment of the present application;
FIG. 4 is a schematic flow chart diagram illustrating another operation control method provided by an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure;
fig. 6 is a block diagram of functional units of an operation control device according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," and the like in the description and claims of the present application and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
The electronic device according to the embodiment of the present application may include various handheld devices, vehicle-mounted devices, wearable devices, computing devices or other processing devices connected to a wireless modem, and various forms of User Equipment (UE), Mobile Stations (MS), terminal devices (terminal device), and the like. For convenience of description, the above-mentioned apparatuses are collectively referred to as electronic devices.
The following describes embodiments of the present application in detail.
Referring to fig. 1A, fig. 1A is a schematic structural diagram of an electronic device 100 according to an embodiment of the present invention, where the electronic device 100 includes: a case 110, a display 120 disposed on the case 110, a main board 130 disposed in the case 110, a processor 140 disposed on the main board 130, a memory 150 connected to the processor 140, a brain wave sensor 160, a plurality of sensors 170 except the brain wave sensor, and the like, the processor 140 connected to the display 120, the electronic device 100 further comprising a radio frequency system 180, the radio frequency system 180 comprising a transmitter 181, a receiver 182, a signal processor 183, wherein,
the brain wave sensor 160 is used for collecting brain wave signals of a user;
the plurality of sensors 170 are configured to acquire preset types of user information, where the preset types of user information include gesture information and electromyographic signals;
the processor 140 is configured to determine a target operation instruction according to the brain wave signal and the preset type of user information, and control the electronic device 100 according to the target operation instruction.
The display 120 includes a display driving circuit, a display screen, and a touch panel, wherein the display driving circuit is configured to control the display screen to display content according to display data and display parameters (e.g., brightness, color, saturation, etc.) of a picture, the touch panel is configured to detect a touch operation, and the display screen is an organic light emitting diode display screen OLED.
The plurality of sensors 170 other than the electroencephalogram sensor may be cameras, myoelectricity sensors, or the like, and are not limited herein.
The brain wave sensor 160 may also be referred to as a brain wave chip, a brain wave receiver, or the like, the brain wave sensor 160 is integrated in an electronic device, has a dedicated signal processing circuit, is connected to the processor 140 of the electronic device, and may be divided into a current type brain wave sensor 160 and an electromagnetic type brain wave sensor 160 according to the type of a signal collected, the current type brain wave sensor 160 collects a bioelectric current generated from a cerebral cortex, and the electromagnetic type brain wave sensor 160 collects an electromagnetic wave radiated from the brain of a human being during an activity. It is understood that the specific form of the brain wave sensor 160 may be various, for example, a wearable brain wave sensor, or a chip brain wave sensor, and the like, and is not limited herein.
For example, as shown in fig. 1B and 1C, the electroencephalogram sensor 160 may include a signal receiver 161 and a chip-type signal collector 162, the signal receiver 161 may be housed in the electronic device shown in fig. 1B, and when in use, as shown in fig. 1C, the chip-type signal collector 162 and the electronic device are connected in a wireless manner, and are connected in a communication manner through a wireless communication module in the chip-type signal collector 162.
The size of the main board 130 may be any size and shape that can be accommodated by the electronic device 100, and is not limited herein.
The processor 140 includes an application processor and a baseband processor, the processor 140 is a control center of the electronic device 100, connects various parts of the whole electronic device by using various interfaces and lines, and performs various functions of the electronic device 100 and processes data by running or executing software programs and/or modules stored in the memory 150 and calling data stored in the memory 150, thereby performing overall monitoring of the electronic device 100. The application processor mainly processes an operating system, a user interface, application programs and the like, and the baseband processor mainly processes wireless communication. It will be appreciated that the baseband processor described above may not be integrated into the processor.
The memory 150 may be used for storing software programs and modules, and the processor 140 executes various functional applications and data processing of the electronic device 100 by operating the software programs and modules stored in the memory 150. The memory 150 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to use of the electronic device, and the like. Further, the memory 150 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
In a specific implementation, the electronic device may control the brain wave sensor 160 to operate in a low power mode in a stationary state and in a high frequency mode in a moving state, thereby reducing power consumption.
It can be seen that, in the embodiment of the application, an electronic device firstly acquires brain wave signals of a user through a brain wave sensor, secondly acquires preset types of user information, and finally determines a target operation instruction according to the brain wave signals and the preset types of user information and controls the electronic device through the target operation instruction. Therefore, the electronic device can realize automatic operation control on the electronic device by collecting the brain waves of the user and acquiring the finger information and the myoelectric signals of the user, and is favorable for improving the intelligence of the electronic device, and the brain waves and the myoelectric signals are the same type of data and can be simultaneously acquired by the same equipment, so that the electronic device is controlled by the brain waves and the myoelectric signals, the convenience of control is improved, the hardware cost is low, the control speed is improved, in addition, the gesture information can be accurately expressed by the user, but not the heart rate, the eye spirit and other incompletely controllable data, therefore, the gesture information can accurately express the requirements of the user, and the accuracy of the electronic device is improved.
In one possible example, the plurality of sensors 170 includes an electromyographic sensor 171, the preset type of user information is the electromyographic signal, and in the aspect of determining the target operation instruction according to the brain wave signal and the preset type of user information, the processor 140 is specifically configured to: forming an electroencephalogram from the brain wave signals; the electromyogram is formed according to the electromyogram signals acquired by the electromyogram sensor; and is configured to determine, according to a mapping relationship between a preset electroencephalogram template, an electromyogram template and an operation instruction stored in the memory 150, that the operation instruction corresponding to the electroencephalogram and the electromyogram is the target operation instruction.
The electromyographic sensor 171 is a sensor that can receive action potential waveforms of muscle motor units (muscle fiber cells) and convert the waveforms into usable output signals, and as shown in fig. 1D, the electromyographic sensor 171 and the brain wave sensor 160 may use the same signal receiver 161, and the usage principle of the electromyographic sensor 171 is similar to that of the above-mentioned brain wave sensor, and will not be described in detail here.
In this possible example, in the aspect of determining that the operation instructions corresponding to the electroencephalogram and the electromyogram are the target operation instructions according to the mapping relationship between the preset electroencephalogram template, electromyogram template and operation instructions stored in the memory 150, the processor 140 is specifically configured to: forming a target amplitude frequency spectrogram by using the first amplitude group and the first frequency group of the electroencephalogram and the second amplitude group and the second frequency group of the electromyogram through frequency normalization processing; and an operation instruction corresponding to the amplitude spectrogram template stored in the memory 150 and matched with the target amplitude spectrogram is determined as the target operation instruction, wherein the amplitude spectrogram template is formed by the electroencephalogram template and the electromyogram template.
In one possible example, the plurality of sensors includes an image sensor, the preset type of user information is gesture information, and in the aspect of determining the target operation instruction according to the brain wave signal and the preset type of user information, the processor 140 is specifically configured to: determining an event to be executed of the user according to the gesture information; and the target operation instruction corresponding to the event to be executed is determined according to the brain wave signal.
In one possible example, in the aspect of determining the target operation instruction according to the brain wave signal and the preset type of user information, the processor 140 is specifically configured to: determining a first operation instruction corresponding to the brain wave signal; determining a second operation instruction corresponding to the preset type of user information; when the first operation instruction and the second operation instruction are detected to be the same, determining that the first operation instruction or the second operation instruction is the target operation instruction.
Referring to fig. 2A, fig. 2A is a schematic flow chart of an operation control method applied to the electronic device shown in fig. 1A-1D and applied to an electronic device including a brain wave sensor according to an embodiment of the present application, and as shown in the figure, the operation control method includes:
s201, the electronic device collects brain wave signals of a user through the brain wave sensor;
the brain wave is a method for recording brain activity by using electrophysiological indexes, records the electric wave change of the brain in an active device, and is the overall reflection of the electrophysiological activity of brain nerve cells on the surface of a cerebral cortex or a scalp, so that the brain wave signal is a collective nerve activity signal generated by the common discharge of a plurality of nerves.
S202, the electronic device acquires user information of a preset type, wherein the user information of the preset type comprises gesture information and electromyographic signals;
the preset type of user information may further include heart rate data of a user, eyeball data (e.g., a rotation angle of an eyeball, a focus area of the eyeball, etc.), wherein the gesture data may be acquired by a camera, a 3D camera, a touch screen, or a pressure sensor of the touch screen, etc., the electromyographic signal may be an electromyographic signal of an eye muscle, an electromyographic signal of a hand muscle, or an electromyographic signal around a heart, etc., and the electromyographic signal is acquired by an electromyographic sensor, which is not limited herein.
S203, the electronic device determines a target operation instruction according to the brain wave signal and the preset type of user information, and controls the electronic device through the target operation instruction.
The electroencephalogram signal is a specific word, vocabulary or phrase which is meditated when a user wants the electronic device to execute a target operation instruction, and the preset type of user information is a gesture action or an electromyogram signal (such as eyeball rotation, gesture command and the like) of a certain part which is performed when the user wants the electronic device to execute the target operation instruction, for example, the target operation instruction which the user wants the electronic device to execute is that the vocabulary meditated when the user wants to execute a photographing operation is 'photographing' or '1', and the gesture performed is five-finger opening or habitual action when the user photographs; for example, the target operation command that the user wants the electronic device to execute is "music" or "2" as the specific word meditation is performed when music is played, the gesture is regular beating the display screen, and the like, and for different target operation commands, the words, words or phrases of meditation are different from one another and the action or myoelectric signal to be executed is also different from one another, so the electroencephalogram signal and the preset type of user information received by the electronic device are also different.
The target operation instruction, the electroencephalogram signal and the preset type of user information can be associated in pairs or can be a mapping relation between the three, and the mapping relation is not limited uniquely, the relation between the three can be established in the electronic device through data training by interaction with the electronic device when the user uses the electronic device for the first time, for example, when the mobile phone is unlocked, the user unlocks the two characters through meditation, so that the electroencephalogram sensor receives the electroencephalogram signal of the user, the electromyogram sensor receives the electromyogram signal around the heart of the user, an electroencephalogram template corresponding to the electroencephalogram signal is formed, an electromyogram template formed by the user information and the like is established in the electronic device, and the electroencephalogram data and the preset type of user information when the user performs corresponding actions can also be acquired by the user in the process of using the electronic device, the method is convenient and fast to establish under the condition that a user does not know without interaction with the user, for example, when the user uses a mobile phone to unlock, the electroencephalogram sensor and other types of sensors acquire electroencephalogram signals of the user and preset types of user information, an electroencephalogram template and a user information template are further formed, the corresponding relation among the electroencephalogram template and the user information template is established in the electronic device, in addition, a target operation instruction is determined through more than two types of data (the electroencephalogram signals and other types of user information), and the accuracy of determining the target operation instruction is favorably improved.
The target operation instruction can control the electronic device to perform operations at least including any one of the following operations: audio playing, video playing, photographing, unlocking, quick charging, album displaying, game starting, screen adjusting (including interface adjusting, screen locking and magazine adjusting, wallpaper changing), payment, working mode adjusting, telephone calling and answering, application starting, downloading, searching and the like, and the method is not limited in the specification.
It can be seen that, in the embodiment of the application, an electronic device firstly acquires brain wave signals of a user through a brain wave sensor, secondly acquires preset types of user information, and finally determines a target operation instruction according to the brain wave signals and the preset types of user information and controls the electronic device through the target operation instruction. Therefore, the electronic device can realize automatic operation control on the electronic device by collecting the brain waves of the user and acquiring the finger information and the myoelectric signals of the user, and is favorable for improving the intelligence of the electronic device, and the brain waves and the myoelectric signals are the same type of data and can be simultaneously acquired by the same equipment, so that the electronic device is controlled by the brain waves and the myoelectric signals, the convenience of control is improved, the hardware cost is low, the control speed is improved, in addition, the gesture information can be accurately expressed by the user, but not the heart rate, the eye spirit and other incompletely controllable data, therefore, the gesture information can accurately express the requirements of the user, and the accuracy of the electronic device is improved.
In one possible example, the determining of the target operation instruction according to the brain wave signal and the preset type of user information is an electromyographic signal, and includes:
forming an electroencephalogram from the brain wave signals;
forming an electromyogram according to the electromyogram signal;
and determining the operation instruction corresponding to the electroencephalogram and the electromyogram as the target operation instruction according to the preset mapping relation among the electroencephalogram template, the electromyogram template and the operation instruction.
Wherein the electroencephalogram includes an amplitude of about 50 μ V to about 200 μ V and a frequency of about 1 Hz to about 30 Hz.
The specific implementation manner of forming the electroencephalogram according to the brain wave signals may be that the electronic device identifies the brain wave information by using a hilbert-Huang Transform (HHT) algorithm, and then performs denoising and filtering processing and analog-to-digital a/D conversion on the brain wave signals to obtain target brain wave signals, and then forms the electroencephalogram according to the target brain wave signals.
The preset type of user information is an electromyogram signal, the electronic device acquires an electromyogram through an electromyogram sensor, and forms an electromyogram according to the electromyogram signal, the electromyogram also comprises amplitude and frequency, and the electromyograms of the same part of different users are different, so that whether the user is the user himself can be judged through the electromyogram signal, the safety of the electronic device is improved, the electromyogram signals of different parts of the same user are different, and the electromyogram signal can be an electromyogram signal of a hand when the user holds the electronic device by a hand, an electromyogram signal of an eye muscle when a user rotates an eyeball, an electromyogram signal of a muscle around a lip, or an electromyogram signal around a heart when the user beats, and the like, which are not limited uniquely.
The same electroencephalogram template may correspond to a plurality of electromyogram templates, different electromyogram templates correspond to different target operation instructions, and a unique mapping relation is formed among the electroencephalogram template, the electromyogram template and the operation instructions.
Therefore, in the example, the electronic device correspondingly determines the only target operation instruction through the acquired brain wave signal and the acquired electromyogram signal according to the mapping relation among the electroencephalogram template, the electromyogram template and the operation instruction, the algorithm is simple, convenience in determining the target operation instruction is improved, and the operation control speed is improved.
In this possible example, the determining, according to a mapping relationship between a preset electroencephalogram template, an electromyogram template, and an operation instruction, that the operation instruction corresponding to the electroencephalogram and the electromyogram is the target operation instruction includes:
forming a target amplitude frequency spectrogram by using the first amplitude group and the first frequency group of the electroencephalogram and the second amplitude group and the second frequency group of the electromyogram through frequency normalization processing;
and determining an operation instruction corresponding to an amplitude spectrogram template matched with the target amplitude spectrogram as the target operation instruction, wherein the amplitude spectrogram template is formed by the electroencephalogram template and the electromyogram template.
The electronic device performs normalization processing on the first frequency group and the second frequency group in the electromyogram and the electroencephalogram, presents the electromyogram and the electroencephalogram on a target amplitude spectrogram, and compares the target amplitude spectrogram with an amplitude spectrogram template.
The electrode positions corresponding to the electroencephalogram signals and the electromyogram signals in the amplitude spectrogram templates are the same as the electrode positions obtained by the currently obtained electroencephalogram signals and electromyogram signals.
For example, when the user needs to execute the target operation instruction to unlock the screen, the electronic device obtains an electroencephalogram formed by electroencephalogram signals obtained within a preset sampling time interval, and the electroencephalogram after the electroencephalogram frequency normalization processing is the electroencephalogram shown in fig. 2B, an electromyogram formed by an electromyogram signal obtained by the electronic device being held by the palm of the user's hand, and an electromyogram after the electromyogram frequency normalization processing is the electromyogram shown in fig. 2C, and then the electronic device fuses the electroencephalogram and the electromyogram after the normalization processing into an amplitude spectrum shown in fig. 2D, and maximizes the amplitude at the same frequency to obtain the target amplitude spectrum. In addition, the target amplitude spectrogram may be obtained by taking the minimum value of the amplitudes at the same frequency point after being fused into the legend on the left side in fig. 2D, or taking the average value of the amplitudes at each frequency point, which is not limited herein.
Therefore, in the example, the electronic device determines the target operation instruction by comparing the electroencephalogram data and the electromyogram data, so that the accuracy of operation control is improved, and the electroencephalogram and the electromyogram are fused into an amplitude map and matched with the template by normalizing the frequency of the electroencephalogram and the electromyogram, so that the convenience of matching and comparison is improved.
In one possible example, the preset type of user information is gesture information, and the determining a target operation instruction according to the brain wave signal and the preset type of user information includes:
determining an event to be executed of the user according to the gesture information;
and determining the target operation instruction corresponding to the event to be executed according to the brain wave signal.
The events to be executed can be the above-mentioned events such as audio playing, video playing, photographing, unlocking, quick charging, album displaying, game starting and the like, and different events to be executed correspond to different electroencephalogram templates of the meditation of the user preset by the electronic device.
The gesture information is a trigger condition, and the gesture information may be any gesture information, for example, a habitual gesture of a user, or any gesture information customized by the user, for example, when the gesture information of the current user is the habitual gesture of the user, the corresponding event to be executed is to start the application a.
The specific implementation manner of determining the target operation instruction corresponding to the event to be executed according to the brain wave signal may be: acquiring an electroencephalogram template matched with the event to be executed; forming the brain wave signals into an electroencephalogram; and when the electroencephalogram is successfully matched with the electroencephalogram template, determining the target operation instruction corresponding to the event to be executed.
Therefore, in this example, after the electronic device determines the event to be executed of the user by using the gesture information of the user as the trigger condition, the electroencephalogram matching operation is performed, which is beneficial to avoiding power consumption caused by continuous matching calculation of electroencephalogram images when no trigger condition exists, and is beneficial to reducing power consumption of the electronic device.
In one possible example, the determining a target operation instruction according to the brain wave signal and the preset type of user information includes:
determining a first operation instruction corresponding to the brain wave signal;
determining a second operation instruction corresponding to the preset type of user information;
when the first operation instruction and the second operation instruction are detected to be the same, determining that the first operation instruction or the second operation instruction is the target operation instruction.
The electronic device can comprise a first operation instruction set and a second operation instruction set, wherein the first operation instruction set comprises a plurality of groups of electroencephalograms and corresponding relations between operation instructions, and the second operation instruction set comprises a plurality of groups of corresponding relations between user information and operation instructions.
The electronic device can determine that an operation instruction matched with the electroencephalogram in the first operation instruction set is a first operation instruction, and determine that an operation instruction matched with the preset type of user information in the second operation instruction set is a second operation instruction.
The preset type of user information may be gesture data of a user, and the second operation instruction is a correspondence between a plurality of user gestures and an operation instruction, for example, the electronic device obtains different gestures through a camera, and determines an operation instruction that the user needs to execute according to the gesture.
As can be seen, in this example, the electronic device determines the target operation instruction by determining whether the first operation instruction formed by electroencephalogram and the second operation instruction determined by the user information are the same, thereby implementing dual authentication to determine the operation instruction, and further improving the accuracy of operation control.
Referring to fig. 3, in accordance with the embodiment shown in fig. 2A, fig. 3 is a flowchart illustrating an operation control method according to an embodiment of the present application, applied to an electronic device as shown in fig. 1A-1D, applied to an electronic device including a brain wave sensor, as shown in the figure, the method includes:
and S301, the electronic device collects brain wave signals of the user through the brain wave sensor.
S302, the electronic device acquires the electromyographic signals of the user.
And S303, forming an electroencephalogram according to the brain wave signals by the electronic device.
S304, the electronic device forms an electromyogram according to the electromyogram signal.
S305, the electronic device forms a target amplitude spectrogram through frequency normalization processing on the first amplitude group and the first frequency group of the electroencephalogram and the second amplitude group and the second frequency group of the electromyogram.
S306, the electronic device determines an operation instruction corresponding to the amplitude spectrogram template matched with the target amplitude spectrogram as the target operation instruction.
The amplitude spectrogram template is formed by the electronic device through the acquired electroencephalogram template and the acquired electromyogram template.
S307, the electronic device controls the electronic device through the target operation instruction.
It can be seen that, in the embodiment of the application, an electronic device firstly acquires brain wave signals of a user through a brain wave sensor, secondly acquires preset types of user information, and finally determines a target operation instruction according to the brain wave signals and the preset types of user information and controls the electronic device through the target operation instruction. Therefore, the electronic device can realize automatic operation control on the electronic device by collecting the brain waves of the user and acquiring the finger information and the myoelectric signals of the user, and is favorable for improving the intelligence of the electronic device, and the brain waves and the myoelectric signals are the same type of data and can be simultaneously acquired by the same equipment, so that the electronic device is controlled by the brain waves and the myoelectric signals, the convenience of control is improved, the hardware cost is low, the control speed is improved, in addition, the gesture information can be accurately expressed by the user, but not the heart rate, the eye spirit and other incompletely controllable data, therefore, the gesture information can accurately express the requirements of the user, and the accuracy of the electronic device is improved.
In addition, the electronic device determines a target operation instruction by comparing the electroencephalogram data and the electromyogram data, so that the accuracy of operation control is improved, and the electroencephalogram and the electromyogram are fused into an amplitude map and matched with a template by normalizing the frequency of the electroencephalogram and the electromyogram, so that the convenience of matching and comparison is improved.
Referring to fig. 4, fig. 4 is a flowchart illustrating an operation control method according to an embodiment of the present disclosure, and the method is applied to the electronic device shown in fig. 1A-1D. As shown in the figure, the operation control method includes:
s401, the electronic device collects brain wave signals of a user through the brain wave sensor;
s402, the electronic device acquires gesture information of a user;
s403, the electronic device determines an event to be executed of the user according to the gesture information;
s404, the electronic device determines the target operation instruction corresponding to the event to be executed according to the brain wave signal.
S405, the electronic device controls the electronic device through the target operation instruction.
It can be seen that, in the embodiment of the application, an electronic device firstly acquires brain wave signals of a user through a brain wave sensor, secondly acquires preset types of user information, and finally determines a target operation instruction according to the brain wave signals and the preset types of user information and controls the electronic device through the target operation instruction. Therefore, the electronic device can realize automatic operation control on the electronic device by collecting the brain waves of the user and acquiring the finger information and the myoelectric signals of the user, and is favorable for improving the intelligence of the electronic device, and the brain waves and the myoelectric signals are the same type of data and can be simultaneously acquired by the same equipment, so that the electronic device is controlled by the brain waves and the myoelectric signals, the convenience of control is improved, the hardware cost is low, the control speed is improved, in addition, the gesture information can be accurately expressed by the user, but not the heart rate, the eye spirit and other incompletely controllable data, therefore, the gesture information can accurately express the requirements of the user, and the accuracy of the electronic device is improved.
In addition, after the electronic device determines the event to be executed of the user by using the preset type of user information as the trigger condition, the matching operation of the electroencephalogram is performed, so that the power consumption caused by continuous matching calculation of the electroencephalogram images when the trigger condition is not available is avoided, and the power consumption of the electronic device is reduced.
In accordance with the embodiments shown in fig. 2A, fig. 3, and fig. 4, please refer to fig. 5, and fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application, where the electronic device includes a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for performing the following steps;
collecting brain wave signals of a user through the brain wave sensor;
acquiring user information of a preset type, wherein the user information of the preset type comprises gesture information and electromyographic signals;
and determining a target operation instruction according to the brain wave signal and the preset type of user information, and controlling the electronic device through the target operation instruction.
It can be seen that, in the embodiment of the application, an electronic device firstly acquires brain wave signals of a user through a brain wave sensor, secondly acquires preset types of user information, and finally determines a target operation instruction according to the brain wave signals and the preset types of user information and controls the electronic device through the target operation instruction. Therefore, the electronic device can realize automatic operation control on the electronic device by collecting the brain waves of the user and acquiring the finger information and the myoelectric signals of the user, and is favorable for improving the intelligence of the electronic device, and the brain waves and the myoelectric signals are the same type of data and can be simultaneously acquired by the same equipment, so that the electronic device is controlled by the brain waves and the myoelectric signals, the convenience of control is improved, the hardware cost is low, the control speed is improved, in addition, the gesture information can be accurately expressed by the user, but not the heart rate, the eye spirit and other incompletely controllable data, therefore, the gesture information can accurately express the requirements of the user, and the accuracy of the electronic device is improved.
In one possible example, the preset type of user information is an electromyographic signal, and in the aspect of determining the target operation instruction according to the brain wave signal and the preset type of user information, the instruction in the program is specifically configured to perform the following operations: forming an electroencephalogram from the brain wave signals; and for forming an electromyogram from the electromyogram signal; and the system is used for determining the operation instruction corresponding to the electroencephalogram and the electromyogram as the target operation instruction according to the mapping relation among the preset electroencephalogram template, electromyogram template and operation instruction.
In this possible example, in the aspect that the operation instruction corresponding to the electroencephalogram and the electromyogram is determined to be the target operation instruction according to the mapping relationship between the preset electroencephalogram template, electromyogram template and operation instruction, the instructions in the program are specifically configured to perform the following operations: forming a target amplitude frequency spectrogram by using the first amplitude group and the first frequency group of the electroencephalogram and the second amplitude group and the second frequency group of the electromyogram through frequency normalization processing; and the operation instruction corresponding to the amplitude spectrogram template matched with the target amplitude spectrogram is determined to be the target operation instruction, and the amplitude spectrogram template is formed by the electroencephalogram template and the electromyogram template.
In one possible example, the preset type of user information is gesture information, and in the aspect of determining the target operation instruction according to the brain wave signal and the preset type of user information, the instruction in the program is specifically configured to perform the following operations: determining an event to be executed of the user according to the gesture information; and the target operation instruction corresponding to the event to be executed is determined according to the brain wave signal.
In one possible example, in the aspect of determining the target operation instruction according to the brain wave signal and the preset type of user information, the instructions in the program are specifically configured to perform the following operations: determining a first operation instruction corresponding to the brain wave signal; the second operation instruction is used for determining the second operation instruction corresponding to the preset type of user information; and when the first operation instruction and the second operation instruction are detected to be the same, determining that the first operation instruction or the second operation instruction is the target operation instruction.
The above description has introduced the solution of the embodiment of the present application mainly from the perspective of the method-side implementation process. It is understood that the electronic device comprises corresponding hardware structures and/or software modules for performing the respective functions in order to realize the above functions. Those of skill in the art will readily appreciate that the present application is capable of hardware or a combination of hardware and computer software implementing the various illustrative elements and algorithm steps described in connection with the embodiments provided herein. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiment of the present application, the electronic device may be divided into the functional units according to the method example, for example, each functional unit may be divided corresponding to each function, or two or more functions may be integrated into one processing unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit. It should be noted that the division of the unit in the embodiment of the present application is schematic, and is only a logic function division, and there may be another division manner in actual implementation.
Fig. 6 is a block diagram showing functional units of the operation control device 600 according to the embodiment of the present application. The operation control apparatus 600 is applied to an electronic apparatus including a brain wave sensor, the operation control apparatus 600 includes an acquisition unit 601, an acquisition unit 602, and a determination unit 603, wherein,
the acquisition unit 601 is used for acquiring brain wave signals of a user through the brain wave sensor;
the acquiring unit 602 is configured to acquire preset types of user information, where the preset types of user information include gesture information and an electromyographic signal;
the determining unit 603 is configured to determine a target operation instruction according to the brain wave signal acquired by the acquiring unit 601 and the preset type of user information acquired by the acquiring unit 602, and control the electronic apparatus according to the target operation instruction.
It can be seen that, in the embodiment of the application, an electronic device firstly acquires brain wave signals of a user through a brain wave sensor, secondly acquires preset types of user information, and finally determines a target operation instruction according to the brain wave signals and the preset types of user information and controls the electronic device through the target operation instruction. Therefore, the electronic device can realize automatic operation control on the electronic device by collecting the brain waves of the user and acquiring the finger information and the myoelectric signals of the user, and is favorable for improving the intelligence of the electronic device, and the brain waves and the myoelectric signals are the same type of data and can be simultaneously acquired by the same equipment, so that the electronic device is controlled by the brain waves and the myoelectric signals, the convenience of control is improved, the hardware cost is low, the control speed is improved, in addition, the gesture information can be accurately expressed by the user, but not the heart rate, the eye spirit and other incompletely controllable data, therefore, the gesture information can accurately express the requirements of the user, and the accuracy of the electronic device is improved.
In one possible example, the preset type of user information is an electromyographic signal, and in the aspect of determining the target operation instruction according to the brain wave signal and the preset type of user information, the determining unit 603 is specifically configured to: forming an electroencephalogram from the brain wave signals; and for forming an electromyogram from the electromyogram signal; and the system is used for determining the operation instruction corresponding to the electroencephalogram and the electromyogram as the target operation instruction according to the mapping relation among the preset electroencephalogram template, electromyogram template and operation instruction.
In this possible example, in terms of determining that the operation instructions corresponding to the electroencephalogram and the electromyogram are the target operation instructions according to the mapping relationship between the preset electroencephalogram template, electromyogram template, and operation instructions, the determining unit 603 is specifically configured to: forming a target amplitude frequency spectrogram by using the first amplitude group and the first frequency group of the electroencephalogram and the second amplitude group and the second frequency group of the electromyogram through frequency normalization processing; and the operation instruction corresponding to the amplitude spectrogram template matched with the target amplitude spectrogram is determined to be the target operation instruction, and the amplitude spectrogram template is formed by the electroencephalogram template and the electromyogram template.
In one possible example, the preset type of user information is gesture information, and in terms of determining the target operation instruction according to the brain wave signal and the preset type of user information, the determining unit 603 is specifically configured to: determining an event to be executed of the user according to the gesture information; and the target operation instruction corresponding to the event to be executed is determined according to the brain wave signal.
In one possible example, in the aspect of determining the target operation instruction according to the brain wave signal and the preset type of user information, the determining unit 603 is specifically configured to: determining a first operation instruction corresponding to the brain wave signal; the second operation instruction is used for determining the second operation instruction corresponding to the preset type of user information; and when the first operation instruction and the second operation instruction are detected to be the same, determining that the first operation instruction or the second operation instruction is the target operation instruction.
Wherein, the acquisition unit 601 may be a brain wave sensor, the acquisition unit 602 may be a myoelectric sensor or a camera, and the determination unit 603 may be a processor.
Embodiments of the present application also provide a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any one of the methods as described in the above method embodiments.
Embodiments of the present application also provide a computer program product comprising a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods as described in the above method embodiments. The computer program product may be a software installation package.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the above-described division of the units is only one type of division of logical functions, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of some interfaces, devices or units, and may be an electric or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit may be stored in a computer readable memory if it is implemented in the form of a software functional unit and sold or used as a stand-alone product. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a memory, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the above-mentioned method of the embodiments of the present application. And the aforementioned memory comprises: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable memory, which may include: flash Memory disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
The foregoing detailed description of the embodiments of the present application has been presented to illustrate the principles and implementations of the present application, and the above description of the embodiments is only provided to help understand the method and the core concept of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (7)
1. An electronic device characterized by comprising a processor, a brain wave sensor connected to the processor, and a plurality of sensors other than the brain wave sensor, wherein,
the brain wave sensor is used for collecting brain wave signals of a user;
the sensors are used for acquiring preset types of user information, and the preset types of user information comprise gesture information and electromyographic signals;
the processor is used for determining a target operation instruction according to the brain wave signal and the preset type of user information and controlling the electronic device through the target operation instruction;
the electronic device further includes a memory, the plurality of sensors include an electromyographic sensor, and when the preset type of user information is the electromyographic signal, in the aspect of determining the target operation instruction according to the brain wave signal and the preset type of user information, the processor is specifically configured to: forming an electroencephalogram from the brain wave signals; the electromyogram is formed according to the electromyogram signals acquired by the electromyogram sensor; the electroencephalogram and electromyogram control system is used for determining an operation instruction corresponding to the electroencephalogram and the electromyogram as the target operation instruction according to a mapping relation between a preset electroencephalogram template and an electromyogram template stored in the memory and the operation instruction;
in the aspect that the operation instruction corresponding to the electroencephalogram and the electromyogram is determined to be the target operation instruction according to the mapping relationship between the preset electroencephalogram template, electromyogram template and operation instruction stored in the memory, the processor is specifically configured to: forming a target amplitude frequency spectrogram by using the first amplitude group and the first frequency group of the electroencephalogram and the second amplitude group and the second frequency group of the electromyogram through frequency normalization processing; and the operation instruction corresponding to the amplitude spectrogram template stored in the memory and matched with the target amplitude spectrogram is determined to be the target operation instruction.
2. The electronic device according to claim 1, wherein the plurality of sensors includes an image sensor, and when the preset type of user information is gesture information, the processor is specifically configured to, in the determining of the target operation instruction according to the brain wave signal and the preset type of user information: determining an event to be executed of the user according to the gesture information; and the target operation instruction corresponding to the event to be executed is determined according to the brain wave signal.
3. An operation control method applied to an electronic apparatus including a brain wave sensor, the method comprising:
collecting brain wave signals of a user through the brain wave sensor;
acquiring user information of a preset type, wherein the user information of the preset type comprises gesture information and electromyographic signals;
determining a target operation instruction according to the brain wave signal and the preset type of user information, and controlling the electronic device through the target operation instruction;
when the preset type of user information is an electromyographic signal, determining a target operation instruction according to the brain wave signal and the preset type of user information includes:
forming an electroencephalogram from the brain wave signals;
forming an electromyogram according to the electromyogram signal;
determining an operation instruction corresponding to the electroencephalogram and the electromyogram as the target operation instruction according to a preset mapping relation among an electroencephalogram template, an electromyogram template and the operation instruction;
the method for determining the operation instruction corresponding to the electroencephalogram and the electromyogram as the target operation instruction according to the mapping relationship among the preset electroencephalogram template, electromyogram template and operation instruction comprises the following steps:
forming a target amplitude frequency spectrogram by using the first amplitude group and the first frequency group of the electroencephalogram and the second amplitude group and the second frequency group of the electromyogram through frequency normalization processing;
and determining an operation instruction corresponding to an amplitude spectrogram template matched with the target amplitude spectrogram as the target operation instruction, wherein the amplitude spectrogram template is formed by the electroencephalogram template and the electromyogram template.
4. The method according to claim 3, wherein when the preset type of user information is gesture information, the determining a target operation instruction according to the brain wave signal and the preset type of user information comprises:
determining an event to be executed of the user according to the gesture information;
and determining the target operation instruction corresponding to the event to be executed according to the brain wave signal.
5. An operation control device applied to an electronic device including a brain wave sensor, the operation control device including a collecting unit, an acquiring unit, and a determining unit,
the acquisition unit is used for acquiring brain wave signals of a user through the brain wave sensor;
the acquisition unit is used for acquiring preset types of user information, and the preset types of user information comprise gesture information and electromyographic signals;
the determining unit is used for determining a target operation instruction according to the brain wave signal acquired by the acquiring unit and the preset type of user information acquired by the acquiring unit, and controlling the electronic device through the target operation instruction;
when the preset type of user information is an electromyographic signal, in terms of determining a target operation instruction according to the brain wave signal and the preset type of user information, the determining unit is specifically configured to: forming an electroencephalogram from the brain wave signals; and for forming an electromyogram from the electromyogram signal; the electroencephalogram and electromyogram processing system is used for determining an operation instruction corresponding to the electroencephalogram and the electromyogram as the target operation instruction according to a preset mapping relation between the electroencephalogram template and the electromyogram template and the operation instruction;
in the aspect that the operation instruction corresponding to the electroencephalogram and the electromyogram is determined to be the target operation instruction according to a mapping relationship between a preset electroencephalogram template, an electromyogram template and an operation instruction, the determining unit is specifically configured to: forming a target amplitude frequency spectrogram by using the first amplitude group and the first frequency group of the electroencephalogram and the second amplitude group and the second frequency group of the electromyogram through frequency normalization processing; and the operation instruction corresponding to the amplitude spectrogram template matched with the target amplitude spectrogram is determined to be the target operation instruction, and the amplitude spectrogram template is formed by the electroencephalogram template and the electromyogram template.
6. An electronic device comprising a processor, a memory, a communication interface, and one or more programs stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps in the method of any of claims 3-4.
7. A computer-readable storage medium, characterized in that a computer program for electronic data exchange is stored, wherein the computer program causes a computer to perform the method according to any of the claims 3-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810140828.5A CN108415560B (en) | 2018-02-11 | 2018-02-11 | Electronic device, operation control method and related product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810140828.5A CN108415560B (en) | 2018-02-11 | 2018-02-11 | Electronic device, operation control method and related product |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108415560A CN108415560A (en) | 2018-08-17 |
| CN108415560B true CN108415560B (en) | 2020-12-04 |
Family
ID=63128460
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810140828.5A Expired - Fee Related CN108415560B (en) | 2018-02-11 | 2018-02-11 | Electronic device, operation control method and related product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108415560B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109164986B (en) * | 2018-08-27 | 2024-02-09 | 平安科技（深圳）有限公司 | Cloud disk data processing method and device, electronic equipment and storage medium |
| CN109298787B (en) * | 2018-09-14 | 2020-01-17 | 清华大学 | Flexible human-computer interaction device and terminal equipment |
| CN109645989B (en) * | 2018-12-10 | 2021-01-08 | 燕山大学 | Anesthesia depth estimation system |
| CN111857326A (en) * | 2019-04-25 | 2020-10-30 | 北京小米移动软件有限公司 | Signal control method and device |
| CN111290579B (en) * | 2020-02-10 | 2022-05-20 | Oppo广东移动通信有限公司 | Control method and device of virtual content, electronic equipment and computer readable medium |
| CN112965391A (en) * | 2021-01-29 | 2021-06-15 | 青岛海尔空调器有限总公司 | Household appliance control method, household appliance control system and wearable device |
| CN114917544B (en) * | 2022-05-13 | 2023-09-22 | 上海交通大学医学院附属第九人民医院 | Visual method and device for assisting orbicularis stomatitis function training |
| CN115756178B (en) * | 2023-01-10 | 2023-06-06 | 深圳市心流科技有限公司 | Meditation application program control method, meditation application program control device, terminal and storage medium |
| CN116449967A (en) * | 2023-06-20 | 2023-07-18 | 浙江强脑科技有限公司 | Bionic hand teaching aid, control method thereof and main control equipment |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101057795A (en) * | 2007-05-18 | 2007-10-24 | 天津大学 | Artificial hand using muscle electrical and electroencephalogram cooperative control and controlling method thereof |
| CN104516485A (en) * | 2013-09-28 | 2015-04-15 | 南京专创知识产权服务有限公司 | Glasses and mobile phone interaction system based on idea control |
| CN105046887A (en) * | 2015-04-16 | 2015-11-11 | 杨燕 | Stock-trader-state electronic detection system |
| CN105117018A (en) * | 2015-09-08 | 2015-12-02 | 长城信息产业股份有限公司 | System and method for interchanging information by utilizing brain wave and eyeball state |
| CN105138119A (en) * | 2015-08-04 | 2015-12-09 | 湖南七迪视觉科技有限公司 | Stereo vision system with automatic focusing controlled based on human biometrics |
| CN205039859U (en) * | 2015-08-10 | 2016-02-17 | 瑞声声学科技(深圳)有限公司 | Cell -phone shows extend system |
| CN205375373U (en) * | 2016-01-19 | 2016-07-06 | 郑州轻工业学院 | Intelligence house based on brain wave and gesture control |
| CN205485920U (en) * | 2016-01-21 | 2016-08-17 | 京东方科技集团股份有限公司 | Show controlling means and display control system |
| CN106527716A (en) * | 2016-11-09 | 2017-03-22 | 努比亚技术有限公司 | Wearable equipment based on electromyographic signals and interactive method between wearable equipment and terminal |
| CN106980359A (en) * | 2016-01-19 | 2017-07-25 | 美的集团股份有限公司 | Control method, system, headset equipment and the server of household electrical appliance |
| CN107390714A (en) * | 2017-06-13 | 2017-11-24 | 深圳市易成自动驾驶技术有限公司 | Control method, device and the computer-readable recording medium of unmanned plane |
| CN107479704A (en) * | 2017-08-08 | 2017-12-15 | 广东小天才科技有限公司 | Control method, device and equipment of wearable equipment and storage medium |
| CN107678541A (en) * | 2017-09-20 | 2018-02-09 | 深圳市科迈爱康科技有限公司 | Intelligent glasses and its information gathering and transmission method, computer-readable recording medium |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100594858C (en) * | 2008-08-01 | 2010-03-24 | 杭州电子科技大学 | Electric artificial hand combined controlled by brain electricity and muscle electricity and control method |
| CN101776981B (en) * | 2010-01-07 | 2011-11-16 | 天津大学 | Method for controlling mouse by jointing brain electricity and myoelectricity |
| CN102728068B (en) * | 2012-07-03 | 2014-04-09 | 北京理工大学珠海学院 | Control system and control method for electric toy car |
| JP6344084B2 (en) * | 2014-06-20 | 2018-06-20 | オムロン株式会社 | Worker terminal for robot operation |
| CN104398325B (en) * | 2014-11-05 | 2016-08-03 | 西安交通大学 | The device and method of brain-myoelectric-controlled prosthesis based on scene stable state vision inducting |
| CN106227354A (en) * | 2016-08-31 | 2016-12-14 | 博睿康科技（常州）股份有限公司 | A kind of brain-machine interaction donning system |
| CN107357311B (en) * | 2017-07-28 | 2020-03-24 | 南京航空航天大学 | Unmanned aerial vehicle reconnaissance system based on hybrid control technology |
-
2018
- 2018-02-11 CN CN201810140828.5A patent/CN108415560B/en not_active Expired - Fee Related
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101057795A (en) * | 2007-05-18 | 2007-10-24 | 天津大学 | Artificial hand using muscle electrical and electroencephalogram cooperative control and controlling method thereof |
| CN104516485A (en) * | 2013-09-28 | 2015-04-15 | 南京专创知识产权服务有限公司 | Glasses and mobile phone interaction system based on idea control |
| CN105046887A (en) * | 2015-04-16 | 2015-11-11 | 杨燕 | Stock-trader-state electronic detection system |
| CN105138119A (en) * | 2015-08-04 | 2015-12-09 | 湖南七迪视觉科技有限公司 | Stereo vision system with automatic focusing controlled based on human biometrics |
| CN205039859U (en) * | 2015-08-10 | 2016-02-17 | 瑞声声学科技(深圳)有限公司 | Cell -phone shows extend system |
| CN105117018A (en) * | 2015-09-08 | 2015-12-02 | 长城信息产业股份有限公司 | System and method for interchanging information by utilizing brain wave and eyeball state |
| CN205375373U (en) * | 2016-01-19 | 2016-07-06 | 郑州轻工业学院 | Intelligence house based on brain wave and gesture control |
| CN106980359A (en) * | 2016-01-19 | 2017-07-25 | 美的集团股份有限公司 | Control method, system, headset equipment and the server of household electrical appliance |
| CN205485920U (en) * | 2016-01-21 | 2016-08-17 | 京东方科技集团股份有限公司 | Show controlling means and display control system |
| CN106527716A (en) * | 2016-11-09 | 2017-03-22 | 努比亚技术有限公司 | Wearable equipment based on electromyographic signals and interactive method between wearable equipment and terminal |
| CN107390714A (en) * | 2017-06-13 | 2017-11-24 | 深圳市易成自动驾驶技术有限公司 | Control method, device and the computer-readable recording medium of unmanned plane |
| CN107479704A (en) * | 2017-08-08 | 2017-12-15 | 广东小天才科技有限公司 | Control method, device and equipment of wearable equipment and storage medium |
| CN107678541A (en) * | 2017-09-20 | 2018-02-09 | 深圳市科迈爱康科技有限公司 | Intelligent glasses and its information gathering and transmission method, computer-readable recording medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108415560A (en) | 2018-08-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108415560B (en) | Electronic device, operation control method and related product | |
| CN109271081B (en) | Roll the method and electronic equipment of screenshotss | |
| CN108509033B (en) | Information processing method and related product | |
| CN107580114B (en) | Biometric identification method, mobile terminal, and computer-readable storage medium | |
| CN108519811B (en) | Screenshot method and related product | |
| EP4206983A1 (en) | Fingerprint identification method and electronic device | |
| EP3240299A1 (en) | Headphone-based intelligent apparatus control method, device and system | |
| CN108491076B (en) | Display control method and related product | |
| CN109144245B (en) | Equipment control method and related product | |
| CN110688973B (en) | Equipment control method and related product | |
| CN108499111B (en) | Game difficulty adjusting method and related product | |
| CN107330316B (en) | Unlocking processing method and related product | |
| CN110221684A (en) | Apparatus control method, system, electronic device and computer readable storage medium | |
| CN108388340B (en) | Electronic equipment control method and related product | |
| CN108415564B (en) | Electronic device, equipment control method and related product | |
| CN112764523B (en) | Man-machine interaction method and device based on iris recognition and electronic equipment | |
| CN108375912A (en) | Intelligent home furnishing control method and Related product | |
| CN108418960B (en) | Electronic device, operation control method and related product | |
| CN108304073B (en) | Electronic device, unlocking control method and related product | |
| CN108519810B (en) | Electronic device, brain wave unlocking method and related product | |
| CN108491074B (en) | Electronic device, exercise assisting method and related product | |
| CN107463819B (en) | Unlocking processing method and related product | |
| CN110889322A (en) | Method for preventing sedentary sitting and related product | |
| CN108628445B (en) | Brain wave acquisition method and related product | |
| CN108339267B (en) | Game menu control method based on brain wave and related product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Changan town in Guangdong province Dongguan 523860 usha Beach Road No. 18 Applicant after: GUANGDONG OPPO MOBILE TELECOMMUNICATIONS Corp.,Ltd. Address before: 523860 No. 18 Wusha Haibin Road, Chang'an Town, Dongguan City, Guangdong Province Applicant before: GUANGDONG OPPO MOBILE TELECOMMUNICATIONS Corp.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20201204 |