CN112115813B - Labeling method and device for human body electromyographic signals and computing equipment - Google Patents
Labeling method and device for human body electromyographic signals and computing equipment Download PDFInfo
- Publication number
- CN112115813B CN112115813B CN202010898778.4A CN202010898778A CN112115813B CN 112115813 B CN112115813 B CN 112115813B CN 202010898778 A CN202010898778 A CN 202010898778A CN 112115813 B CN112115813 B CN 112115813B
- Authority
- CN
- China
- Prior art keywords
- joint
- electromyographic
- labeling
- sample
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000002372 labelling Methods 0.000 title claims abstract description 117
- 238000000034 method Methods 0.000 claims abstract description 56
- 238000012549 training Methods 0.000 claims description 54
- 238000004590 computer program Methods 0.000 claims description 18
- 230000003183 myoelectrical effect Effects 0.000 claims description 12
- 238000003062 neural network model Methods 0.000 claims description 8
- 230000009471 action Effects 0.000 abstract description 28
- 230000008859 change Effects 0.000 abstract description 12
- 238000012360 testing method Methods 0.000 abstract description 7
- 230000000875 corresponding effect Effects 0.000 description 25
- 239000011159 matrix material Substances 0.000 description 13
- 210000000629 knee joint Anatomy 0.000 description 12
- 210000003205 muscle Anatomy 0.000 description 10
- 230000008569 process Effects 0.000 description 10
- 230000001133 acceleration Effects 0.000 description 9
- 230000006399 behavior Effects 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 210000001503 joint Anatomy 0.000 description 7
- 210000002414 leg Anatomy 0.000 description 7
- 210000000544 articulatio talocruralis Anatomy 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 4
- 210000004394 hip joint Anatomy 0.000 description 4
- 210000003141 lower extremity Anatomy 0.000 description 4
- 210000003314 quadriceps muscle Anatomy 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 210000002683 foot Anatomy 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 210000000689 upper leg Anatomy 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 210000000988 bone and bone Anatomy 0.000 description 1
- 244000309466 calf Species 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000004377 microelectronic Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003387 muscular Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Signal Processing (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The application provides a labeling method, a labeling device and a computing device of human body electromyographic signals, and relates to the technical field of human body behavior analysis, wherein the method comprises the following steps: acquiring an angle signal and an electromyographic signal of a target joint, determining the joint moment of the target joint according to the angle signal, and finally labeling the electromyographic signal of the target joint according to the joint moment based on a predetermined advance labeling quantity. According to the technical scheme provided by the application, as the change of the joint moment is not influenced by individual differences and action differences, the joint moment is adopted to label the electromyographic signals, so that the accuracy of the labeling result can be improved; in addition, repeated testing and manual labeling are not needed, so that the labeling efficiency can be improved. In addition, when marking is carried out, the electromyographic signals of the target joint can be marked according to the joint moment based on the predetermined advance marking quantity, so that the accuracy of marking results can be further improved.
Description
Technical Field
The application relates to a human body behavior analysis technology, in particular to a labeling method, a labeling device and a computing device for human body electromyographic signals, and belongs to the technical field of human body behavior analysis signal processing.
Background
Along with the development of microelectronic technology, the development of human behavior analysis technology is driven. The human behavior analysis technology is widely applied to the fields of medical monitoring, auxiliary health treatment, human-type robots, motion prediction and the like at present.
The human body behavior analysis technology is a technology for converting abstract behavior actions into specific data by recognizing the behavior actions of a human body and then analyzing the data, wherein the conversion process from the actions to the data is a key of the technology. In the existing human behavior analysis technology, an electromyographic signal is often adopted as analyzed data, and in order to obtain a high-quality electromyographic signal, the electromyographic signal needs to be marked, and the action and the start and stop points corresponding to the electromyographic signal are determined.
The starting and stopping of each section of electromyographic signals are mainly judged according to personal experience by the existing electromyographic signal labeling work, but a plurality of muscles are needed to participate in the completion of one action, and the difference of the stress conditions of different muscles is large, so that the starting and stopping of each section of electromyographic signals can be greatly error judged only by personal experience, and the starting and stopping time corresponding to the action can be accurately found only by carrying out a plurality of tests, so that the existing data labeling work has the problems of low efficiency, time and labor waste.
Disclosure of Invention
In view of the above, the application provides a labeling method, a labeling device and a computing device for human myoelectric signals, which are used for improving the efficiency of data labeling work.
In order to achieve the above object, in a first aspect, an embodiment of the present application provides a method for labeling a human body electromyographic signal, including:
acquiring an angle signal and an electromyographic signal of a target joint;
determining joint moment of the target joint according to the angle signal;
based on a predetermined advance labeling quantity, labeling the electromyographic signals of the target joint according to the joint moment, wherein the advance labeling quantity is the generation time difference between the joint moment and the electromyographic signals.
Optionally, determining the joint moment of the target joint according to the angle signal includes:
and inputting the angle signal into a pre-trained joint moment prediction model to obtain the joint moment corresponding to the target joint.
Optionally, labeling the electromyographic signal of the target joint according to the joint moment based on a predetermined advance labeling amount comprises:
Marking the electromyographic signals of the target joint according to a predetermined advance marking quantity and joint moment;
optionally, labeling the electromyographic signal of the target joint according to the joint moment based on a predetermined advance labeling amount comprises:
The joint moment and the electromyographic signals are input into a pre-trained labeling model to obtain labeled electromyographic signals, the labeling model is obtained based on training of a training sample set, the training sample set comprises a plurality of training samples, each training sample comprises a sample electromyographic signal, a sample joint moment and labeling results of the sample electromyographic signals, and the labeling results of the sample electromyographic signals are determined according to the sample electromyographic signals, the sample joint moment and a pre-determined pre-labeling amount.
Optionally, the training method of the labeling model comprises the following steps:
Extracting signal characteristics of a sample electromyographic signal in each training sample;
And training the neural network model to be trained according to the signal characteristics of the sample electromyographic signals in each training sample, the sample joint moment and the labeling result of the sample electromyographic signals to obtain a labeling model.
Correspondingly, the joint moment and the electromyographic signals are input into a pre-trained labeling model to obtain labeled electromyographic signals, and the method comprises the following steps:
Extracting signal characteristics of the electromyographic signals;
and inputting the joint moment and the signal characteristics of the electromyographic signals into a pre-trained labeling model to obtain labeled electromyographic signals.
Optionally, the advance labeling quantity is determined according to the angle signal corresponding to the target action and the ending time of the electromyographic signal.
In a second aspect, an embodiment of the present application provides a labeling device for a human body electromyographic signal, including:
The acquisition module is used for acquiring an angle signal and an electromyographic signal of the target joint;
The determining module is used for determining the joint moment of the target joint according to the angle signal;
The labeling module is used for labeling the electromyographic signals of the target joint according to the joint moment based on a predetermined advance labeling quantity, wherein the advance labeling quantity is the generation time difference between the joint moment and the electromyographic signals.
Optionally, the determining module is specifically configured to:
and inputting the angle signal into a pre-trained joint moment prediction model to obtain the joint moment corresponding to the target joint.
Optionally, the labeling module is specifically configured to:
and marking the electromyographic signals of the target joint according to the predetermined advance marking quantity and joint moment.
Optionally, the labeling module is specifically configured to:
The joint moment and the electromyographic signals are input into a pre-trained labeling model to obtain labeled electromyographic signals, the labeling model is obtained based on training of a training sample set, the training sample set comprises a plurality of training samples, each training sample comprises a sample electromyographic signal, a sample joint moment and labeling results of the sample electromyographic signals, and the labeling results of the sample electromyographic signals are determined according to the sample electromyographic signals, the sample joint moment and a pre-determined pre-labeling amount.
Optionally, the training method of the labeling model comprises the following steps:
Extracting signal characteristics of a sample electromyographic signal in each training sample;
training the neural network model to be trained according to the signal characteristics of the sample electromyographic signals in each training sample, the sample joint moment and the labeling results of the sample electromyographic signals to obtain a labeling model;
correspondingly, the joint moment and the electromyographic signals are input into a pre-trained labeling model to obtain labeled electromyographic signals, and the method comprises the following steps:
Extracting signal characteristics of the electromyographic signals;
and inputting the joint moment and the signal characteristics of the electromyographic signals into a pre-trained labeling model to obtain labeled electromyographic signals.
Optionally, the advance labeling quantity is determined according to the angle signal corresponding to the target action and the ending time of the electromyographic signal.
In a third aspect, an embodiment of the present application provides an electronic device, including: a memory and a processor, the memory for storing a computer program; the processor is configured to perform the method of the first aspect or any implementation of the first aspect when the computer program is invoked.
In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the method according to the first aspect or any implementation of the first aspect.
The labeling method of the human electromyographic signals provided by the embodiment of the application can acquire the kinetic signals and the electromyographic signals of the target joint, determine the joint moment of the target joint according to the kinetic signals, and finally label the electromyographic signals of the target joint according to the joint moment. Because the change of the joint moment is not influenced by individual differences and action differences, the joint moment is adopted to label the electromyographic signals, and the accuracy of the labeling result can be improved; moreover, any tested person can be selected, and the tested person can be allowed to do any action for any time, so that the application range is wide; in addition, repeated testing and manual labeling are not needed, so that the labeling efficiency can be improved. In addition, when marking is carried out, the electromyographic signals of the target joint can be marked according to the joint moment based on the predetermined advance marking quantity, so that the accuracy of marking results can be further improved.
Drawings
Fig. 1 is a flow chart of a labeling method of human body electromyographic signals provided by an embodiment of the application;
FIG. 2 is a flow chart of a method for determining joint moment according to an embodiment of the present application;
fig. 3 is a schematic diagram of electromyographic signal labeling according to an embodiment of the present application;
FIG. 4 is a schematic diagram of comparing the advance mark amount according to the embodiment of the present application;
fig. 5 is a schematic structural diagram of a labeling device for human myoelectric signals according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
At present, the marking of the electromyographic signals mainly judges the start and stop of each section of electromyographic signals according to personal experience, but the completion of one action needs a plurality of muscles to participate, the difference of the stress conditions of different muscles is large, and the time for producing the electromyographic signals by the muscles at different positions is different, so that the start and stop time corresponding to the action can be accurately found by carrying out multiple tests. On the other hand, because different testees have different habits in acting, the stress condition of muscles is also influenced, so in order to accurately mark the electromyographic signals, only one person is often used as the testee, and the actions in the collecting process are standard actions, so that the testee cannot act for a long time in order to reduce the difference between the actions, and the limitation is large.
In order to solve the above problems, the embodiments of the present application provide a method for labeling human myoelectric signals, and the following describes the technical solution of the present application in detail with specific embodiments. The following embodiments may be combined with each other, and some embodiments may not be repeated for the same or similar concepts or processes.
The labeling method of the human electromyographic signals provided by the embodiment of the application can be applied to electronic equipment such as notebook computers, tablet computers or mobile terminals, and the specific type of the electronic equipment is not limited in the embodiment of the application. The myoelectric signals in the embodiment of the application are all surface myoelectric signals.
Fig. 1 is a flow chart of a labeling method of human body electromyographic signals provided by an embodiment of the application, as shown in fig. 1, the method includes the following steps:
s110, acquiring a kinetic signal and an electromyographic signal of the target joint.
The electronic device may be obtained from various sensors placed on the body of the tested person, or may be input into the electronic device after the user collects the kinetic signals and the myoelectric signals of the target joint through the sensors, and in this embodiment, the kinetic signals and the myoelectric signals obtained from the sensors by the electronic device are exemplified. The dynamic signal may include an angle signal or a plantar pressure signal. The target joints may include hip joints, knee joints, ankle joints, and/or the like of the legs of the subject.
Specifically, before the dynamic signal and the electromyographic signal of the target joint are acquired, various sensors for testing can be placed on the body of the tested person, and the sensors can comprise an IMU sensor for acquiring an angle signal, a pressure sensor for acquiring a plantar pressure signal and an electromyographic sensor for acquiring the electromyographic signal. The IMU sensor can comprise a triaxial gyroscope, a triaxial accelerometer and a triaxial magnetometer, wherein the gyroscope can measure angular speed (namely the speed of rotation of an object), and the IMU sensor can multiply the speed and time to obtain the angle of rotation of the object in a certain time period; the accelerometer may measure acceleration of the object; the magnetometer may measure the yaw angle in the horizontal direction. The IMU sensor or the electronic device may fuse the three data (collectively referred to as IMU signals) in a complementary filtering or kalman filtering manner, so as to obtain angle signals of the target joint in three directions, that is, the angle signals may be determined based on the IMU signals.
For example, the electromyographic sensor can be fixed on the surfaces of rectus femoris, lateral rectus femoris, biceps femoris, gastrocnemius and tibialis anterior of the lower limb of the tested person so as to obtain electromyographic signals of each muscle of the lower limb; the IMU sensors can be placed on the thigh, the shank and the foot of the tested person to acquire the lower limb hip joint movement angle signals, the knee joint movement angle signals and the ankle joint movement angle signals of the tested person; the pressure sensor can also be placed on the sole of the tested person to acquire the sole pressure information of the tested person. After the myoelectric sensor, the pressure sensor and the IMU sensor are placed on the lower limb part of the tested person according to the requirements, corresponding actions such as walking, running, jumping and the like can be performed, and at the moment, the electronic equipment can acquire a large number of myoelectric signals of rectus femoris, lateral rectus femoris, biceps femoris, gastrocnemius and tibialis anterior, angle signals of thigh, calf and foot, and plantar pressure signals.
S120, determining the joint moment of the target joint according to the dynamics signals.
The electronic device can determine the joint moment of the target joint according to the dynamics signals, and specifically can determine the joint moment of the target joint by adopting the following methods:
first, a joint moment of a target joint is determined based on an inverse kinetic model.
Fig. 2 is a schematic flow chart of a method for determining joint moment according to an embodiment of the present application, as shown in fig. 2, the method includes the following steps:
s121, determining the joint angle of the target joint according to a predetermined gesture rotation matrix and an angle signal.
As mentioned above, the angle signal is acquired by the sensor, and the signal is the movement angle of the target joint in the inertial coordinate system; the joint angle of the target joint is the movement angle of the target joint under the joint coordinate system, the joint coordinate system can be established in advance by taking the target joint as a reference before the joint angle of the target joint is determined, and the gesture rotation matrix is determined according to the relative position relationship between the IMU sensor corresponding to the angle signal and the target joint. The gesture rotation matrix is a conversion matrix between an inertial coordinate system and a joint coordinate system.
For example, after the sensors are placed, a tester may set up a joint coordinate system of the hip joint, a joint coordinate system of the knee joint, and a joint coordinate system of the ankle joint with reference to the hip joint, the knee joint, and the ankle joint, respectively, and determine a posture rotation matrix corresponding to each target joint according to a relative positional relationship between the sensors and each target joint.
After the preparation is completed, the electronic device may determine the joint angle of the target joint according to the determined gesture rotation matrix and the angle signal.
S122, determining the joint angular velocity and the joint angular acceleration of the target joint according to the joint angle.
The electronic device can respectively perform first-order algebraic difference operation and second-order algebraic difference operation on the joint angle of the target joint to obtain the joint angular velocity and the joint angular acceleration of the target joint.
S123, determining the joint moment of the target joint based on the inverse dynamics model according to the plantar pressure signal, the joint angle, the joint angular velocity and the joint angular acceleration.
The joints of the human body can be regarded as rigid bodies, and the bones can be regarded as rod members, so that the lower limbs of the tested person can be regarded as a multi-link structure. Thus, the electronic device may employ Lagrangian or Newton's Euler method to build the inverse kinetic model. In the embodiment of the application, a Newton Euler method is taken as an example, and an inverse dynamic model is established.
The inverse kinetic model may be expressed by the following formula:
Wherein q represents the joint angle of the target joint, Represents the joint angular velocity of the target joint,The joint angular acceleration of the target joint is represented, a (q) represents the inertia matrix of the target joint,Representing offset moments, including centripetal and Ge forces, external forces and moments, a matrix of muscular moment arms, and gravitational moments, F representing plantar pressure signals, and T representing joint moment vectors of the target joint.
The electronic device can determine the joint moment of the target joint based on the inverse dynamics model according to the plantar pressure signal, the joint angle, the joint angular velocity and the joint angular acceleration.
Second, a joint moment of the target joint is determined based on the joint moment prediction model.
In this embodiment, the joint moment prediction model may be trained in advance according to a training sample of the joint moment prediction model. When the joint moment is determined, the electronic device can input the angle signal into a pre-trained joint moment prediction model to obtain the joint moment corresponding to the target joint.
Specifically, the training sample of the joint moment prediction model may include a sample angle signal and a sample joint moment determined according to the sample angle signal, where the sample joint moment may be obtained based on a method of an inverse kinetic model.
The training process of the joint moment prediction model may include: the electronic equipment can extract signal characteristics of sample angle signals in each training sample, and then train the neural network model to be trained according to the signal characteristics of the sample angle signals in each training sample and the sample joint moment to obtain a joint moment prediction model. The signal characteristics of the sample angle signals can be extracted by adopting a Fourier transform method, a wavelet transform method and the like; the neural network model may be a deep feed forward neural network, a recurrent neural network, a deep convolutional network, etc., which is not particularly limited in this embodiment.
When the joint moment is determined by using the pre-trained joint moment prediction model, the electronic device may not acquire the plantar pressure signal of the tested person in step S110.
In the embodiment of the application, the joint moment can be directly obtained according to the angle signal by adopting the joint moment prediction model electronic equipment, so that the processing step of the angle signal and the solving process according to the inverse dynamics model are omitted, and the whole electromyographic signal labeling work is more efficient.
S130, marking the electromyographic signals of the target joint according to the joint moment.
Because the joint moment is a representation of the forces between the joints, the joint moment will also change as the motion changes when the subject is doing the motion. Therefore, the electromyographic signals of the corresponding target joints can be marked by adopting the joint moment, and the starting and ending points of the electromyographic signals of the corresponding actions are determined based on the change condition of the joint moment.
For example, fig. 3 is a schematic illustration of electromyographic signal labeling provided in the embodiment of the present application, as shown in fig. 3, fig. 3 (a) is a curve of a joint moment of a knee joint, and fig. 3 (b) is a curve of an electromyographic signal of a knee joint. As shown in fig. 3 (b), when the subject changes from a standing posture to a squatting posture, an electromyographic signal generated by muscles near the knee joint changes; meanwhile, the stress condition of the knee joint of the tested person is inconsistent with the stress condition of the knee joint of the tested person when standing and squatting, so that the joint moment of the knee joint is changed when the tested person changes from the standing posture to the squatting posture as shown in (a) of fig. 3. Namely, the joint moment can reflect the change condition of human body actions, the association relation exists between the joint moment and the electromyographic signals, and the electromyographic signals corresponding to the joint moment marks can be adopted. As shown in fig. 3, the point a is a starting change point of the joint moment, the point B is an ending change point of the joint moment, the electromyographic signal at the corresponding moment of the point a is the point C, the electromyographic signal at the corresponding moment of the point B is the point D, that is, the electromyographic signal at the point C can be marked by the joint moment of the point a, the electromyographic signal at the point D can be marked by the joint moment of the point B, wherein the point a and the point B are starting points of the joint moment corresponding to the above action, and the point C and the point D are the starting points of the electromyographic signal corresponding to the above action.
According to research, the electromyographic signals are generated in the contraction and relaxation process of muscle movement, and the generation of the electromyographic signals is advanced by about 30-150 milliseconds compared with the force of the muscle. In order to improve accuracy of the labeling result, in the embodiment of the application, the electronic device can label the electromyographic signals of the target joint based on the predetermined advanced labeling quantity and the predetermined joint moment. The early labeling quantity is the generation time difference between the joint moment and the electromyographic signals.
Specifically, the joint moment is determined based on the angle signal, so the generation time difference between the joint moment and the electromyographic signal can be determined from the generation time difference between the angle signal and the electromyographic signal. In practical application, if the electronic equipment acquires an angle signal from the IMU sensor, the generation time difference between the joint moment and the electromyographic signal can be determined according to the generation time difference between the angle signal and the electromyographic signal; if the electronic device obtains the IMU signal from the IMU sensor, the generating time difference between the angle signal and the electromyographic signal can be determined according to the generating time difference between the IMU signal and the electromyographic signal, and then the generating time difference between the joint moment and the electromyographic signal is determined.
Furthermore, the angle signal and the electromyographic signal have a time difference in generating time, and the angle signal and the electromyographic signal have a corresponding time difference in ending time, so that the electronic device can also determine the advance marking quantity according to the ending time between the angle signal and the electromyographic signal corresponding to the target action.
Specifically, during the testing process, the tested person can stop for a period of time after each action is performed, and then the next action is performed. For example, fig. 4 is a schematic diagram of comparing the advance labeling amounts, as shown in fig. 4, where (a) in fig. 4 is a change curve of an angle signal of a knee joint, and (b) in fig. 4 is a change curve of an electromyographic signal of the knee joint. The subject may first take his or her left leg, then stop for 1 second, then take his or her right leg, and then stop for 1 second. After the action is stopped, the electromyographic signals acquired by the electromyographic sensors are ended firstly, and then the angle signals acquired by the IMU sensors are ended, so that the point E is the ending point of the angle signals when the testee takes the place of the left leg, the point F is the ending point of the angle signals when the testee takes the place of the right leg, the point G is the ending point of the electromyographic signals when the testee takes the place of the left leg, and the point H is the ending point of the electromyographic signals when the testee takes the place of the right leg. Therefore, the end time difference of the electromyographic signals and the angle signals can be determined according to the E point and the G point or according to the F point and the H point, and further the marking advance corresponding to the leg-taking action can be determined. In addition, in order to obtain the marking advance more accurately, the tested person can repeatedly do the same action for a plurality of times, and then the average value of the marking advances is taken as the marking advance, which is not described in detail herein.
After the advance labeling quantity is determined, the electronic equipment can label the electromyographic signals of the target joint according to the advance labeling quantity and the joint moment.
Specifically, the electronic device may determine a start-stop point of the electromyographic signal according to the advance labeling amount. For example, referring to fig. 3, point a is a start change point of the joint moment, point B is an end change point of the joint moment, the electronic device determines that the predetermined advance mark amount is 50 ms, the start point of the electromyographic signal is point C 'and the end point is point D'.
In order to be more convenient and quick, in the embodiment, a labeling model can be trained first, and then joint moment and electromyographic signals are input into the trained labeling model to obtain labeled electromyographic signals.
Specifically, the labeling model can be obtained based on training of a training sample set, the training sample set can comprise a plurality of training samples, each training sample can comprise a sample electromyographic signal, a sample joint moment and a labeling result of the sample electromyographic signal, and the labeling result of the sample electromyographic signal can be determined according to the sample electromyographic signal, the sample joint moment and a predetermined advance labeling amount.
The training process of the labeling model may include: the electronic equipment can extract the signal characteristics of the sample electromyographic signals in each training sample, and then train the neural network model to be trained according to the signal characteristics of the sample electromyographic signals in each training sample, the sample joint moment and the labeling results of the sample electromyographic signals to obtain a labeling model. The signal characteristics of the sample electromyographic signals can be extracted by adopting a Fourier transform method, a wavelet transform method and the like; the neural network model may be a deep feed forward neural network, a recurrent neural network, a deep convolutional network, etc., which is not particularly limited in this embodiment.
Correspondingly, when marking is carried out, the electronic equipment can firstly extract the signal characteristics of the electromyographic signals, and then input the joint moment and the signal characteristics of the electromyographic signals into a pre-trained marking model to obtain the marked electromyographic signals. The electronic equipment can directly obtain the electromyographic signals after marking according to the joint moment and the electromyographic signals by adopting the marking model, and the accuracy of marking results is ensured without acquiring the marking quantity in advance, so that the whole electromyographic signal marking work is more efficient.
The labeling method of the human electromyographic signals can acquire the kinetic signals and the electromyographic signals of the target joints, determine the joint moment of the target joints according to the kinetic signals, and finally label the electromyographic signals of the target joints according to the joint moment. Because the change of the joint moment is not influenced by individual differences and action differences, the joint moment is adopted to label the electromyographic signals, and the accuracy of the labeling result can be improved; moreover, any tested person can be selected, and the tested person can be allowed to do any action for any time, so that the application range is wide; in addition, repeated testing and manual labeling are not needed, so that the labeling efficiency can be improved.
In addition, when marking is carried out, the electromyographic signals of the target joint can be marked according to the joint moment based on the predetermined advance marking quantity, so that the accuracy of marking results can be further improved.
Based on the same inventive concept, as an implementation of the method, the embodiment of the present application provides a labeling device for human myoelectric signals, where the embodiment of the device corresponds to the embodiment of the method, and for convenience of reading, the embodiment of the device does not describe details in the embodiment of the method one by one, but it should be clear that the device in the embodiment can correspondingly implement all the details in the embodiment of the method.
Fig. 5 is a schematic structural diagram of a labeling device for human myoelectric signals according to an embodiment of the present application, where, as shown in fig. 5, the device provided in this embodiment includes:
An acquisition module 110 for acquiring a kinetic signal and an electromyographic signal of a target joint;
A determining module 120 for determining a joint moment of the target joint based on the kinetic signal;
The labeling module 130 is configured to label the electromyographic signals of the target joint according to the joint moment.
Optionally, the dynamics signal includes an angle signal and a plantar pressure signal, where the angle signal is a movement angle of the target joint in the inertial coordinate system, and the determining module 120 is specifically configured to:
Determining a joint angle of a target joint according to a predetermined gesture rotation matrix and an angle signal, wherein the gesture rotation matrix is a conversion matrix between an inertial coordinate system and a joint coordinate system, the joint coordinate system is a coordinate system established by taking the target joint as a reference, and the joint angle is a movement angle of the target joint under the joint coordinate system;
determining the joint angular velocity and the joint angular acceleration of the target joint according to the joint angle;
and determining the joint moment of the target joint based on the inverse dynamics model according to the plantar pressure signal, the joint angle, the joint angular velocity and the joint angular acceleration.
Alternatively, the inverse kinetic model is built based on newton's euler method or lagrangian method.
Alternatively, the inverse kinetic model is:
Wherein q represents the joint angle of the target joint, Represents the joint angular velocity of the target joint,The joint angular acceleration of the target joint is represented, a (q) represents the inertia matrix of the target joint,The offset moment is represented by F, the plantar pressure signal is represented by F, and the joint moment vector of the target joint is represented by T.
Optionally, the determining module 120 is specifically configured to:
and determining a posture rotation matrix according to the relative position relation between the sensor corresponding to the angle signal and the target joint.
Optionally, the angle signal is determined based on an inertial measurement unit IMU signal.
Optionally, the labeling module 130 is specifically configured to:
based on a predetermined advance labeling quantity, labeling the electromyographic signals of the target joint according to the joint moment, wherein the advance labeling quantity is the generation time difference between the joint moment and the electromyographic signals.
Optionally, the determining module 120 is specifically configured to:
and inputting the angle signal into a pre-trained joint moment prediction model to obtain the joint moment corresponding to the target joint.
Optionally, the labeling module 130 is specifically configured to:
and marking the electromyographic signals of the target joint according to the predetermined advance marking quantity and joint moment.
Optionally, the labeling module 130 is specifically configured to:
The joint moment and the electromyographic signals are input into a pre-trained labeling model to obtain labeled electromyographic signals, the labeling model is obtained based on training of a training sample set, the training sample set comprises a plurality of training samples, each training sample comprises a sample electromyographic signal, a sample joint moment and labeling results of the sample electromyographic signals, and the labeling results of the sample electromyographic signals are determined according to the sample electromyographic signals, the sample joint moment and a pre-determined pre-labeling amount.
Optionally, the training method of the labeling model comprises the following steps:
Extracting signal characteristics of a sample electromyographic signal in each training sample;
training the neural network model to be trained according to the signal characteristics of the sample electromyographic signals in each training sample, the sample joint moment and the labeling results of the sample electromyographic signals to obtain a labeling model;
correspondingly, the joint moment and the electromyographic signals are input into a pre-trained labeling model to obtain labeled electromyographic signals, and the method comprises the following steps:
Extracting signal characteristics of the electromyographic signals;
and inputting the joint moment and the signal characteristics of the electromyographic signals into a pre-trained labeling model to obtain labeled electromyographic signals.
Optionally, the advance labeling quantity is determined according to the angle signal corresponding to the target action and the ending time of the electromyographic signal.
The device provided in this embodiment may perform the above method embodiment, and its implementation principle is similar to that of the technical effect, and will not be described herein again.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
Based on the same inventive concept, the embodiment of the application also provides electronic equipment. Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application, as shown in fig. 6, where the electronic device provided in this embodiment includes: a memory 21 and a processor 20, the memory 21 for storing a computer program 23; the processor 20 is arranged to perform the method described in the method embodiments above when the computer program 23 is called.
The electronic device provided in this embodiment may execute the above method embodiment, and its implementation principle is similar to that of the technical effect, and will not be described herein again.
The embodiment of the application also provides a computer readable storage medium, on which a computer program is stored, which when being executed by a processor, implements the method described in the above method embodiment.
The integrated units described above, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application may implement all or part of the flow of the method of the above embodiments, and may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable storage medium may include at least: any entity or device capable of carrying computer program code to a photographing device/terminal apparatus, recording medium, computer Memory, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), electrical carrier signals, telecommunications signals, and software distribution media. Such as a U-disk, removable hard disk, magnetic or optical disk, etc. In some jurisdictions, computer readable media may not be electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/device and method may be implemented in other manners. For example, the apparatus/device embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.
It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in the present description and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
Furthermore, the terms "first," "second," "third," and the like in the description of the present specification and in the appended claims, are used for distinguishing between descriptions and not necessarily for indicating or implying a relative importance.
Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the application.
Claims (8)
1. The labeling method of the human body electromyographic signals is characterized by comprising the following steps of:
acquiring an angle signal and an electromyographic signal of a target joint;
Determining a joint moment of the target joint according to the angle signal;
The joint moment and the electromyographic signals are input into a pre-trained labeling model to obtain labeled electromyographic signals, the labeling model is obtained by training based on a training sample set, the training sample set comprises a plurality of training samples, each training sample comprises a sample electromyographic signal, a sample joint moment and a labeling result of the sample electromyographic signal, the labeling result of the sample electromyographic signal is determined according to the sample electromyographic signal, the sample joint moment and a pre-determined early labeling amount, and the early labeling amount is a generation time difference between the joint moment and the electromyographic signals.
2. The method of claim 1, wherein said determining the joint moment of the target joint from the angle signal comprises:
and inputting the angle signal into a pre-trained joint moment prediction model to obtain the joint moment corresponding to the target joint.
3. The method of claim 1, wherein the method of training the annotation model comprises:
Extracting signal characteristics of a sample electromyographic signal in each training sample;
training a neural network model to be trained according to signal characteristics of sample electromyographic signals in each training sample, sample joint moment and labeling results of the sample electromyographic signals to obtain a labeling model;
correspondingly, the step of inputting the joint moment and the electromyographic signals into a pre-trained labeling model to obtain labeled electromyographic signals comprises the following steps:
extracting signal characteristics of the electromyographic signals;
inputting the joint moment and the signal characteristics of the electromyographic signals into the pre-trained labeling model to obtain the labeled electromyographic signals.
4. A method according to any one of claims 1-3, wherein the advance mark is determined based on the angle signal corresponding to the target motion and the end time of the electromyographic signal.
5. A labeling device for human myoelectric signals, comprising:
The acquisition module is used for acquiring an angle signal and an electromyographic signal of the target joint;
The determining module is used for determining the joint moment of the target joint according to the angle signal;
the marking module is used for inputting the joint moment and the electromyographic signals into a pre-trained marking model to obtain marked electromyographic signals, the marking model is obtained by training based on a training sample set, the training sample set comprises a plurality of training samples, each training sample comprises a sample electromyographic signal, a sample joint moment and a marking result of the sample electromyographic signals, the marking result of the sample electromyographic signals is determined according to the sample electromyographic signals, the sample joint moment and a pre-determined early marking amount, and the early marking amount is a generation time difference between the joint moment and the electromyographic signals.
6. An electronic device, comprising: a memory and a processor, the memory for storing a computer program; the processor is configured to perform the method of any of claims 1-4 when the computer program is invoked.
7. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the method according to any of claims 1-4.
8. A chip system comprising a processor coupled to a memory, the processor executing a computer program stored in the memory to implement the method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010898778.4A CN112115813B (en) | 2020-08-31 | 2020-08-31 | Labeling method and device for human body electromyographic signals and computing equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010898778.4A CN112115813B (en) | 2020-08-31 | 2020-08-31 | Labeling method and device for human body electromyographic signals and computing equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112115813A CN112115813A (en) | 2020-12-22 |
| CN112115813B true CN112115813B (en) | 2024-08-13 |
Family
ID=73805644
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010898778.4A Active CN112115813B (en) | 2020-08-31 | 2020-08-31 | Labeling method and device for human body electromyographic signals and computing equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112115813B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114343649A (en) * | 2022-01-07 | 2022-04-15 | 中山大学附属第一医院 | Joint isometric contraction muscle force analysis method, equipment and computer readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106388819A (en) * | 2016-10-28 | 2017-02-15 | 许昌学院 | Human upper limb muscle state monitoring system based on surface electromyogram signals and judging method thereof |
| CN109276245A (en) * | 2018-11-01 | 2019-01-29 | 重庆中科云丛科技有限公司 | A kind of surface electromyogram signal characteristic processing and joint angles prediction technique and system |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080221487A1 (en) * | 2007-03-07 | 2008-09-11 | Motek Bv | Method for real time interactive visualization of muscle forces and joint torques in the human body |
| EP3487402B1 (en) * | 2016-07-25 | 2021-05-05 | Facebook Technologies, LLC | Methods and apparatus for inferring user intent based on neuromuscular signals |
| CN109559826A (en) * | 2019-01-30 | 2019-04-02 | 福州大学 | A kind of knee joint torque estimation method based on surface myoelectric and motor message |
| CN110555472B (en) * | 2019-08-27 | 2023-02-03 | 安徽心之声医疗科技有限公司 | Electrocardiosignal data labeling method based on active learning |
| CN110522457B (en) * | 2019-09-30 | 2022-03-15 | 南京伟思医疗科技股份有限公司 | Real-time gait analysis method based on motor current and sensor feedback |
-
2020
- 2020-08-31 CN CN202010898778.4A patent/CN112115813B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106388819A (en) * | 2016-10-28 | 2017-02-15 | 许昌学院 | Human upper limb muscle state monitoring system based on surface electromyogram signals and judging method thereof |
| CN109276245A (en) * | 2018-11-01 | 2019-01-29 | 重庆中科云丛科技有限公司 | A kind of surface electromyogram signal characteristic processing and joint angles prediction technique and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112115813A (en) | 2020-12-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9078585B2 (en) | Muscle synergy analysis method, muscle synergy analyzer, and muscle synergy interface | |
| Zhang et al. | Lower-limb joint torque prediction using LSTM neural networks and transfer learning | |
| CN108338791B (en) | Detection device and detection method for unsteady motion data | |
| CN108836344B (en) | Step length step frequency estimation method and device and gait detector | |
| CN109446972B (en) | Gait recognition model establishing method, recognition method and device based on electromyographic signals | |
| CN112115813B (en) | Labeling method and device for human body electromyographic signals and computing equipment | |
| Xiong et al. | Feature selection of input variables for intelligence joint moment prediction based on binary particle swarm optimization | |
| CN113515967B (en) | Motion intention recognition model generation method, device, equipment and storage medium | |
| CN110327054A (en) | A kind of gait analysis method and device based on acceleration and angular speed sensor | |
| Xia et al. | Predicting free achilles tendon strain from motion capture data using artificial intelligence | |
| CN112107290B (en) | System, method and software application for predicting KAM for multiple gait cycles of a subject | |
| Loose et al. | A Public Dataset of Overground and Treadmill Walking in Healthy Individuals Captured by Wearable IMU and sEMG Sensors. | |
| CN208591046U (en) | The detection device of unstable motion data | |
| JP6259301B2 (en) | Moving motion analysis apparatus, method and system, and program | |
| CN112115812A (en) | Human motion signal labeling method and device and computing equipment | |
| CN113208636B (en) | Method for processing stress analysis data of inner side and outer side of knee joint | |
| Yang et al. | Biomechanics analysis of human walking with load carriage | |
| Shanthini et al. | Interpretation of Gait Supervising Mechanism Using Sensor Integrated Makeshift and Analysing Pattern by K-Means Clustering Algorithm | |
| JP6486200B2 (en) | Mobile motion analysis apparatus, system, and program | |
| Pierleoni et al. | Activity Monitoring Through Wireless Sensor Networks Embedded Into Smart Sport Equipments: The Nordic Walking Training Utility | |
| Stetter | Wearable Sensors and Machine Learning based Human Movement Analysis–Applications in Sports and Medicine | |
| Muhamad et al. | Design and Implementation of Wearable IMU Sensor System for Heel-Strike and Toe-Off Gait Parameter Measurement | |
| Nilsson | Measuring Athletic Performance UsingAccelerometers | |
| CN116869521B (en) | Human body movement pattern real-time identification method of lower limb assistance exoskeleton system | |
| EP3915473B1 (en) | Information processing program, information processing method, and information processing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |