CN118592917A

CN118592917A - Heart rate variability prediction method and device based on millimeter wave radar signal

Info

Publication number: CN118592917A
Application number: CN202410801965.4A
Authority: CN
Inventors: 孙启彬; 王浩宇; 张冬
Original assignee: Hefei Zhongke Zhiqi Information Technology Co ltd
Current assignee: Hefei Zhongke Zhiqi Information Technology Co ltd
Priority date: 2024-06-20
Filing date: 2024-06-20
Publication date: 2024-09-06

Abstract

The invention provides a heart rate variability prediction method based on millimeter wave radar signals, which comprises the following steps: reading a plurality of reference signals related to heart beat movements of a heart part of a measured object from a data storage area, wherein the reference signals comprise three-dimensional spatial features and time features, the plurality of reference signals correspond to a plurality of spatial position points, the reference signals of each spatial position point correspond to a plurality of time windows, the plurality of reference signals are obtained by processing a plurality of original signals, and the plurality of original signals are: a plurality of millimeter wave radar echo signals reflected by a radiated area of a measured object; and predicting a plurality of reference signals by using a trained heart beat rhythm prediction model, and outputting a target prediction result for representing the heart beat rhythm of the tested object, wherein the heart beat rhythm prediction model comprises a space-time feature extraction basic network, a space feature compression network, a space feature reduction network, a time feature compression network, a space-time feature fusion network and a feature mapping network.

Description

Heart rate variability prediction method and device based on millimeter wave radar signal

Technical Field

The invention relates to the technical field of intelligent perception, in particular to a heart rate variability prediction method and device based on millimeter wave radar signals.

Background

Heart rate variability (HEART RATE Variability, HRV for short) refers to the natural fluctuations in the inter-heart beat time (i.e., RR interval). This fluctuation reflects the regulation of the heart by the autonomic nervous system, where the activity of the sympathetic and parasympathetic nerves affects the rate of heart rate. HRV is commonly used to assess the autonomic regulatory function and overall health of the heart and is associated with a variety of cardiovascular diseases. A normal HRV indicates that the heart is able to adapt to changes in the body's demands, while a decrease in HRV may be associated with various heart diseases and adverse health consequences. Therefore, accurate measurement of heart rate variability is particularly important.

In the process of implementing the inventive concept, the inventor finds that at least the following problems exist in the related art: the HRV monitoring equipment commonly used at present depends on contact type electrocardiograph equipment and has diagnosis and treatment defects of difficult daily monitoring, limited application scene, contact skin injury and the like. In addition, the existing heart rate variability prediction method based on non-contact only considers and analyzes single-point chest motion information, so that the actual spatial attribute which does not accord with an HRV monitoring task is processed, the signal to noise ratio of the acquired signal quality is unstable, and the accuracy of the rate variability prediction method is affected.

Disclosure of Invention

In view of the above, the present invention provides a heart rate variability prediction method based on millimeter wave radar signals, which is performed by an electronic device.

In a first aspect of the present invention, there is provided a heart rate variability prediction method based on millimeter wave radar signals, performed by an electronic device, comprising:

Reading a plurality of reference signals related to heart beat movements of a heart part of a measured object from a data storage area, wherein the reference signals comprise three-dimensional spatial features and time features, the plurality of reference signals correspond to a plurality of spatial position points, the reference signals of each spatial position point correspond to a plurality of time windows, the plurality of reference signals are obtained by processing a plurality of original signals, and the plurality of original signals are: a plurality of millimeter wave radar echo signals reflected by a radiated area of a measured object;

And predicting a plurality of reference signals by using a trained heart beat rhythm prediction model, and outputting a target prediction result for representing the heart beat rhythm of the tested object, wherein the heart beat rhythm prediction model comprises a space-time feature extraction basic network, a space feature compression network, a space feature reduction network, a time feature compression network, a space-time feature fusion network and a feature mapping network.

According to the embodiment of the invention, the prediction of the plurality of reference signals by using the heart beat rhythm prediction model comprises the steps of executing a plurality of prediction operations, wherein in each prediction operation, the heart beat rhythm prediction model outputs a group of local prediction results based on original input characteristics, and a plurality of groups of local prediction results corresponding to the plurality of prediction operations are taken together as target prediction results, wherein the characteristic dimensions of the original input characteristics comprise three-dimensional space characteristic dimensions, time characteristic dimensions and channel characteristic dimensions;

The kth one of the multiple prediction operations includes:

Selecting a part of reference signals from the plurality of reference signals as a Kth selection signal, and performing feature stitching on the Kth selection signal and a Kth-1 st local prediction result output by the heart beat rhythm prediction model to generate a Kth original input feature;

And inputting the Kth original input characteristic into a heart beat rhythm prediction model, and outputting a Kth local prediction result.

According to an embodiment of the present invention, feature stitching of the kth selection signal and the kth-1 st local prediction result output by the heart beat rhythm prediction model includes:

And performing feature stitching on the K-th selection signal based on the three-dimensional space feature dimension and the time feature dimension, and performing feature stitching on the K-1-th local prediction result based on the channel feature dimension to generate the K-th original input feature.

According to an embodiment of the present invention, inputting the kth original input feature into the heart beat rhythm prediction model, outputting the kth local prediction result includes:

Inputting the K-th original input features into a space-time feature extraction basic network to perform preliminary extraction of space-time features, and generating initial space-time features;

Isolating time features in the initial space-time features, and inputting the space features in the initial space-time features into a space feature compression network for feature compression to generate compressed space features;

inputting the compressed space features into a space feature restoration network for feature extraction and restoration to generate restored space features;

Inputting the time features in the initial space-time features into a time feature compression network to perform feature compression, and generating compressed time features;

Inputting the restored space features and the compressed time features into a space-time feature fusion network to perform feature fusion, so as to generate fusion space-time features;

and inputting the fused space-time characteristics into a characteristic mapping network to perform characteristic mapping, and outputting a K-th local prediction result.

According to an embodiment of the present invention, a spatio-temporal feature extraction base network includes:

The space feature extraction unit is used for extracting three-dimensional space features in the original input features;

the channel feature extraction unit is used for extracting channel features in the original input features;

and the time feature extraction unit is used for extracting the time features in the original input features.

According to an embodiment of the present invention, further comprising:

Transmitting millimeter wave radar transmitting signals to a measured object and receiving a plurality of millimeter wave radar echo signals reflected by a radiated area of the measured object;

Performing first signal processing on the millimeter wave radar echo signals to obtain a plurality of first-stage signals, wherein the first signal processing comprises: performing coherent accumulation operation on a plurality of millimeter wave radar echo signals defined by a plurality of channels formed by N transmitting antennas and M receiving antennas, wherein the signals in the first stage comprise three-dimensional space characteristics and time characteristics;

performing second signal processing on the plurality of first-stage signals to generate second-stage signals for representing the central position of the human body of the tested object, wherein the second-stage signals contain three-dimensional space features;

Taking a preset neighborhood range of the spatial position point of the second-stage signal as a target three-dimensional space region, and performing third signal processing on a plurality of millimeter wave radar echo signals in the target three-dimensional space region to obtain a plurality of third-stage signals related to the mechanical movement of the heart part of the measured object, wherein the third-stage signals comprise three-dimensional space features and time features;

and performing fourth signal processing on the third-stage signals to eliminate the influence of mechanical movement caused by the respiration of the tested object and obtain a plurality of reference signals related to the heart beat movement of the heart part of the tested object.

According to an embodiment of the present invention, performing second signal processing on a plurality of first-stage signals, generating a second-stage signal for characterizing a human body center position of a measured object includes:

Performing feature accumulation on signal values corresponding to each spatial position point in the plurality of first-stage signals based on a plurality of time windows to generate a plurality of time accumulation signals corresponding to the plurality of spatial position points one by one, wherein the time accumulation signals comprise three-dimensional spatial features;

Calculating a signal difference value between each time-integrated signal and a noise signal group, wherein the noise signal group is composed of a plurality of target signals in a preset three-dimensional neighborhood range of the time-integrated signal, and the plurality of target signals are not directly adjacent to the time-integrated signal;

a second stage signal for characterizing the human body center position of the object to be measured is determined from the plurality of time-integrated signals based on the signal difference value corresponding to each time-integrated signal.

According to an embodiment of the present invention, performing a coherent accumulation operation on a plurality of millimeter wave radar echo signals defined by a plurality of channels includes adopting the following method:

wherein: in the plurality of first-stage signals obtained after the coherent integration operation, the ith signal corresponding to the spatial position (x, y, z) and the signal acquisition period t is obtained;

N is the number of receiving antennas;

m is the number of transmitting antennas;

t is the total sampling point number in each frame period of the radar;

Is a sampling point within a period;

Is a plurality of millimeter wave radar echo signals corresponding to the first signal During acquisition period, the channels defined by the receiving antenna and the mth transmitting antennaIs a signal of (2);

Is the sampling period;

the change rate of the transmitting frequency of the millimeter wave radar transmitting signal;

Is the first Multiple transmitting antennas to target locationAnd return to the firstRound trip distance of the receiving antennas;

Is the speed of light;

Is the wavelength of the millimeter wave radar transmit signal.

According to an embodiment of the invention, wherein performing fourth signal processing on the plurality of third stage signals comprises:

Performing second order differential processing on the plurality of third stage signals to eliminate an influence of mechanical motion caused by breathing of the subject, wherein performing second order differential processing on the plurality of third stage signals includes:

wherein:

Is a third stage signal;

For the ith Performing second-order differential processing to obtain an ith reference signal; h is the period of each second stage signal;

Is that A third stage signal of the nth sample, n=1, 2, 3;

Is that A third phase signal sampled by the nth sample;

is the value of i samples apart in the time series.

In a second aspect of the present invention, there is provided a heart rate variability predicting device based on millimeter wave radar signals, comprising:

The reading module is used for reading a plurality of reference signals related to heart beat movements of a heart part of a measured object from the data storage area, wherein the reference signals comprise three-dimensional space features and time features, the plurality of reference signals correspond to a plurality of space position points, the reference signals of each space position point correspond to a plurality of time windows, the plurality of reference signals are obtained by processing a plurality of original signals, and the plurality of original signals are: a plurality of millimeter wave radar echo signals reflected by a radiated area of a measured object;

The prediction module is used for predicting a plurality of reference signals by using a trained heart beat rhythm prediction model and outputting a target prediction result for representing the heart beat rhythm of the tested object, wherein the heart beat rhythm prediction model comprises a space-time feature extraction basic network, a space feature compression network, a space feature reduction network, a time feature compression network, a space-time feature fusion network and a feature mapping network.

A third aspect of the present invention provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method.

A fourth aspect of the invention also provides a computer readable storage medium having stored thereon a computer program or instructions which when executed by a processor performs the steps of the above method.

The fifth aspect of the invention also provides a computer program product comprising a computer program or instructions which, when executed by a processor, carries out the steps of the method described above.

According to an embodiment of the invention, the plurality of reference signals stored in the data storage area are derived from an original signal processing, wherein, in a signal processing stage, the human body position is determined in a spatial range by performing coarse-grained filtering processing on the first stage signal. Then, the signal is further subjected to fine-granularity spatial positioning in a preset neighborhood range of the spatial position point of the signal in the second stage so as to more accurately acquire the signal condition of the surrounding area of the human body, and as can be seen, the input signal is subjected to two times of spatial filtering, firstly, the coarse-granularity spatial positioning is performed to determine the position of the human body, and then, the fine-granularity spatial positioning is performed, so that the signal condition of the surrounding area of the human body can be acquired more accurately, and the signal to noise ratio of the input signal is improved.

According to the embodiment of the invention, a plurality of reference signals stored in a data storage area are read, a space-time relation between a body reflection signal and a heartbeat is established according to a pre-trained heartbeat rhythm prediction model, and the mapping from the mechanical heart activity to the electrical heart activity is completed through the extracted mechanical heart activity signal and the heartbeat rhythm prediction model to obtain the electrocardiographic sign prediction information, specifically, the heartbeat waveform is obtained.

Specifically, the original input features are input into the space-time feature extraction basic network through the space-time feature extraction basic network, and the time, space and channel features are extracted through the three networks respectively, so that various abstract features in the original signals can be extracted more accurately, and heartbeat information is extracted accurately, so that the accuracy of a prediction result is improved; the spatial characteristics are compressed through the spatial characteristic compression network, so that the calculation amount surge caused by the increase of the dimension is avoided; the compressed spatial characteristics are restored through the spatial characteristic restoring network, so that the damage of the common compression step to the characteristics is prevented, and the loss of important characteristics is avoided; in the characteristic compression process, the spatial characteristic and the temporal characteristic are isolated and processed independently, so that the targeted compression processing based on the respective characteristic types of the multidimensional characteristic is realized, different compression strategies are adopted, and the characteristic precision is ensured on the basis of reducing the complexity of the characteristic processing. Compressing the time characteristics through a time characteristic compression network, and reducing the storage and calculation amount; and realizing fusion of time features and space features through a space-time feature fusion network, and mapping through a feature mapping network to obtain heartbeat waveforms.

According to the embodiment of the invention, based on each network in the heart beat rhythm prediction model, the heart beat rhythm prediction model integrally realizes the mechanical activity sensing of human chest heart jogging based on the millimeter wave radar, and realizes the mapping of the time-space relation between the reflected signal and the heart beat. Meanwhile, the spatial distribution of the whole chest surface and the possible difference of signals at different positions are considered, the mapping of various indexes from mechanical activity to electrical activity of the chest heart of a human body is completed, the heart rate variability prediction with non-contact fine granularity is realized, the automatic mapping processing of artificial intelligence is realized, the prediction result is provided for a user to refer, a new information dimension is provided, and the user experience is improved.

According to the embodiment of the invention, the method adopts the three-dimensional space signal information with fine granularity to acquire and monitor the Heart Rate Variability (HRV) for the first time, and compared with the traditional single-point signal processing method and the prior one-dimensional (distance) or two-dimensional (distance and angle) neural network input, the method considers more and more accurate information near the thoracic cavity, and realizes non-contact fine-granularity heart rate variability prediction.

On the other hand, the non-contact detection is performed based on millimeter wave radar signals, so that discomfort caused by wearing electrodes and sensing equipment can be avoided, cross infection can be avoided, and compared with a traditional contact type measurement method, the user experience and the application range are improved to a large extent (for example, the traditional measurement method cannot be implemented due to poor physical health conditions and cannot be directly contacted with a body part, and the method is not limited by the method).

Drawings

The foregoing and other objects, features and advantages of the invention will be apparent from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

fig. 1 schematically shows a flow chart of a heart rate variability prediction method based on millimeter wave radar signals according to an embodiment of the present invention;

fig. 2 schematically shows an external view of a millimeter wave radar according to an embodiment of the present invention;

Fig. 3 schematically shows a flow chart of a method of acquiring a reference signal according to an embodiment of the invention;

FIG. 4 schematically illustrates a network architecture diagram of a heart beat rhythm prediction model according to an embodiment of the invention;

FIG. 5 schematically illustrates a scenario in which a subject is tested using a heart rate variability prediction method according to an embodiment of the present invention;

FIG. 6 schematically shows an external view of an electrocardiographic monitoring device according to an embodiment of the present invention;

FIG. 7 schematically illustrates a heartbeat waveform contrast plot of an embodiment of the present invention with a true solid electrical monitoring device;

FIG. 8 schematically illustrates an IBI value versus graph for an embodiment of the present invention versus a true solid electrical monitoring device;

Fig. 9 schematically shows a block diagram of a heart rate variability predicting device according to an embodiment of the present invention.

Detailed Description

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. It should be understood that the description is only illustrative and is not intended to limit the scope of the invention. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the invention. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the present invention.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.

Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a convention should be interpreted in accordance with the meaning of one of skill in the art having generally understood the convention (e.g., "a system having at least one of A, B and C" would include, but not be limited to, systems having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).

Radio frequency signal monitoring is a non-contact monitoring technique that indirectly measures heart rate and heart rate variability by capturing and analyzing small electromagnetic wave changes generated by heart activity wirelessly. In practice, however, the surface of the chest cavity of the human body is not a single point, but a spatially distributed area. This means that the signals generated by the heart activity will exhibit different characteristics at different locations on the chest surface. The prior art only analyzes chest motion information at one specific point (single point) and does not consider the spatial distribution across the chest surface. This approach ignores the possible differences in the signal at different locations. In addition, because only single-point information is analyzed, the acquired signal may contain more noise, so that the signal-to-noise ratio of the signal is unstable, and the accuracy and reliability of HRV monitoring are affected. Therefore, in order to accurately monitor HRV, the reflected signal of the entire body surface needs to be considered.

The radar sensor detects human body movement by emitting radio waves and receiving reflected wave signals. However, radar signals are reflected from the entire torso surface of the human body (e.g., the chest and abdomen), which makes it difficult for radar sensors to separate only useful heartbeat signals from body surface motion caused by heart activity. That is, in addition to body surface movements caused by heart beating, movements of other body parts may also generate reflected signals, which may lead to confusion of monitoring results. And the radio signal is subject to interference from various factors during the propagation process, such as environmental noise, signal interference of other electronic devices, and the like. These noise and interference can affect the quality of the signal, making it difficult to accurately extract heartbeat information from the radar signal.

Based on the method, the heart rate variability prediction method based on the millimeter wave radar signals, which is executed by the electronic equipment, is provided, more comprehensive physiological information is obtained by considering reflection signals from all parts of a body, and further, a space-time relation model between the body reflection signals and heartbeats is established by a deep learning network model so as to predict the non-contact HRV.

Fig. 1 schematically shows a flow chart of a heart rate variability prediction method based on millimeter wave radar signals according to an embodiment of the invention. Fig. 2 schematically shows an external view of a millimeter wave radar according to an embodiment of the present invention. This is explained in detail below in connection with fig. 1 and 2.

As shown in FIG. 1, the heart rate variability prediction method includes operations S110-S120.

In operation S110, a plurality of reference signals related to cardiac exercise of a heart portion of the measured object are read from the data storage area, wherein the reference signals include three-dimensional spatial features and temporal features, the plurality of reference signals correspond to a plurality of spatial location points, the reference signal of each spatial location point corresponds to a plurality of time windows, the plurality of reference signals are obtained by processing a plurality of original signals, and the plurality of original signals are: a plurality of millimeter wave radar echo signals reflected by a radiated area of the object under test.

According to an embodiment of the present invention, as shown in fig. 2, the millimeter wave radar may be a multi-transmitting multi-receiving device, including N transmitting antennas and M receiving antennas. Such as 3 transmit antennas and 4 receive antennas. The specific signal acquisition method can be, for example, that a human body lies on a bed, and millimeter wave radar shown in fig. 2 is utilized to transmit millimeter wave signals to the chest position of the human body and receive echo signals.

According to the embodiment of the invention, millimeter wave signals are transmitted to a measured object by utilizing millimeter wave radars. After the millimeter wave signal is received by the measured object, the received millimeter wave signal can be reflected by the radiation area of the measured object, so that a plurality of original signals are obtained. Wherein the radiation area may be a radiation range of a millimeter wave radar. Including but not limited to, for example, the chest region of the subject. Further, the plurality of original signals include a millimeter wave radar echo signal generated by the detection of a target signal, namely, the heartbeat motion, and are doped with various interference signals, such as a millimeter wave radar echo signal generated by respiration, a millimeter wave radar echo signal generated by other radiation areas except for the heart part, and the like. Therefore, a plurality of original signals need to be processed to obtain a plurality of reference signals used for representing the heart beat motion rule of the heart part of the tested object.

According to an embodiment of the invention, the plurality of processed reference signals are stored in a data storage area, and the relevant reference signals are read directly from the data storage area when the heart rate variability prediction method is applied. Each reference signal comprises a three-dimensional space feature and a time feature, a plurality of reference signals correspond to a plurality of space position points, and the reference signal of each space position point corresponds to a plurality of time windows.

In operation S120, a plurality of reference signals are predicted by using a trained heart beat rhythm prediction model, and a target prediction result for representing a heart beat rhythm of a measured object is output, where the heart beat rhythm prediction model includes a space-time feature extraction base network, a space feature compression network, a space feature restoration network, a time feature compression network, a space-time feature fusion network, and a feature mapping network.

According to an embodiment of the present invention, the target prediction result for characterizing the heart beat rhythm may reflect the heart rate of the measured subject, i.e. the number of heart beats per minute. The heart rate of the tested object and the regularity thereof indicate whether the heart beat is in the normal rate range and whether the heart beat is uniform.

It should be noted that, the heart rhythm prediction result output by the heart rhythm prediction model has a similar information characterization function to the conventional heart rhythm variability detection result, and the prediction result is only the reference information for diagnosis by medical staff, so as to obtain an intermediate result of processing information parameters and is not a direct diagnosis result. In addition, all the steps of the method are information processing methods implemented by devices such as a computer, the whole process does not need participation of medical staff, the direct purpose is not to obtain a diagnosis result or a health condition, and the obtained information can not directly obtain the diagnosis result or the health condition of the disease.

Fig. 3 schematically shows a flow chart of a method of acquiring a reference signal according to an embodiment of the invention.

As shown in fig. 3, obtaining a plurality of reference signals for characterizing a heart beat motion rule of a heart portion of a measured object includes operations S310 to S350.

In operation S310, a millimeter wave radar transmission signal is transmitted to a measured object, and a plurality of millimeter wave radar echo signals reflected by a radiated area of the measured object are received.

According to the embodiment of the invention, the virtual antenna array surface can be constructed on the basis of the physical arrangement of the millimeter wave radar antennas, the phase shift vector is constructed according to the antenna spacing and the signal bandwidth in the virtual antenna array surface, the space beam forming is calculated, and the airspace filtering of the millimeter wave radar echo signals is completed. The propagation distances of reflected signals at different positions of the chest can be mapped into phase changes by utilizing an array antenna of the radio frequency equipment and a frequency modulation continuous wave system, and finally, the reflected signals at all positions in the whole chest are restored by performing coherent accumulation operation on the received signals in a virtual channel formed by different transmitting and receiving antennas.

In operation S320, a plurality of millimeter wave radar echo signals are subjected to a first signal processing to obtain a plurality of first stage signals.

According to the embodiment of the invention, after receiving the echo of the radar transmitting signal, the first signal processing is used for carrying out preliminary coarse-granularity spatial domain filtering, so that a plurality of first-stage signals used for representing the body surface mechanical vibration rule of the radiated area of the measured object can be obtained, and the signals from the non-target area are removed or suppressed, so that the quality of the signals is improved, and the subsequent signal analysis is more accurate. Wherein the first signal processing includes: and performing coherent accumulation operation on a plurality of millimeter wave radar echo signals defined by a plurality of channels formed by N transmitting antennas and M receiving antennas, wherein the signals in the first stage comprise three-dimensional space characteristics and time characteristics.

According to an embodiment of the present invention, coherent accumulation operation is performed on a plurality of millimeter wave radar echo signals defined by a plurality of channels as shown in formula (1):

（1）

N is the number of receiving antennas;

m is the number of transmitting antennas;

t is the total sampling point number in each frame period of the radar;

Is a sampling point within a period;

j is an imaginary representation in mathematics;

Is the sampling period;

Is the speed of light;

Is the wavelength of the millimeter wave radar transmit signal.

In operation S330, a plurality of first-stage signals are subjected to a second signal processing to generate a second-stage signal for characterizing a human body center position of the measured object. The second-stage signal includes three-dimensional features, and specifically includes steps 11 to 13.

In step 11, the signal values corresponding to each spatial location point in the plurality of first-stage signals are subjected to feature accumulation based on a plurality of time windows, so as to generate a plurality of time accumulation signals corresponding to the plurality of spatial location points one by one, wherein the time accumulation signals comprise three-dimensional spatial features. Here, the feature accumulation is performed based on a plurality of time windows, so that the generated second-stage signal only includes three-dimensional spatial features and does not include time features.

In step 12, a signal difference value between each time-integrated signal and a noise signal group is calculated, wherein the noise signal group is composed of a plurality of target signals within a predetermined three-dimensional neighborhood of the time-integrated signal, the plurality of target signals not being directly adjacent to the time-integrated signal. The signals which are not directly adjacent to the time accumulation signals are treated as noise signals, so that the noise signals can be reasonably distinguished, and the calculation accuracy is convenient to improve.

In step 13, a second-stage signal for characterizing the human body center position of the object to be measured is determined from the plurality of time-integrated signals based on the signal difference value corresponding to each time-integrated signal.

According to the embodiment of the invention, the accurate position of the chest of the human body is adaptively detected by adopting a three-dimensional unit average-constant false positive rate (CA-CFAR) algorithm. Firstly, based on a plurality of time windows, adding signal amplitudes in a time dimension to enhance signals and reduce the influence of noise, so that the accuracy of subsequent signal extraction is improved; and estimating the noise level of each point by the difference between the signal amplitude in the three-dimensional space and the noise signals in the surrounding neighborhood. Wherein the predetermined three-dimensional neighborhood range may be empirically set, the larger the signal difference value between the time-integrated signal and the noise signal group, the higher the likelihood of possibly belonging to the human body position. The detection threshold is adjusted based on the estimated noise level to determine when the signal amplitude exceeds background noise to determine the location where the object is most likely to be the rough location of the human body.

In operation S340, a predetermined neighborhood range of the spatial position point of the second-stage signal is used as the target three-dimensional space region, and a plurality of millimeter wave radar echo signals in the target three-dimensional space region range are subjected to a third signal processing, so as to obtain a plurality of third-stage signals related to the mechanical movement of the heart part of the measured object, wherein the third-stage signals comprise three-dimensional space features and time features. The third signal processing is used for searching the accurate position (x, y, z) where the human body is located based on the second-stage signal. The method specifically comprises the steps of processing an original signal by adopting the method of the formula (1) to obtain the spatial information of accurate positioning. In the above operation, the plurality of time-integrated signals obtained by performing the feature accumulation based on the plurality of time windows are used to determine the approximate range of the position (x, y, z) where the human body is located. The method for processing the signals comprises the following steps: the raw signal is first processed by equation (1) with coarse spatial position x, y, z values to obtain coarse positioning data. And then adding the time dimension of the amplitude value of the coarse positioning data, and only retaining the superposition information on the coarse positioning data space. And then, according to superposition information on the coarse positioning data space, searching accurate x, y and z values of the human body, and processing the original signal by using a formula (1) to obtain the accurate positioning space information.

The predetermined neighborhood range may be a space range for characterizing the heart position, which is empirically set, for example, the predetermined neighborhood range is 1.5x1.5x1.5 m (for coarse positioning), and further the predetermined neighborhood range is 0.5m x 0.5m x 0.5m (for accurate positioning). The third phase signal contains both three-dimensional and temporal features.

According to the embodiment of the invention, the human body position is determined in a space range by carrying out coarse-granularity filtering processing on the first-stage signals. Then, the signal is further subjected to fine-granularity spatial positioning in a preset neighborhood range of the spatial position point of the signal in the second stage so as to more accurately acquire the signal condition of the surrounding area of the human body, and as can be seen, the input signal is subjected to two times of spatial filtering, firstly, the coarse-granularity spatial positioning is performed to determine the position of the human body, and then, the fine-granularity spatial positioning is performed, so that the signal condition of the surrounding area of the human body can be acquired more accurately, and the signal to noise ratio of the input signal is improved.

In operation S350, a fourth signal processing is performed on the plurality of third-stage signals to eliminate the influence of the mechanical motion caused by the respiration of the subject, and a plurality of reference signals related to the heart beat motion of the heart portion of the subject are obtained.

According to the embodiment of the present invention, the non-heart portion in the irradiated region also has the body surface vibration, and therefore the fourth signal processing is required for the plurality of third-stage signals to eliminate the influence of the body surface vibration of the non-heart portion in the irradiated region.

According to an embodiment of the present invention, performing fourth signal processing on a plurality of third-stage signals includes: performing a second order differential processing on the plurality of third-stage signals to eliminate the influence of mechanical motion caused by breathing of the subject, wherein the plurality of third-stage signals are subjected to the second order differential processing, specifically referring to formula (2):

（2）

Wherein, Is a third stage signal;

Is that A third stage signal of the nth sample, n=1, 2, 3;

Is that A third phase signal sampled by the nth sample;

is the value of i samples apart in the time series.

According to the embodiment of the invention, the influence of body surface vibration caused by respiration and body surface vibration of a non-heart part in the millimeter wave radar echo signal is eliminated, so that a plurality of finally obtained reference signals can more accurately represent the law of heart beat motion of the heart part of the measured object.

According to the embodiment of the invention, the Heart Rate Variability (HRV) is obtained and monitored by adopting the three-dimensional space signal information with fine granularity, and compared with the traditional single-point signal processing method and the one-dimensional (distance) or two-dimensional (distance and angle) neural network input, the prediction method of the embodiment of the invention comprises more and more accurate information near the thoracic cavity.

Further, after obtaining a plurality of reference signals, the reference signals can be stored in a data storage area for subsequent use or directly input into a heart beat rhythm prediction model to obtain a target prediction result for representing the heart beat rhythm of the tested object.

Fig. 4 schematically shows a network structure diagram of a heart beat rhythm prediction model according to an embodiment of the invention.

As shown in fig. 4, the heart beat rhythm prediction model includes a spatio-temporal feature extraction base network (Basic Blocks network), a spatial feature compression network (compact 1 network), a spatial feature restoration network (transformer encoder 1), a temporal feature compression network (compact 2 network), a spatio-temporal feature fusion network (transformer encoder 2), and a feature mapping network (MLP).

According to an embodiment of the present invention, predicting a plurality of reference signals using a heart beat rhythm prediction model includes performing a plurality of prediction operations in each of which the heart beat rhythm prediction model outputs a set of local prediction results based on an original input characteristic signal, and a plurality of sets of local prediction results corresponding to the plurality of prediction operations are collectively used as a target prediction result. For example, a set of original input characteristic signals with the duration of 10s is input, a set of local prediction results with the duration of 10s is output, and under the condition that the number of prediction operations is 5, the local prediction results with the duration of 10s obtained by 5 times are spliced to obtain a target prediction result.

According to the embodiment of the present invention, the number of prediction operations may be empirically set, as shown in fig. 4, for example, the number of prediction operations is set to 3, then 3 original input features are required to be input, 3 sets of local prediction results are obtained, and 3 sets of local prediction results corresponding to the 3 prediction operations are taken together as target prediction results. The feature dimensions of the original input features include, among others, three dimensional spatial feature dimensions (e.g., denoted by H, W, L), temporal feature dimensions (e.g., denoted by T), and channel feature dimensions (e.g., the value "2" in fig. 3).

Specifically, the reference signal is input into the heart beat rhythm prediction model, wherein the application process of the prediction model is explained by taking the kth prediction operation of the multiple prediction operations as an example, and specifically comprises the steps 11 and 12.

In step 11, a part of reference signals are selected from the plurality of reference signals to be used as a Kth selection signal, and the Kth selection signal and the Kth-1 st local prediction result output by the heart beat rhythm prediction model are subjected to feature stitching to generate a Kth original input feature.

According to an embodiment of the present invention, K is a positive integer of 1 or more. A portion of the reference signals is selected from the plurality of reference signals as a K-th selection signal. The method includes the steps of obtaining a reference signal with a period of time of 0-10 s, selecting 6s as a selection signal, and inputting a heart beat rhythm prediction model.

For example, when K is equal to 1, i.e. the first time a selection signal is input to the heart beat rhythm prediction model, the local prediction result of the K-1 th time cannot be obtained, i.e. the local prediction result of the K-1 th time is set to 0, and the first time original input feature is generated.

When K is equal to 2, namely, inputting the selection signal to the heart beat rhythm prediction model for the second time, obtaining a first (last) local prediction result, namely, a first original input characteristic, and performing characteristic splicing with the selection signal to be input for the second time to jointly serve as a second input to generate a second original input characteristic.

According to an embodiment of the present invention, in step 11, performing feature stitching on the kth selection signal and the kth-1 st local prediction result output by the heart beat rhythm prediction model includes:

In step 12, the kth original input feature is input into the heart beat rhythm prediction model, and the kth local prediction result is output, including steps 121-126.

In step 121, the K-th original input feature is input into the space-time feature extraction base network to perform preliminary extraction of the space-time feature, and an initial space-time feature is generated.

As shown in fig. 4, the spatio-temporal feature extraction base network is a Basic Blocks network (Basic Blocks network) including a spatial feature extraction unit, a channel feature extraction unit, and a temporal feature extraction unit. The spatial feature extraction unit is used for extracting three-dimensional spatial features in the original input features, and can be a 3D CNN model; a channel feature extraction unit for extracting channel features in the original input features, for example, extracting channel information using bottleneck-shaped two-layer full connection; and the time feature extraction unit is used for extracting time features in the original input features, for example, extracting time information by using bottleneck-shaped two-layer full connection. Further, through the space-time feature extraction basic network, three-dimensional space features, channel features and time features in the selection signals can be respectively extracted to obtain initial space-time features.

According to the embodiment of the invention, in step 121, the spatial features, the temporal features and the channel features are separated, and the three networks are utilized to extract the temporal features, the spatial features and the channel features respectively, and by processing the feature separation, various abstract features in the original signal can be extracted more accurately, so that the method can be suitable for processing data with large distribution difference in various complex scenes, for example, heartbeat information can be extracted accurately, and the accuracy of the prediction result can be improved.

In step 122, the temporal features in the initial temporal-spatial features are isolated, and the spatial features in the initial temporal-spatial features are input into a spatial feature compression network for feature compression, resulting in compressed spatial features.

As shown in fig. 3, the spatial feature compression network is a compact 1 network, the time features in the initial space-time features are put into batch processing for isolation, and meanwhile, only the spatial features are input into the spatial feature compression network (compact 1 network) for feature compression, so as to generate compressed spatial features.

In step 123, the compressed spatial features are input into a spatial feature restoration network for feature extraction and restoration, and restored spatial features are generated. As shown in fig. 3, the spatial feature recovery network is a transducer encoder1, and for example, 6 layers Transformer encoder are used to extract compressed spatial information.

According to the embodiment of the invention, in the steps 122-123, the spatial features are compressed, so that the calculation amount is prevented from increasing rapidly due to the increase of dimensions, the compressed spatial features are restored, the damage to the features caused by the common compression step is prevented, and the important features are prevented from being lost.

In step 124, the temporal features in the initial spatio-temporal features are input into a temporal feature compression network for feature compression, generating compressed temporal features. As shown in fig. 3, the time feature compression network is a compact 2 network. The time dimension may be compressed to a size of 32 for the subsequent spatio-temporal mixing module to extract the spatio-temporal features.

In step 125, the restored spatial features and the compressed time features are input into a spatio-temporal feature fusion network for feature fusion, resulting in fused spatio-temporal features. As shown in fig. 3, the spatio-temporal feature fusion network is a transducer encoder2, for example, using 6 layers Transformer encoder2 to perform feature fusion on the restored spatial features and the compressed temporal features. The space-time feature fusion network comprises a common space-time feature extraction unit and a global space-time feature extraction unit, wherein the common space-time feature extraction unit comprises space-time information extracted from a space-time feature extraction basic network; the global space-time feature extraction unit is used for compressing and extracting the features related to the HRV monitoring task from all the common space-time feature extraction units.

In step 126, the fused spatio-temporal features are input into a feature mapping network for feature mapping, and the kth local prediction result is output. As shown in fig. 3, the feature mapping network is a multi-layer perceptron (Multilayer Perceptron, MLP). Two-layer multi-layer perceptrons are used to map the extracted spatiotemporal features to heart activity. In addition, there will be 2 seconds of history data spliced to the signal output after zero padding repetition, forming the input signal to the network.

And further, splicing according to the local prediction results obtained multiple times, and jointly using the local prediction results as target prediction results.

According to the embodiment of the invention, based on a deep learning architecture with the mapping of the time-space relation between the learned radar signal and the heart activity, the prediction result of the heart rhythm activity of the human body is output by inputting the extracted thoracoabdominal radar heart micro-motion data, so that the heart rhythm prediction is realized through the jump rhythm prediction model.

According to the embodiment of the invention, in the characteristic compression process, the spatial characteristic and the temporal characteristic are isolated and processed independently (the temporal characteristic is isolated in the spatial characteristic compression process and the spatial characteristic is isolated in the temporal characteristic compression process), so that the targeted compression processing based on the respective characteristic types of the multidimensional characteristic is realized, different compression strategies are adopted, and the characteristic precision is ensured on the basis of reducing the complexity of the characteristic processing.

The invention is further illustrated by the following examples and related test experiments. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the invention. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, the details of the various embodiments below may be arbitrarily combined into other viable embodiments without conflict.

Fig. 5 schematically shows a schematic view of a scenario in which a subject is tested using a heart rate variability prediction method according to an embodiment of the present invention.

As shown in fig. 5, the subject lies on the bed, and the millimeter wave radar emits and collects signals at a distance of 0.4-0.5m right above the chest of the human body. In the experiment, the radar is in a 3-transmission 4-reception working state, the initial frequency of a set signal is 77GHz, the bandwidth is 4GHz, and the frame rate is 100Hz. And acquiring signals of 30s, and outputting the heart rate sign prediction information by adopting the heart rate variability prediction method provided by the embodiment of the invention.

Fig. 6 schematically shows an external view of an electrocardiographic monitoring device according to an embodiment of the present invention. Heart rate variability may be acquired using the apparatus shown in fig. 6, and in particular, the schematic view of the electrocardiographic detection by the electrocardiographic monitoring apparatus shown in fig. 6 is the same as that of the prior art, and is not shown in the figure. An electrocardiogram of the subject was acquired by an electrocardiographic monitoring device as shown in fig. 6, with an electrocardiographic device output of 500HZ. In the experiment, subjects were lying on bed for 30s of data acquisition.

Fig. 7 schematically illustrates a heartbeat waveform comparison of an embodiment of the present invention with a truly solid electrical monitoring device. The heartbeat waveform generated by using the heart rate variability prediction method in the embodiment of the present invention is shown as a blue line in fig. 7. The heartbeat waveform generated by employing the conventional electrocardiographic monitoring device shown in fig. 6 is shown as an orange line in fig. 7. And, black dashed lines (R-waves) are used to characterize systole.

FIG. 8 schematically illustrates a graph of IBI values versus a true solid electrical monitoring device for an embodiment of the present invention. The Interval between heartbeats (IBI) refers to the time Interval between two successive heartbeats. By calculating the time difference between successive heart beats, the value of IBI can be derived. The value of IBI obtained by calculation of the heartbeat waveform generated by the heart rate variability prediction method in the embodiment of the present invention is shown as a blue line in fig. 8. The values of IBI calculated by using the heartbeat waveform generated by the conventional electrocardiographic monitoring device as shown in fig. 6 are shown as black lines in fig. 8.

As shown in fig. 7 and 8, it can be seen that the prediction result obtained by the heart rate variability prediction method in the embodiment of the present invention is closer to the result detected by the conventional electrocardiograph monitoring device shown in fig. 6, that is, the heart rate variability prediction method of the present invention is closer to the real detection result, and the prediction result has a reference value.

According to an embodiment of the invention, the overall performance of the model in the embodiment of the invention is measured by the ratio between absolute beat interval error and true value, and is commonly used in heart rate variability analysis: the monitoring ability of the model to heart rate variability in the embodiments of the present invention is measured by the above three indicators, the square root of the sum of squares of consecutive heart beat interval differences (Root Mean Square of the Successive Differences, RMSSD), the standard deviation of consecutive normal R-R interval differences (Standard Deviation of THE DIFFERENCES Between Adjacent Normal R-R INTERVALS, SDSD), the logarithmic scale of consecutive normal heart beat interval differences exceeding 50 milliseconds (Proportion of Pairs of Successive NN INTERVALS THAT DIFFER by More than 50ms, pn 50).

Specifically, as shown in table 1, the median and the average of IBI, RMSSD, SDSD, pNN50 0 four indices were obtained from the calculation.

TABLE 1

The results of the experiments according to the examples of the present invention are shown in fig. 8 and table 1. From the above results, it can be seen that the present embodiment achieves more accurate monitoring performance in the heart rate variability monitoring of various conditions such as sinus arrhythmia, sinus tachycardia, sinus bradycardia, ST segment abnormalities, sinus rhythm, and completely normal heart rate. The smaller the error value, the higher the consistency of the disclosed embodiments with conventional electrocardiographic methods. As shown in fig. 8, with 800ms as the standard IBI interval, the relative errors of various diseases measured according to the embodiments of the present disclosure are all within 4%, which indicates that the measured results of the embodiments of the present disclosure are highly consistent with the standard implementation, further demonstrating the accuracy of the variability prediction method of the central rate of the embodiments of the present disclosure.

As shown in fig. 9, the heart rate variability predicting means 900 of this embodiment includes a reading module 910 and a predicting module 920.

The reading module 910 is configured to read, from a data storage area, a plurality of reference signals related to cardiac exercise of a heart portion of a measured object, where the reference signals include three-dimensional spatial features and temporal features, the plurality of reference signals correspond to a plurality of spatial location points, the reference signal of each spatial location point corresponds to a plurality of time windows, the plurality of reference signals are obtained by processing a plurality of original signals, and the plurality of original signals are: a plurality of millimeter wave radar echo signals reflected by a radiated area of the object under test. In an embodiment, the reading module 910 may be configured to perform the operation S110 described above, which is not described herein.

The prediction module 920 is configured to predict a plurality of reference signals by using a trained heart beat rhythm prediction model, and output a target prediction result for representing a heart beat rhythm of a measured object, where the heart beat rhythm prediction model includes a space-time feature extraction base network, a space feature compression network, a space feature restoration network, a time feature compression network, a space-time feature fusion network, and a feature mapping network. In an embodiment, the prediction module 920 may be configured to perform the operation S120 described above, which is not described herein.

According to the embodiment of the invention, the heart rhythm prediction model establishes a space-time relation model between the body reflection signal and the heart beat through the deep learning network model, and in the training process, an end-to-end deep learning network is constructed by using the extracted heart mechanical activity signal, namely the heart rhythm prediction model is constructed, and the heart mechanical activity to heart electrical activity mapping is completed by using the heart rhythm prediction model to obtain heart electrical sign prediction information, specifically, heart waveform is obtained.

According to the embodiment of the invention, through the device, the mechanical activity sensing of micro motion of the chest heart of the human body based on the millimeter wave radar is realized by utilizing a signal processing technology in combination with an end-to-end deep learning network, the mapping of the spatial distribution of the surface of the whole chest and the possible difference of the signals at different positions are considered, the mapping of various indexes from the mechanical activity of the chest heart of the human body to the electrical activity is completed, the heart rate variability prediction with non-contact fine granularity is realized, the automatic mapping processing of artificial intelligence is realized, the prediction result is provided for a user to refer to, a new information dimension is provided, and the user experience is improved.

According to the embodiment of the invention, in each prediction operation, the heart beat rhythm prediction model outputs a group of local prediction results based on the original input characteristics, and a plurality of groups of local prediction results corresponding to a plurality of prediction operations are taken as target prediction results together, wherein the characteristic dimensions of the original input characteristics comprise three-dimensional space characteristic dimensions, time characteristic dimensions and channel characteristic dimensions; the prediction module 920 includes a selection sub-module and an output sub-module.

The selecting sub-module is used for selecting part of reference signals from the plurality of reference signals as a Kth selection signal, and carrying out characteristic splicing on the Kth selection signal and a Kth-1 th local prediction result output by the heart beat rhythm prediction model to generate a Kth original input characteristic;

And the output sub-module is used for inputting the Kth original input characteristic into the heart beat rhythm prediction model and outputting the Kth local prediction result.

According to an embodiment of the invention, the selection submodule comprises a feature stitching unit.

The characteristic splicing unit is used for carrying out characteristic splicing on the K-th selection signal based on the three-dimensional space characteristic dimension and the time characteristic dimension, carrying out characteristic splicing on the K-1-th local prediction result based on the channel characteristic dimension, and generating the K-th original input characteristic.

According to an embodiment of the invention, the output submodule comprises an input unit, an isolation unit, a feature reduction unit, a feature compression unit, a feature fusion unit and a feature mapping unit.

The input unit is used for inputting the K-th original input feature into the space-time feature extraction basic network to perform preliminary extraction of the space-time feature, and generating an initial space-time feature;

the isolation unit is used for isolating the time features in the initial space-time features, inputting the space features in the initial space-time features into the space feature compression network for feature compression, and generating compressed space features;

The feature reduction unit is used for inputting the compressed space features into a space feature reduction network to perform feature extraction and reduction, so as to generate reduced space features;

The feature compression unit is used for inputting the time features in the initial space-time features into the time feature compression network to perform feature compression, so as to generate compression time features;

the feature fusion unit is used for inputting the restored space features and the compressed time features into the space-time feature fusion network to perform feature fusion, so as to generate fusion space-time features;

And the feature mapping unit is used for inputting the fused space-time features into a feature mapping network to perform feature mapping and outputting a K-th local prediction result.

According to an embodiment of the invention, the heart rate variability predicting device further comprises a transmitting module, a first processing module, a second processing module, a third processing module and a fourth processing module.

The transmitting module is used for transmitting millimeter wave radar transmitting signals to the measured object and receiving a plurality of millimeter wave radar echo signals reflected by a radiated area of the measured object;

The first processing module is used for performing first signal processing on the millimeter wave radar echo signals to obtain a plurality of first-stage signals, wherein the first signal processing comprises: performing coherent accumulation operation on a plurality of millimeter wave radar echo signals defined by a plurality of channels formed by N transmitting antennas and M receiving antennas, wherein the signals in the first stage comprise three-dimensional space characteristics and time characteristics;

The second processing module is used for performing second signal processing on the plurality of first-stage signals and generating second-stage signals used for representing the central position of the human body of the tested object, wherein the second-stage signals contain three-dimensional space features;

The third processing module is used for performing third signal processing on a plurality of millimeter wave radar echo signals in the target three-dimensional space area by taking a preset neighborhood range of the spatial position point of the second stage signal as the target three-dimensional space area to obtain a plurality of third stage signals related to the mechanical movement of the heart part of the tested object, wherein the third stage signals comprise three-dimensional space features and time features;

And the fourth processing module is used for performing fourth signal processing on the plurality of third-stage signals so as to eliminate the influence of mechanical movement caused by the respiration of the tested object and obtain a plurality of reference signals related to the heart beat movement of the heart part of the tested object.

According to an embodiment of the invention, the second processing module comprises a feature accumulation sub-module, a calculation sub-module and a determination sub-module.

The characteristic accumulation sub-module is used for carrying out characteristic accumulation on the signal value corresponding to each space position point in the plurality of first-stage signals based on a plurality of time windows to generate a plurality of time accumulation signals corresponding to the plurality of space position points one by one, wherein the time accumulation signals comprise three-dimensional space characteristics;

a calculation sub-module for calculating a signal difference value between each time-integrated signal and a noise signal group, wherein the noise signal group is composed of a plurality of target signals within a predetermined three-dimensional neighborhood range of the time-integrated signal, and the plurality of target signals are not directly adjacent to the time-integrated signal;

And the determining submodule is used for determining a second-stage signal used for representing the human body center position of the tested object from the time accumulation signals according to the signal difference value corresponding to each time accumulation signal.

Any of the plurality of modules of the reading module 910 and the prediction module 920 may be combined in one module to be implemented, or any of the plurality of modules may be split into a plurality of modules according to an embodiment of the present invention. Or at least some of the functionality of one or more of the modules may be combined with, and implemented in, at least some of the functionality of other modules. According to embodiments of the invention, at least one of the read module 910 and the predict module 920 may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or in hardware or firmware, such as any other reasonable way of integrating or packaging the circuit, or in any one of or a suitable combination of three of software, hardware, and firmware. Or at least one of the reading module 910 and the prediction module 920 may be at least partially implemented as a computer program module, which, when executed, may perform the corresponding functions.

The embodiments of the present invention are described above. These examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the invention is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the invention, and such alternatives and modifications are intended to fall within the scope of the invention.

Claims

1. A heart rate variability prediction method based on millimeter wave radar signals performed by an electronic device, comprising:

Predicting the plurality of reference signals by using a trained heart beat rhythm prediction model, and outputting a target prediction result for representing the heart beat rhythm of the tested object, wherein the heart beat rhythm prediction model comprises a space-time feature extraction basic network, a space feature compression network, a space feature reduction network, a time feature compression network, a space-time feature fusion network and a feature mapping network.

2. The method according to claim 1, wherein:

predicting the plurality of reference signals by using the heart beat rhythm prediction model comprises the steps of executing a plurality of prediction operations, wherein in each prediction operation, the heart beat rhythm prediction model outputs a group of local prediction results based on original input characteristics, the local prediction results corresponding to the plurality of prediction operations are used as target prediction results together, and the characteristic dimensions of the original input characteristics comprise three-dimensional space characteristic dimensions, time characteristic dimensions and channel characteristic dimensions;

the kth one of the multiple prediction operations includes:

and inputting the Kth original input characteristic into the heart beat rhythm prediction model, and outputting a Kth local prediction result.

3. The method of claim 2, wherein feature stitching the kth selection signal with the kth-1 st local prediction result output by the heart beat rhythm prediction model comprises:

And performing feature stitching on the Kth selection signal based on the three-dimensional space feature dimension and the time feature dimension, and performing feature stitching on the Kth-1 th local prediction result based on the channel feature dimension to generate a Kth original input feature.

4. The method of claim 2, wherein inputting the kth original input feature into the heart beat rhythm prediction model, outputting a kth local prediction result comprises:

inputting the Kth original input feature into the space-time feature extraction basic network to perform preliminary extraction of space-time features, and generating initial space-time features;

Isolating the time features in the initial space-time features, and inputting the space features in the initial space-time features into a space feature compression network for feature compression to generate compressed space features;

inputting the compressed space features into a space feature restoration network to perform feature extraction and restoration, so as to generate restored space features;

Inputting the restored space features and the compressed time features into the space-time feature fusion network to perform feature fusion, so as to generate fusion space-time features;

And inputting the fused space-time features into the feature mapping network to perform feature mapping, and outputting a K-th local prediction result.

5. The method of claim 4, wherein the spatio-temporal feature extraction base network comprises:

6. The method of claim 1, further comprising:

Performing first signal processing on the millimeter wave radar echo signals to obtain a plurality of first-stage signals, wherein the first signal processing comprises: performing coherent accumulation operation on the millimeter wave radar echo signals defined by a plurality of channels consisting of N transmitting antennas and M receiving antennas, wherein the signals in the first stage comprise three-dimensional space characteristics and time characteristics;

Performing second signal processing on the plurality of first-stage signals to generate second-stage signals for representing the human body center position of the tested object, wherein the second-stage signals comprise three-dimensional space features;

Taking a preset neighborhood range of the spatial position point of the second-stage signal as a target three-dimensional space region, and performing third signal processing on a plurality of millimeter wave radar echo signals in the target three-dimensional space region range to obtain a plurality of third-stage signals related to the mechanical movement of the heart part of the tested object, wherein the third-stage signals comprise three-dimensional space features and time features;

and performing fourth signal processing on the plurality of third-stage signals to eliminate the influence of mechanical motion caused by the respiration of the tested object and obtain the plurality of reference signals related to the heart beat motion of the heart part of the tested object.

7. The method of claim 6, wherein performing second signal processing on the plurality of first stage signals to generate a second stage signal that characterizes a human body center position of the subject comprises:

Performing feature accumulation on signal values corresponding to each spatial position point in the plurality of first-stage signals based on the plurality of time windows to generate a plurality of time accumulation signals corresponding to the plurality of spatial position points one by one, wherein the time accumulation signals comprise three-dimensional spatial features;

calculating a signal difference value between each of the time-integrated signal and a noise signal group, wherein the noise signal group is composed of a plurality of target signals within a predetermined three-dimensional neighborhood range of the time-integrated signal, and the plurality of target signals are not directly adjacent to the time-integrated signal;

And determining a second-stage signal for representing the human body center position of the tested object from the time accumulation signals according to the signal difference value corresponding to each time accumulation signal.

8. The method of claim 6, wherein performing a coherent accumulation operation on the plurality of millimeter wave radar echo signals defined by the plurality of channels comprises employing the method of:

N is the number of receiving antennas;

m is the number of transmitting antennas;

t is the total sampling point number in each frame period of the radar;

Is a sampling point within a period;

for a plurality of millimeter wave radar echo signals, corresponding to a channel defined by an nth receiving antenna and an mth transmitting antenna in an acquisition period Is a signal of (2);

Is the sampling period;

Is the speed of light;

Is the wavelength of the millimeter wave radar transmit signal.

9. The method of claim 6, wherein fourth signal processing the plurality of third stage signals comprises:

Performing second-order differential processing on the plurality of third-stage signals to eliminate the influence of mechanical motion caused by respiration of the subject, wherein performing second-order differential processing on the plurality of third-stage signals includes:

wherein:

Is a third stage signal;

Is that A third stage signal of the nth sample, n=1, 2, 3;

Is that A third phase signal sampled by the nth sample;

is the value of i samples apart in the time series.

10. A heart rate variability prediction device based on millimeter wave radar signals, comprising:

The reading module is used for reading a plurality of reference signals related to heart beat movement of a heart part of a tested object from the data storage area, wherein the reference signals comprise three-dimensional space features and time features, the plurality of reference signals correspond to a plurality of space position points, the reference signals of each space position point correspond to a plurality of time windows, the plurality of reference signals are obtained by processing a plurality of original signals, and the plurality of original signals are: a plurality of millimeter wave radar echo signals reflected by a radiated area of a measured object;

The prediction module is used for predicting the plurality of reference signals by using a trained heart beat rhythm prediction model and outputting a target prediction result for representing the heart beat rhythm of the tested object, wherein the heart beat rhythm prediction model comprises a space-time feature extraction basic network, a space feature compression network, a space feature reduction network, a time feature compression network, a space-time feature fusion network and a feature mapping network.