CN113456061A - Sleep posture monitoring method and system based on wireless signals - Google Patents
Sleep posture monitoring method and system based on wireless signals Download PDFInfo
- Publication number
- CN113456061A CN113456061A CN202110665441.3A CN202110665441A CN113456061A CN 113456061 A CN113456061 A CN 113456061A CN 202110665441 A CN202110665441 A CN 202110665441A CN 113456061 A CN113456061 A CN 113456061A
- Authority
- CN
- China
- Prior art keywords
- neural network
- multipath
- signal
- sleep posture
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000012544 monitoring process Methods 0.000 title claims abstract description 32
- 230000033001 locomotion Effects 0.000 claims abstract description 40
- 238000012549 training Methods 0.000 claims abstract description 35
- 238000001914 filtration Methods 0.000 claims abstract description 34
- 238000013528 artificial neural network Methods 0.000 claims abstract description 28
- 230000000241 respiratory effect Effects 0.000 claims abstract description 28
- 230000029058 respiratory gaseous exchange Effects 0.000 claims abstract description 24
- 238000003062 neural network model Methods 0.000 claims description 21
- 238000013527 convolutional neural network Methods 0.000 claims description 14
- 238000013526 transfer learning Methods 0.000 claims description 9
- 239000013598 vector Substances 0.000 claims description 9
- 230000004044 response Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 claims description 4
- 238000009499 grossing Methods 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 claims description 3
- 125000004122 cyclic group Chemical group 0.000 claims description 3
- 230000003247 decreasing effect Effects 0.000 claims description 3
- 230000006870 function Effects 0.000 claims description 3
- 238000005316 response function Methods 0.000 claims description 3
- 238000002156 mixing Methods 0.000 claims description 2
- 230000000875 corresponding effect Effects 0.000 description 10
- 208000004210 Pressure Ulcer Diseases 0.000 description 3
- 230000002596 correlated effect Effects 0.000 description 3
- 208000008784 apnea Diseases 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000002980 postoperative effect Effects 0.000 description 2
- 230000006641 stabilisation Effects 0.000 description 2
- 238000011105 stabilization Methods 0.000 description 2
- 206010061818 Disease progression Diseases 0.000 description 1
- 208000018737 Parkinson disease Diseases 0.000 description 1
- 206010042434 Sudden death Diseases 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000005750 disease progression Effects 0.000 description 1
- 230000001037 epileptic effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000001356 surgical procedure Methods 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1116—Determining posture transitions
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/08—Detecting, measuring or recording devices for evaluating the respiratory organs
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1126—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Veterinary Medicine (AREA)
- Animal Behavior & Ethology (AREA)
- Public Health (AREA)
- Theoretical Computer Science (AREA)
- Surgery (AREA)
- Medical Informatics (AREA)
- Heart & Thoracic Surgery (AREA)
- Pathology (AREA)
- Physiology (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Dentistry (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Mathematical Physics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Pulmonology (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention discloses a sleep posture monitoring method and a system based on wireless signals, wherein the method comprises the following steps: transmitting low-power radio signals, receiving reflection signals of a human body and the surrounding environment, and extracting RF snapshots; identifying the motion events of the RF snapshots, defining intervals among the motion events as stable time periods, and calculating according to the relative signal power to obtain the multipath distribution of the stable time periods; performing respiratory filtration on multipath characteristics of the surrounding environment, and reserving a reflected signal from a human body; inputting the multipath feature files after the respiration filtration into a full-connection neural network, training the neural network, and predicting the sleeping posture corresponding to each multipath feature file; learning in the training data of the source user infers the target user's sleep posture using a small number of calibration points of the target user. Has the advantages that: the invention can provide accurate sleep posture monitoring under the condition of not influencing the privacy and the sleeping comfort of the user.
Description
Technical Field
The invention relates to the field of medical equipment and physiological signal detection, in particular to a sleep posture monitoring method and system based on wireless signals.
Background
Clinical studies have shown that sleep posture is an important marker of disease progression and has a significant impact on human health. For example, a reduced number of nighttime turns may mean a worsening of the condition of a patient with parkinson's disease, infrequent turns may lead to bed sores in the patient after surgery, improper sleeping posture may even increase the risk of sudden death in an epileptic patient. These examples all show us the importance of continuous, fully automatic sleep posture monitoring.
Two main monitoring methods in the past: the installation of a camera in a bedroom and the use of an accelerometer, a pressure sensor and other bed sensors are not suitable methods, the camera invades the privacy of a user, and the performance of the camera is affected under the condition of poor light at night; while the use of various sensors affects the comfort of the user's sleep.
An effective solution to the problems in the related art has not been proposed yet.
Disclosure of Invention
The invention provides a sleep posture monitoring method and system based on wireless signals, aiming at the problems in the related art and aiming at overcoming the technical problems in the prior related art.
Therefore, the invention adopts the following specific technical scheme:
according to an aspect of the present invention, there is provided a sleep posture monitoring method based on wireless signals, the method comprising the steps of:
s1, transmitting low-power radio signals, receiving reflection signals of human bodies and surrounding environments, and extracting RF snapshots;
s2, identifying the movement events of the RF snapshots, defining the intervals between the movement events as stable time periods, and calculating the multipath distribution of the stable time periods according to the relative signal power;
s3, performing respiratory filtration on multipath characteristics of the surrounding environment, and reserving a reflected signal from a human body;
s4, inputting the multipath feature files after the respiration filtration into a full-connection neural network, training the neural network, and predicting the sleep posture corresponding to each multipath feature file;
s5, learning in the training data of the source user infers the target user' S sleep posture using a small number of calibration points of the target user.
Further, the defining the interval between the motion events as the stable period in S2 further includes the following steps:
defining a short segment of the fixed duration observations as short observations, defining a short observation respiratory-to-noise ratio BS(s-BNR) is the ratio of respiratory energy to total energy within the short observation;
identifying motion events from a motion image, wherein the motion image is a matrix, the rows represent locations, the columns represent discrete time points, and the cells (i, j) represent short observations of the jth location at the ith time point, the values of which are the respiratory-to-noise ratio B of the short observationsS(s-BNR)；
After a moving image is obtained, training a classifier based on a convolutional neural network, detecting human body motion, classifying each column in the moving image, wherein an image provided for the convolutional neural network is a small image [ i-k, i + k ] comprising all rows and columns from i-k to i + k, wherein k is a self-defined small number, the convolutional neural network outputs 1 to represent the human body motion, and otherwise, 0 is output;
wherein, BSThe calculation method comprises the following steps:
performing fast Fourier transform on the short observation signal, and finding out the fast Fourier transform frequency band with maximum energy in the human respiration range, BSIs the ratio between the energy of this band and the sum of the energies of all fast fourier transform bands.
Further, the step of calculating the multipath profile of the stable time period according to the relative signal power in S2 further includes the following steps:
and calculating the variance of each voxel in the RF snapshot in the stable period to obtain the multipath distribution in the stable period.
Further, the respiratory filtering the multipath characteristics of the surrounding environment in S3 further includes the following steps:
s31, extracting the respiratory signal of the subject in the RF snapshot of the stable period;
s32, correlating the extracted respiration signal with the time series of signal amplitudes of each voxel in the RF snapshot, calculating the absolute value of the pearson correlation coefficient between the respiration signal and the signal amplitude of the corresponding RF voxel, and providing a spatial filter with the correlation;
s33, multiplying the whole multipath characteristics by a spatial filter, extracting the multipath characteristics after respiratory filtration, and filtering the influence of the environment;
wherein the step of extracting the respiratory signal of the subject in the RF snapshot of the stable period in S31 further comprises the steps of:
assuming that the reflectors do not move in each scan of the wireless device, the time domain representation of the signal received by the system with a single reflector during the t-th scan cycle is:
the frequency response of the reflector at distance d (t) at carrier frequency f is:
l [ d ] (t)，f]Written as l [ D + delta [ ]i(t)，f]Where D is the mean position of the reflector, δi(t) is the minute time-varying motion corresponding to respiration, expanded to the first order using a taylor series, with a frequency response function of: l [ d (t), f]＝l(D，f)+δi(t) l' (D, f); where the first term is the average frequency response over time and the second term is δi(t) corresponds to a time-varying signal related to respiratory motion.
Further, the training of the neural network in S4 includes the following steps;
the method comprises the steps that a subject wears an accelerometer to collect a real angle of the body, and the angle values measured by the accelerometer in a stable period are averaged to obtain a real sleep posture of the subject in the period;
training a neural network by comparing the true bottom of intersection with the predicted angle;
wherein the sleeping posture is described by an included angle between two normal vectors, the two normal vectors are respectively the bed surface and the front trunk surface of the user;
when comparing the true intersection base with the predicted angle, which results in discontinuities and produces a large loss when calculating their difference, in order to guarantee the smoothness of the loss function, the cyclic loss is defined as follows:
Further, learning in the training data of the source user to infer the sleep posture of the target user using the small number of calibration points of the target user in S5 further includes the following steps:
s51, realizing the virtual bed alignment by increasing or decreasing the travel distance of all radio frequency reflections in the multipath profile, and aligning the direction of the user relative to the equipment by turning the angle;
s52, normalizing the power distribution of each voxel in the filtered multipath characteristic diagram;
s53, aligning data points from the source user and the target user, and giving one calibration point (x) of the target user0，y0) Selecting a product satisfying y0And yiAll points (x) between which the angular difference is less than the threshold valuei，yi)；
S54, similarity according to distance of the selected point from the calibration point: | xi-x0||2Sorting the selected points and selecting the 30 data points that are most similar to the calibration points, the enhanced set of data points being referred to as a virtual target;
s55, training a neural network model by combining the data points of the expanded virtual target with the data points of the source user so as to improve the performance of the neural network model on the target user;
and S56, verifying the precision of the neural network model on the target calibration point, eliminating the neural network model with poor precision, and performing majority voting in the neural network model with high precision.
Further, the method for measuring the bed position in the S51 comprises the following steps:
the pixel sum is performed on all the filtered multipath profiles and a gaussian filter with a standard deviation of 1 is applied to eliminate the small position mismatch, then the pixel with the highest sum is the bed position.
Further, the step of normalizing the power distribution of each voxel in the filtered multipath profile in S52 further includes the steps of:
for each data point, the mean of the data point distribution was subtracted and divided by its standard deviation.
Further, in S56, verifying the accuracy of the neural network model at the target calibration point, and eliminating the neural network model with poor accuracy, and performing majority voting in the neural network model with high accuracy further includes the following steps:
creating a histogram of the prediction angle, smoothing the histogram through a Gaussian filter with the standard deviation of 20, and selecting the maximum value after smoothing as the final prediction angle.
According to another aspect of the present invention, there is provided a wireless signal based sleep posture monitoring system, the system comprising:
the system comprises an RF snapshot extracting module, a sleep cycle multipath feature extracting module, a breath filtering module, a neural network training module and a transfer learning module;
the RF snapshot extracting module is used for transmitting low-power radio signals, receiving reflection signals of a human body and the surrounding environment and extracting RF snapshots;
the module for extracting the multipath characteristics of the sleep cycle is used for identifying the movement events of the RF snapshot, defining the interval between the movement events as a stable time period, and calculating the multipath distribution of the stable time period according to the relative signal power;
the breath filtering module is used for carrying out breath filtering on multipath characteristics of the surrounding environment and reserving a reflection signal from a human body;
the neural network training module is used for inputting the multipath feature files after the respiration filtration into the fully-connected neural network, training the neural network and predicting the sleeping posture corresponding to each multipath feature file;
the transfer learning module is used for learning and deducing the sleeping posture of the target user by using a small number of calibration points of the target user in the training data of the source user.
The invention has the beneficial effects that: the invention provides a sleep posture monitoring method and system based on wireless signals, which is a non-contact and non-invasive mode, infers the sleep posture of a testee by analyzing radio frequency reflection signals, and can be popularized to new users in new environments for use only by little extra training. The method works by extracting radio frequency reflection signals in the environment, the signals reflected from a user are separated from other multipath signals, then multipath characteristics after respiratory filtration are analyzed through a designed neural network model, the sleeping posture of the user is further deduced, accurate sleeping posture monitoring can be provided under the condition that the privacy and sleeping comfort of the user are not influenced, and the method is very important for evaluating the sleeping quality, avoiding postoperative bedsores, reducing apnea, tracking disease progress and the like.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a flow chart of a wireless signal based sleep posture monitoring method according to an embodiment of the present invention;
FIG. 2 is a flow chart of breath filtering based on a wireless signal sleep posture monitoring method according to an embodiment of the present invention;
fig. 3 is a flow chart of implementing a transfer learning scheme based on a wireless signal sleep posture monitoring method according to an embodiment of the present invention.
Detailed Description
For further explanation of the various embodiments, the drawings which form a part of the disclosure and which are incorporated in and constitute a part of this specification, illustrate embodiments and, together with the description, serve to explain the principles of operation of the embodiments, and to enable others of ordinary skill in the art to understand the various embodiments and advantages of the invention, and, by reference to these figures, reference is made to the accompanying drawings, which are not to scale and wherein like reference numerals generally refer to like elements.
According to the embodiment of the invention, a sleep posture monitoring method and a system based on wireless signals are provided, the aim is to utilize multipath effect, analyze a multipath characteristic diagram of received reflection and deduce the posture of a human body, and one key challenge for realizing the aim is that radio frequency signals can reflect from a plurality of objects in the environment, not only from the human body. And only the reflected signal comes from the human body to be related to the sleeping posture, and thus, it is necessary to extract the signal directly or indirectly reflected from the human body. Signals reflected from the torso region by a person during sleep are all subject to respiratory modulation, and techniques may be used to separate signals along different paths and correlate these signals with the respiratory signal of the subject to identify the radio frequency reflected signal from the person. And then, by designing a neural network model, the sleep posture is deduced by using the multipath characteristics after the respiratory filtration. The key problem of the neural network model is portability between different users in different environments, and because radio frequency reflection and multipath effect depend on the environment, the invention designs a model which is easy to transplant, learns and identifies the basic characteristics of each sleep posture, and transplants the model into a new environment by using a small amount of additional tag data for use.
The basic working principle is that low-power radio signals are transmitted, reflected signals in the surrounding environment are received, the received multipath characteristic diagram is subjected to respiratory filtration, and radio frequency signals directly or indirectly reflected from a subject are extracted; inputting the filtered multipath feature map into a fully-connected neural network, and inferring the sleeping posture of the subject; and then adopting a designed transfer learning model to infer the sleeping posture of the new user by using a small amount of additional label data.
Referring now to the drawings and the detailed description, the present invention will be further described, as shown in fig. 1, in accordance with an embodiment of the present invention, a sleep posture monitoring method based on wireless signals, the method includes the following steps:
s1, transmitting low-power radio signals, receiving reflection signals of human bodies and surrounding environments, and extracting RF snapshots;
the working principle of the radio system is as follows: low power radio signals are transmitted and reflections in the surrounding environment are received, which when the signal is incident on the human body, will be reflected based on the body direction. At each time instant, signal values from various points in space are output, called an RF snapshot. Dividing the space into N corners and M distances, each RF snapshot is an N × M matrix, where each element represents a spatial point, called an RF voxel, and the value of each element represents the size of the RF reflection from that spatial point.
S2, identifying the motion events of the RF snapshots, defining the intervals between the motion events as stable time periods, and calculating the multipath distribution of the stable time periods according to the relative signal power (dividing sleep cycles and extracting the multipath characteristics of each cycle);
people typically sleep in one position for a period of time, then move, and then enter another sleep position. The sleep posture monitoring system divides night into a series of stable sleep periods, the body direction is basically kept unchanged in each stable period, and the system extracts the sleep posture from each stable period.
First, the system identifies motion events from a series of RF snapshots, defining the intervals between the motion events as stable periods. The specific method comprises the following steps: defining a short segment of the fixed duration observations as short observations, defining a short observation respiratory-to-noise ratio BS(s-BNR) is the ratio of respiratory energy to total energy in the short observation, BSThe calculation method comprises the following steps: performing Fast Fourier Transform (FFT) on the short observation signal to find the Fast Fourier Transform (FFT) frequency band with maximum energy in the human respiration range, BSIs the ratio between the energy of this band and the sum of the energies of all Fast Fourier Transform (FFT) bands. Identifying motion events from a motion image, wherein the motion image is a matrix, the rows represent locations, the columns represent discrete time points, and the cells (i, j) represent short observations of the jth location at the ith time point, the values of which are the respiratory-to-noise ratio B of the short observationsS(s-BNR); after obtaining the moving image, training a classifier based on a Convolutional Neural Network (CNN) to detect human body movement, wherein the Convolutional Neural Network (CNN) adopts a classic VGG16 framework to classify each column in the moving image, and the image provided to the Convolutional Neural Network (CNN) is a small image [ i-k, i + k ] comprising all rows and the (i-k) th to (i + k) th columns]Where k is a custom small number, the Convolutional Neural Network (CNN) outputs a "1" indicating body motion, otherwise outputs a "0".
Then, the multipath distribution in the stable period is represented by the relative signal power, which is specifically done by: and calculating the variance of each voxel in the RF snapshot in the stable period to obtain the multipath distribution in the stable period.
S3, performing respiratory filtration on multipath characteristics of the surrounding environment, and reserving a reflection signal from the human body (respiratory filtration);
the overall multipath signature over a period of stable sleep includes both reflected signals from the subject's body and reflected signals from the environment. The reflected signals from the environment are independent of the sleeping posture and the bedroom environment of different users is highly specific, so the reflections from the environment do not facilitate the sleeping posture to be resolved in real time.
Since the motion of the human torso changes the multipath signal corresponding to the human body in a manner correlated with the respiration signal, the multipath signal correlated with the environment is not correlated with respiration. This property can be used to filter out the multipath feature contributions of the environment, leaving only the contribution of the reflected signal from the human body. The specific method is as shown in figure 2:
(1) a respiration signal of the subject is extracted from the RF snapshot of the stabilization period, the respiration signal being a time sequence. (the extraction of the respiration signal is as follows: assuming the reflectors do not move in each scan of the wireless device, the time domain representation of the signal received by the system with a single reflector during the t-th scan cycle is:
the frequency response of the reflector at distance d (t) at carrier frequency f is:
1) mixing l [ d (t), f)]Written as l [ D + delta [ ]i(t)，f]Where D is the mean position of the reflector (mean position of the chest during breathing), δi(t) is the minute time-varying motion corresponding to respiration, developed to the first order term, frequency, using a Taylor seriesThe rate response function can be approximated as:
2)l[d(t)，f]＝l(D，f)+δi(t) l' (D, f); where the first term is the average frequency response over time and the second term is δi(t) corresponds to a time-varying signal related to respiratory motion. )
(2) The extracted respiratory signal is associated with a time series of signal amplitudes for each voxel in the RF snapshot. The specific method comprises the following steps: the absolute value of the pearson correlation coefficient between the respiratory signal and the signal amplitude of the corresponding RF voxel is calculated to provide a spatial filter with the correlation.
(3) And multiplying the integral multipath characteristics by a filter, extracting the multipath characteristics after the respiratory filtration, and filtering out the influence of the environment.
S4, inputting the multi-path feature files after the respiration filtration into a full-connection neural network, training the neural network, and predicting the sleep posture (training the neural network, predicting the sleep posture) corresponding to each multi-path feature file;
the breathing filtered multipath profiles are input to a fully-connected neural network, which is trained to predict a sleep posture corresponding to each multipath profile. The sleeping posture is described by the angle between two normal vectors, the bed surface and the front torso surface of the user. To train the network, the predicted angle needs to be compared to the true angle. The true angle of the body is collected by the subject wearing an accelerometer. The system of the present invention averages the angular values measured by the accelerometer during the stabilization period to obtain the true sleep posture of the subject during this period.
A direct comparison of the angles results in discontinuities and a simple calculation of their difference results in a large penalty. Therefore, to ensure the smoothness of the loss function, the cyclic loss is defined as follows:
S5, learning in the training data of the source user to infer the sleep posture of the target user by using a small number of calibration points of the target user (transfer learning to be popularized to the use of a new user);
migration between different homes is described as a semi-supervised domain adaptation problem: there are multiple source users, each with rich tag data, while the target user has only a small amount of tag data, called calibration points. The design model learns from the training data of the source user to infer the sleep posture of the target user using a small number of calibration points, as shown in fig. 3:
aligning the bed position: to ensure that all direct path reflected signals have the same travel distance, the positions of the bed and radio of all users are aligned. Since the user cannot be required to move his bed, the virtual alignment is achieved by increasing or decreasing the travel distance of all radio frequency reflections in the multipath profile. While aligning the orientation of the user relative to the device by the flip angle. The position of the bed is measured as follows: the pixel sum is performed on all the filtered multipath profiles, and a gaussian filter with σ of 1 is applied to eliminate small position mismatch, so the pixel with the highest sum is the position estimate of the bed.
Power normalization: since the power of the radio frequency signal attenuates with distance, the power distribution of each voxel of the filtered multipath profile is normalized by: for each data point, the mean of the distribution was subtracted and then divided by its standard deviation.
And (3) target data enhancement: data points from a source user and a target user are first aligned. Then a calibration point (x) of the target user is given0，y0) Selecting a product satisfying y0And yiAll points (x) between which the angular difference is less than a certain threshold (default 20 degrees)i，yi). Then, for these selected points, based on their similarity to the L2 distance of the calibration point: | xi-x0||2They are sorted. Finally, 30 data which are most similar to the calibration points are selectedThe set of enhanced data points is referred to as a virtual target.
Adjusting the neural network model: and combining the data points of the expanded virtual target with the data points of the source user to train the model so as to improve the performance of the model on the target user.
Majority voting screens the model with the highest precision: and verifying the precision of the model on the target calibration point, removing the model with poor precision (which is 10% or more lower than the best model precision), and performing majority voting in the model with high precision. The specific method comprises the following steps: a histogram of predicted angles is created, where each angle has its own histogram. The histogram was smoothed with a gaussian filter with a standard deviation of 20. And finally, selecting the smoothed maximum value as a final prediction angle.
According to another aspect of the present invention, there is provided a wireless signal based sleep posture monitoring system, the system comprising:
the system comprises an RF snapshot extracting module, a sleep cycle multipath feature extracting module, a breath filtering module, a neural network training module and a transfer learning module;
the RF snapshot extracting module is used for transmitting low-power radio signals, receiving reflection signals of a human body and the surrounding environment and extracting RF snapshots;
the module for extracting the multipath characteristics of the sleep cycle is used for identifying the movement events of the RF snapshot, defining the interval between the movement events as a stable time period, and calculating the multipath distribution of the stable time period according to the relative signal power;
the breath filtering module is used for carrying out breath filtering on multipath characteristics of the surrounding environment and reserving a reflection signal from a human body;
the neural network training module is used for inputting the multipath feature files after the respiration filtration into the fully-connected neural network, training the neural network and predicting the sleeping posture corresponding to each multipath feature file;
the transfer learning module is used for learning and deducing the sleeping posture of the target user by using a small number of calibration points of the target user in the training data of the source user.
In summary, the present invention provides a sleep posture monitoring method and system based on wireless signals, which is a non-contact and non-invasive mode, and which infers the sleep posture of a subject by analyzing radio frequency reflection signals, and can be popularized to new users in new environments with little additional training. The method works by extracting radio frequency reflection signals in the environment, the signals reflected from a user are separated from other multipath signals, then multipath characteristics after respiratory filtration are analyzed through a designed neural network model, the sleeping posture of the user is further deduced, accurate sleeping posture monitoring can be provided under the condition that the privacy and sleeping comfort of the user are not influenced, and the method is very important for evaluating the sleeping quality, avoiding postoperative bedsores, reducing apnea, tracking disease progress and the like.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. A sleep posture monitoring method based on wireless signals is characterized by comprising the following steps:
s1, transmitting low-power radio signals, receiving reflection signals of human bodies and surrounding environments, and extracting RF snapshots;
s2, identifying the movement events of the RF snapshots, defining the intervals between the movement events as stable time periods, and calculating the multipath distribution of the stable time periods according to the relative signal power;
s3, performing respiratory filtration on multipath characteristics of the surrounding environment, and reserving a reflected signal from a human body;
s4, inputting the multipath feature files after the respiration filtration into a full-connection neural network, training the neural network, and predicting the sleep posture corresponding to each multipath feature file;
s5, learning in the training data of the source user infers the target user' S sleep posture using a small number of calibration points of the target user.
2. The sleep posture monitoring method based on wireless signals as claimed in claim 1, wherein the defining of the interval between the motion events as the stable period in S2 further comprises the steps of:
defining a short segment of the fixed duration observations as short observations, defining a short observation respiratory-to-noise ratio BS(s-BNR) is the ratio of respiratory energy to total energy within the short observation;
identifying motion events from a motion image, wherein the motion image is a matrix, the rows represent locations, the columns represent discrete time points, and the cells (i, j) represent short observations of the jth location at the ith time point, the values of which are the respiratory-to-noise ratio B of the short observationsS(s-BNR)；
After a moving image is obtained, training a classifier based on a convolutional neural network, detecting human body motion, classifying each column in the moving image, wherein an image provided for the convolutional neural network is a small image [ i-k, i + k ] comprising all rows and columns from i-k to i + k, wherein k is a self-defined small number, the convolutional neural network outputs 1 to represent the human body motion, and otherwise, 0 is output;
wherein, BSThe calculation method comprises the following steps:
performing fast Fourier transform on the short observation signal, and finding out the fast Fourier transform frequency band with maximum energy in the human respiration range, BSIs the ratio between the energy of this band and the sum of the energies of all fast fourier transform bands.
3. The sleep posture monitoring method based on wireless signals as claimed in claim 2, wherein the step of calculating the multi-path distribution of the stable period according to the relative signal power in S2 further comprises the steps of:
and calculating the variance of each voxel in the RF snapshot in the stable period to obtain the multipath distribution in the stable period.
4. The sleep posture monitoring method based on wireless signals as claimed in claim 3, wherein the step of respiratory filtering the multipath characteristics of the surrounding environment in S3 further comprises the steps of:
s31, extracting the respiratory signal of the subject in the RF snapshot of the stable period;
s32, correlating the extracted respiration signal with the time series of signal amplitudes of each voxel in the RF snapshot, calculating the absolute value of the pearson correlation coefficient between the respiration signal and the signal amplitude of the corresponding RF voxel, and providing a spatial filter with the correlation;
s33, multiplying the whole multipath characteristics by a spatial filter, extracting the multipath characteristics after respiratory filtration, and filtering the influence of the environment;
wherein the step of extracting the respiratory signal of the subject in the RF snapshot of the stable period in S31 further comprises the steps of:
assuming that the reflectors do not move in each scan of the wireless device, the time domain representation of the signal received by the system with a single reflector during the t-th scan cycle is:
u∈[0，Ts](ii) a Where A is the amplitude of the received signal and F0Is the minimum frequency of the sweep, TsIs the scan period, Ks＝Bw/TsIs the sweep frequency, d (t) is the distance of the reflector, τ [ d (t)]2d (t)/C is the transmission time of the signal, C is the speed of light;
the frequency response of the reflector at distance d (t) at carrier frequency f is:
mixing l [ d (t), f)]Written as l [ D + delta [ ]i(t)，f]Where D is the mean position of the reflector, δi(t) is the minute time-varying motion corresponding to respiration, expanded to the first order using a taylor series, with a frequency response function of: l [ d (t), f]＝l(D，f)+δi(t) l' (D, f); where the first term is the average frequency response over time and the second term is δi(t) corresponds to a time-varying signal related to respiratory motion.
5. The sleep posture monitoring method based on wireless signals as claimed in claim 1, wherein the training of neural network in S4 includes the following steps;
the method comprises the steps that a subject wears an accelerometer to collect a real angle of the body, and the angle values measured by the accelerometer in a stable period are averaged to obtain a real sleep posture of the subject in the period;
training a neural network by comparing the true bottom of intersection with the predicted angle;
wherein the sleeping posture is described by an included angle between two normal vectors, the two normal vectors are respectively the bed surface and the front trunk surface of the user;
when comparing the true intersection base with the predicted angle, which results in discontinuities and produces a large loss when calculating their difference, in order to guarantee the smoothness of the loss function, the cyclic loss is defined as follows:
6. The wireless-signal-based sleep posture monitoring method as claimed in claim 1, wherein learning in the training data of the source user to infer the sleep posture of the target user using a small number of calibration points of the target user in the S5 further comprises the steps of:
s51, realizing the virtual bed alignment by increasing or decreasing the travel distance of all radio frequency reflections in the multipath profile, and aligning the direction of the user relative to the equipment by turning the angle;
s52, normalizing the power distribution of each voxel in the filtered multipath characteristic diagram;
s53, aligning data points from the source user and the target user, and giving one calibration point (x) of the target user0，y0) Selecting a product satisfying y0And yiAll points (x) between which the angular difference is less than the threshold valuei，yi)；
S54, similarity according to distance of the selected point from the calibration point: | xi-x0||2Sorting the selected points and selecting the 30 data points that are most similar to the calibration points, the enhanced set of data points being referred to as a virtual target;
s55, training a neural network model by combining the data points of the expanded virtual target with the data points of the source user so as to improve the performance of the neural network model on the target user;
and S56, verifying the precision of the neural network model on the target calibration point, eliminating the neural network model with poor precision, and performing majority voting in the neural network model with high precision.
7. The sleep posture monitoring method based on wireless signals as claimed in claim 6, wherein the measuring method of bed in S51 comprises the following steps:
the pixel sum is performed on all the filtered multipath profiles and a gaussian filter with a standard deviation of 1 is applied to eliminate the small position mismatch, then the pixel with the highest sum is the bed position.
8. The wireless-signal-based sleep posture monitoring method of claim 6, wherein the step of normalizing the power distribution of each voxel in the filtered multipath feature map in S52 further comprises the steps of:
for each data point, the mean of the data point distribution was subtracted and divided by its standard deviation.
9. The sleep posture monitoring method based on wireless signals as claimed in claim 1, wherein the step S56 of verifying the accuracy of the neural network model at the target calibration point, and eliminating the neural network model with poor accuracy, and the step of majority voting in the neural network model with high accuracy further comprises the steps of:
creating a histogram of the prediction angle, smoothing the histogram through a Gaussian filter with the standard deviation of 20, and selecting the maximum value after smoothing as the final prediction angle.
10. A sleep posture monitoring system based on wireless signals, for implementing the steps of a sleep posture monitoring method based on wireless signals of any one of claims 1-9, the system comprising:
the system comprises an RF snapshot extracting module, a sleep cycle multipath feature extracting module, a breath filtering module, a neural network training module and a transfer learning module;
the RF snapshot extracting module is used for transmitting low-power radio signals, receiving reflection signals of a human body and the surrounding environment and extracting RF snapshots;
the module for extracting the multipath characteristics of the sleep cycle is used for identifying the movement events of the RF snapshot, defining the interval between the movement events as a stable time period, and calculating the multipath distribution of the stable time period according to the relative signal power;
the breath filtering module is used for carrying out breath filtering on multipath characteristics of the surrounding environment and reserving a reflection signal from a human body;
the neural network training module is used for inputting the multipath feature files after the respiration filtration into the fully-connected neural network, training the neural network and predicting the sleeping posture corresponding to each multipath feature file;
the transfer learning module is used for learning and deducing the sleeping posture of the target user by using a small number of calibration points of the target user in the training data of the source user.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110665441.3A CN113456061A (en) | 2021-06-16 | 2021-06-16 | Sleep posture monitoring method and system based on wireless signals |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110665441.3A CN113456061A (en) | 2021-06-16 | 2021-06-16 | Sleep posture monitoring method and system based on wireless signals |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113456061A true CN113456061A (en) | 2021-10-01 |
Family
ID=77870163
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110665441.3A Pending CN113456061A (en) | 2021-06-16 | 2021-06-16 | Sleep posture monitoring method and system based on wireless signals |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113456061A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114869241A (en) * | 2022-07-11 | 2022-08-09 | 西南交通大学 | Sleep respiratory event prediction method, device, equipment and readable storage medium |
| CN116563887A (en) * | 2023-04-21 | 2023-08-08 | 华北理工大学 | Sleeping posture monitoring method based on lightweight convolutional neural network |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015102713A2 (en) * | 2013-10-09 | 2015-07-09 | Massachusetts Institute Of Technology | Motion tracking via body radio reflections |
| CN109363647A (en) * | 2018-10-29 | 2019-02-22 | 北京大学（天津滨海）新代信息技术研究院 | Sleep quality monitoring method, device and terminal device |
| US20200064444A1 (en) * | 2015-07-17 | 2020-02-27 | Origin Wireless, Inc. | Method, apparatus, and system for human identification based on human radio biometric information |
| CN111387936A (en) * | 2019-01-02 | 2020-07-10 | 中国移动通信有限公司研究院 | Sleep stage identification method, device and equipment |
-
2021
- 2021-06-16 CN CN202110665441.3A patent/CN113456061A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015102713A2 (en) * | 2013-10-09 | 2015-07-09 | Massachusetts Institute Of Technology | Motion tracking via body radio reflections |
| US20200064444A1 (en) * | 2015-07-17 | 2020-02-27 | Origin Wireless, Inc. | Method, apparatus, and system for human identification based on human radio biometric information |
| CN109363647A (en) * | 2018-10-29 | 2019-02-22 | 北京大学（天津滨海）新代信息技术研究院 | Sleep quality monitoring method, device and terminal device |
| CN111387936A (en) * | 2019-01-02 | 2020-07-10 | 中国移动通信有限公司研究院 | Sleep stage identification method, device and equipment |
Non-Patent Citations (2)
| Title |
|---|
| SHICHAO YUE 等: "BodyCompass: Monitoring Sleep Posture with Wireless Signals", PROC. ACM INTERACT. MOB. WEARABLE UBIQUITOUS TECHNOL, pages 1 - 66 * |
| SHICHAO YUE 等: "Extracting Multi-Person Respiration from Entangled RF Signals", PROC. ACM INTERACT. MOB. WEARABLE UBIQUITOUS TECHNOL, pages 1 - 86 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114869241A (en) * | 2022-07-11 | 2022-08-09 | 西南交通大学 | Sleep respiratory event prediction method, device, equipment and readable storage medium |
| CN114869241B (en) * | 2022-07-11 | 2022-09-23 | 西南交通大学 | Sleep respiratory event prediction method, device, equipment and readable storage medium |
| CN116563887A (en) * | 2023-04-21 | 2023-08-08 | 华北理工大学 | Sleeping posture monitoring method based on lightweight convolutional neural network |
| CN116563887B (en) * | 2023-04-21 | 2024-03-12 | 华北理工大学 | Sleeping posture monitoring method based on lightweight convolutional neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Liu et al. | TagSheet: Sleeping posture recognition with an unobtrusive passive tag matrix | |
| Kim et al. | 1D CNN based human respiration pattern recognition using ultra wideband radar | |
| EP3866685B1 (en) | Systems and methods for micro impulse radar detection of physiological information | |
| IL245185B (en) | Motion monitor | |
| CN113435283B (en) | Ultra-wideband radar identity recognition method based on breath sample space | |
| CN110276271A (en) | Merge the non-contact heart rate estimation technique of IPPG and depth information anti-noise jamming | |
| CN113456061A (en) | Sleep posture monitoring method and system based on wireless signals | |
| CN103717124A (en) | Device and method for obtaining and processing measurement readings of a living being | |
| CN109431465B (en) | Old people sleep quality monitoring method based on radio frequency tag technology | |
| CN105354555B (en) | A kind of three-dimensional face identification method based on probability graph model | |
| Wang et al. | CSIID: WiFi-based human identification via deep learning | |
| Liu et al. | Human biometric signals monitoring based on wifi channel state information using deep learning | |
| CN113056228A (en) | System and method for detecting physiological information using multi-modal sensors | |
| CN107260173A (en) | A kind of breath measuring method based on camera Yu spherical label | |
| CN115474901A (en) | Non-contact living state monitoring method and system based on wireless radio frequency signals | |
| CN107847146B (en) | Method and system for automatically tagging activity on ECG data | |
| CN112363139A (en) | Human body breathing time length detection method and device based on amplitude characteristics and storage medium | |
| CN113273998A (en) | Human body sleep information acquisition method and device based on RFID label matrix | |
| CN106485232A (en) | A kind of personal identification method based on nose image feature in respiratory | |
| CN117055004A (en) | Three-dimensional human skeleton estimation method based on millimeter wave radar sparse point cloud | |
| CN109993116A (en) | A kind of pedestrian mutually learnt based on skeleton recognition methods again | |
| KR102317075B1 (en) | Non-contact stress inference apparatus and method using uwb radar sensor | |
| Li et al. | Respiratory motion estimation of tumor using point clouds of skin surface | |
| CN110338798B (en) | Human body respiration quantity evaluation system and method based on RFID (radio frequency identification) tag | |
| CN114098679B (en) | Vital sign monitoring waveform recovery method based on deep learning and radio frequency sensing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20211001 |