CN111353425A - Sleeping posture monitoring method based on feature fusion and artificial neural network - Google Patents

Abstract

The present invention is a sleeping posture monitoring method based on feature fusion and artificial neural network. The method aims at the characteristics of six sleeping posture types. The image is subjected to targeted preprocessing. While removing noise and improving image quality, it effectively retains as much useful information as possible to prepare for subsequent feature extraction and overall monitoring accuracy. A more complete and more characteristic sleeping position image. The combination of multi-feature fusion and artificial neural network achieves a higher recognition accuracy, reaching 99.17%, with better real-time performance. The experiment shows that it only takes 0.13s to recognize 180 pictures. The invention directly collects the pressure data between the human body and the mattress to generate the sleeping posture image, the data processing time is short, the real-time performance of the sleeping posture recognition is improved, and the relationship model between the sleeping posture transformation and the dynamic pressure is established later.

Description

A sleeping posture monitoring method based on feature fusion and artificial neural network

technical field

The invention relates to the field of physiological information monitoring, in particular to an unconstrained and interference-free sleeping posture monitoring method, in particular to a sleeping posture monitoring method based on feature information fusion and artificial neural network.

Background technique

Sleep is an important part of our life, and sleep state is directly related to people's psychological and physical health. In sleep state monitoring, sleep posture is one of the keys to objectively evaluate sleep quality. Effective monitoring of sleep posture in the home environment can achieve early diagnosis and prevention of respiratory diseases and pressure ulcers.

At present, the method of monitoring the sleeping posture without restraint and interference has gradually become the main research direction. Ye Yinqiu et al. ): 91-93.) Obtain the static sleeping position and dynamic video of the human body through the camera, and use the algorithm based on BP neural network to identify the four sleeping positions. The average recognition accuracy is 73%, which is low and easy to cause privacy risks. . Zhang Yichao et al. (Zhang Yichao, Yuan Zhenming, Sun Xiaoyan. Sleeping posture recognition based on cardiac shock signal [J]. Computer Engineering and Applications (1): 135-140.) Using piezoelectric thin film sensors to collect tiny vibrations in the human body in a non-contact way Signal, using the algorithm based on BP neural network for sleeping posture recognition, the average correct recognition rate is 93%, but only four sleeping postures are recognized and the recognition accuracy is low, which is difficult to achieve practical application.

SUMMARY OF THE INVENTION

The invention aims to overcome the shortcomings and shortcomings of the above sleeping posture monitoring methods, provides a non-interference and unconstrained sleeping posture monitoring method based on the fusion of sleeping posture features and artificial neural networks, improves the accuracy of the unconstrained sleeping posture recognition, and provides home furnishing services. Provide technical support for sleeping position recognition and monitoring in the environment, and realize early diagnosis and early prevention of respiratory diseases and pressure ulcers.

The technical scheme adopted by the present invention to solve the above problems is:

A sleeping posture monitoring method based on feature fusion and artificial neural network, the steps of the method are:

Step 1: Collection of sleeping posture images

The human body pressure data of six sleeping positions (supine, prone, right trunk type, right fetal type, left trunk type, and left fetal type) were collected using a sheet-type large-area pressure sensor array, and were reorganized and sorted. After converting into a two-dimensional sleeping position image, the coordinates of the pressure value in the two-dimensional sleeping position image correspond to the position of the sensor unit one-to-one, so as to effectively restore the direction and position of the human body lying on the sensor;

Step 2: Preprocessing of sleeping position images

Make the difference between the sleeping position image and the pressure data output by the sensor under no-load condition, eliminate the noise brought by the sensor itself, obtain the actual sleeping position pressure image, and perform histogram analysis on it. Sleeping posture segmentation and morphological de-noising methods are used to obtain preprocessed sleeping posture images with more distinct limb features;

Step 3: Feature extraction and fusion of images

Perform feature extraction on the sleeping posture image after the second step of preprocessing. First, extract the HOG feature of the image, divide the image into units according to the size of m × m pixels, and calculate its gradient size and direction, and then block according to the size of n × n units. Divide, take the block as the window sliding scan image to obtain the HOG feature F _hog ;

Then extract the GLCM features of the sleeping posture image after the second step of preprocessing, convert the 256 grayscale image into 8 grayscale levels, calculate the grayscale co-occurrence matrix of different directions θ and different distances d, and solve to get Contrast CON, correlation COOR, angular second-order matrix ASM, and inverse difference moment HOM are the values of four features, and the mean M and variance S of different angles at the same distance are obtained as the final GLCM features, and the formulas for each feature are respectively for:

in:

i and j are the grayscales of the pixels; p(i,j,d,theta) is the frequency of a pair of pixel grayscale values separated by a distance d in the direction of theta, i and j respectively; L is the grayscale of the image Level; suppose there are four directions, and the number of d values is k;

When d=1, M _CON1 is the mean value of contrast, and S _CON1 is the variance of contrast, then its specific formula is:

The formulas for the remaining three features are the same. When d=1, the vector set of the four features is:

M ₁ ={M _CON1 ,S _CON1 ,M _COOR1 ,S _COOR1 ,M _ASM1 ,S _ASM1 ,M _HOM1 ,S _HOM1 }(8)

Change the value of the distance d, and finally get the GLCM feature vector F _glcm whose expression is:

F _glcm ={M ₁ ,M ₂ ,...,M _k }(9)

Next, extract local features from the sleeping posture image after the second step of preprocessing: the local features are the sleeping posture area A, the number of leg areas N, the center coordinate point H of the head area, the center coordinate point H of the head area and the leg area. The angle α formed by the connection line _of the regional center coordinate point H1 and the horizontal direction;

The area A of the sleeping posture area is the number of pixels in the sleeping posture area. After preprocessing, the size of the sleeping posture image is 64×32, and the gray value of the background area is 255. According to the number of pixels other than 255 in the image, the sleeping posture can be calculated. The area of the posture area A;

Then, the edge is extracted from the image, and after obtaining the sleeping posture outline, the lower left corner of the preprocessed sleeping posture image is taken as the coordinate origin, the length direction of the bed is the Y direction, and the positive Y direction is defined as the direction from the legs to the head , establish a coordinate system with the width direction of the bed as the X direction, the X axis is the number of columns, the Y axis is the number of rows, and the area of Y∈[5764], X∈[032] is divided into the head area S _head , then the head area The formula for calculating the center coordinate point is:

Where x _min , x _max are the minimum and maximum values of the abscissa of the head profile, respectively; y _min , y _max are the minimum and maximum values of the ordinate of the head profile;

Divide the area of Y∈[016], X∈[032] into the leg area S _leg , and take the number of connected domains in this area as the number of leg areas N, then the coordinate H ₁ of the pressure center point of the leg area is The calculation formula is:

Among them, x _min1 and x _max1 are the minimum and maximum values of the abscissa of the leg area, respectively; y _min1 and y _max1 are the minimum and maximum values of the ordinate of the leg area.

According to the coordinate values of H and H ₁ obtained above, the calculation formula of the included angle α can be obtained as follows:

After the above features are extracted, a local feature set F _s ={A,N,H,α} is generated, and then the local feature set, HOG feature and GLCM feature are fused into a fusion feature vector F that reflects the six static pressure sleeping posture images, Its expression is:

F={F _hog ,F _glcm ,F _s } (13)

Step 4: Training of Sleeping Posture Classification Model

Using the algorithm based on artificial neural network to train the fusion feature vector of six static pressure sleeping position images to obtain the classification model of different sleeping positions;

Step 5: Real-time monitoring and recognition of sleeping position

Display the pressure data collected in real time on the front-end interface of the system, and repeat the first three steps to obtain the fusion feature vector of the current sleeping position image, and then use the classification model trained in the fourth step to continuously perform the fusion feature vector of the current sleeping position image. Type recognition of sleeping postures, generate log records of the recognition results, and realize long-term monitoring of human sleeping postures.

The specific process of the fourth step is: the first is to build a network structure based on artificial neural network; then the fusion feature vector extracted in the third step is used as the sleeping posture data set, and the labels of the six sleeping postures are set as y _i ={0, 1, 2, 3, 4, 5}, combine the feature vector and the corresponding label to get the training set of sleeping posture samples, which is used as the input of the neural network. After training, the classification models of different sleeping postures are obtained, and the recognition accuracy is If it reaches more than 99%, it will be saved as a classification operator and used directly for classification and recognition of sleeping postures.

The network structure of the artificial neural network includes an input layer, a hidden layer, and an output layer. The activation function of the hidden layer is the sigmoid function, and the function of the output layer is softmax; the number of hidden layer nodes is 100.

In HOG feature extraction, m=2, n=2, and the window sliding step is 1 unit; in GLCM feature extraction, k=10, and the four directions are: θ ₁ =0°, θ ₂ =45°, θ ₃ =90°, θ ₄ =135°.

The present invention also protects an application system of the above monitoring method. The system includes: a large-area pressure sensor, a data acquisition device, and a host computer terminal; the pressure sensor is directly laid on the mattress, and the coverage area of the pressure sensor is larger than that of the user. Horizontal and vertical width and height, and the user lies on the pressure sensor in the center; the pressure sensor and the acquisition device are connected through a cable, and the data acquisition device and the host computer terminal are connected through the USB interface; the above-mentioned trained classification is loaded on the host computer terminal Model, image preprocessing and feature fusion methods.

Compared with the prior art, the advantages of the present invention are mainly reflected in:

(1) Aiming at the characteristics of six types of sleeping postures, the present invention performs histogram analysis on the sleeping posture images, and adopts a combination of various image processing techniques to perform targeted preprocessing on the images to remove noise and improve image quality. At the same time, it effectively retains as much useful information as possible to prepare for the subsequent feature extraction and overall monitoring accuracy. Targeted preprocessing methods are used to obtain more complete and more obvious sleeping posture images.

(2) The present invention extracts a variety of features for the sleeping posture image, and the statistical features (HOG+GLCM) and local features (the area of the sleeping posture area, the number of leg areas, the angle α, etc.) are fused to generate the feature vector, which is It includes the overall differences of different sleeping position types, and also reflects the differences in details. Strengthening the characteristics between different sleeping positions can effectively improve the performance of the classification model and the recognition accuracy during training, and the increase in extraction methods will not cause algorithm overfitting or information interference problems.

(3) The present invention combines multi-feature fusion and artificial neural network to obtain higher recognition accuracy, reaching 99.17%, with better real-time performance. Experiments show that it only takes 0.13s to recognize 180 pictures. The invention directly collects the pressure data between the human body and the mattress to generate the sleeping posture image, the data processing time is short, the real-time performance of the sleeping posture recognition is improved, and the relationship model between the sleeping posture transformation and the dynamic pressure is established later.

(4) The present invention uses a large-area pressure sensor to collect the sleeping posture of the human body, and can collect the complete sleeping posture of the human body without restraining and interfering with the human body, which improves the practicability of the method for monitoring the sleeping posture in a home environment. It overcomes the problem that the existing sleeping posture recognition method needs to bind the sensor to the human body or use the camera, which has hidden privacy problems.

Description of drawings

Figure 1 is the composition diagram of the sleeping posture monitoring and recognition system; the numeral 1 is the pressure sensor, the 2 is the bed body, the 3 and 5 are the data acquisition equipment, the 4 is the host computer terminal, the 6 is the ordinary mattress, and the 7 is the user.

Figure 2(a) shows the original trunk-shaped sleeping posture map and its histogram on the left, the original sleeping posture map on the left, that is, the converted two-dimensional sleeping posture image, and the histogram on the right.

Figure 2(b) shows the actual sleeping position pressure map and its histogram obtained by subtracting the static pressure from the left trunk shape.

Figure 2(c) shows the sleeping position image and its histogram after the left trunk shape is inverted.

Figure 2(d) shows the sleeping position image and its histogram after partial equalization of the left trunk.

Figure 3(a) is the left tree trunk-type binarized sleeping image,

Figure 3(b) is the sleeping image after the left trunk-shaped segmentation,

Figure 3(c) is the sleeping position image after morphological filtering.

Figure 4(a) is a schematic diagram of the unit division and block division of the image in the HOG feature extraction,

Figure 4(b) is the extracted HOG feature map.

FIG. 5 is a flowchart of an artificial neural network training a sleeping posture classification model.

FIG. 6 is a flowchart of real-time monitoring and recognition of sleeping posture.

Detailed ways

In order to realize the above-mentioned functions of the present invention, the present invention will be further described below with reference to the accompanying drawings and examples. The embodiments of the present invention include the following examples, which are not intended to limit the protection scope of the present application.

The present invention is a sleeping posture monitoring method based on feature fusion and artificial neural network, which utilizes a pressure sensor to collect a sleeping posture data set, trains a classification model through a neural network algorithm, and realizes real-time monitoring and recognition of the sleeping posture, including the collection of sleeping posture images. The main steps are as follows:

Step 1: Collection of sleeping posture images

The human body pressure data of six sleeping positions (supine, prone, right trunk type, right fetal type, left trunk type, and left fetal type) were collected using a sheet-type large-area pressure sensor array, and were reorganized and sorted. Then, it is converted into a two-dimensional sleeping position image. As shown in Figure 2(a), the left side is an example image of the left trunk-shaped sleeping position image, and the right side is its histogram. The pixel value in the histogram is the abscissa, and the number is the vertical Coordinates, the coordinates of the pressure value in the image correspond to the position of the sensor unit one-to-one, which effectively restores the direction and position of the human body lying on the sensor.

Step 2: Preprocessing of sleeping position images

Make the difference between the sleeping position image and the pressure data output by the pressure sensor under no-load condition, eliminate the noise caused by the pressure sensor itself, obtain the actual sleeping position pressure image, and perform histogram analysis on it. , sleeping position segmentation, morphological denoising and other preprocessing methods to obtain sleeping position images with obvious features.

Taking the preprocessing process of the left trunk type collected in the first step as an example, assuming that when the sensor is under pressure, the pressure value is g(x,y); The pressure value p(x,y)=g(x,y)-f(x,y), the difference between the sleeping position image and the pressure data of the pressure sensor under no-load condition can eliminate the noise brought by the sensor itself, and get the actual The sleeping position pressure image (see Figure 2(b)). Then invert the image to visualize the gray or white details in the image (see Figure 2(c)); according to the histogram of the inverted image above, to determine the effective gray range of the sleeping posture area in the image, it can be known that the sleeping position The effective pressure value range of the posture image is 160-240, and the pixel values in the range are locally equalized to enhance the contrast between the sleeping posture area and the background (see Figure 2(d)). The formula is as follows:

Among them, P _r (r _u ) is the number of original image grayscale values of r _u , (r ₀ to r ₈₀ correspond to 160 to 240 one-to-one), and r _v is the processed pixel value. Then, the maximum inter-class variance method is used for binarization, and then the sleeping position is segmented to remove background noise, and the cracks in the image are bridged by morphological technology to eliminate the isolated noise, and obtain a sleeping position image with more distinct features of the limbs (see Figure 3(c)).

Step 3: Feature extraction and fusion of images

Perform feature extraction on the preprocessed sleeping position image in the second step. First, extract the HOG feature of the image. The size of the sleeping position image is 64×32 pixels. First, divide the image into 2×2 pixels and calculate its gradient size. and the direction, and then divide the image into blocks by 2×2 units, take the block as the window to slide, the step size is 1 unit (see Figure 4(a)), and scan the image to obtain the HOG feature F _hog (see Figure 4(b) ).

Then extract the GLCM features of the image, convert the preprocessed sleeping image in the second step from 256 gray levels to 8 gray levels, that is (0-255 corresponds to 1-8), and calculate different directions θ ₁ =0 °, θ ₂ =45°, θ ₃ =90°, θ ₄ =135° and the grayscale co-occurrence matrix of different distances d, and solve the contrast CON, correlation COOR, angular second-order matrix ASM, inverse difference moment HOM, a total of four For each feature, the mean M and variance S of the four eigenvalues at different angles at the same distance are taken as the final feature parameters. The formulas for each feature are:

in:

i and j are the grayscales of the pixels, p(i,j,d,theta) is the frequency of a pair of pixel grayscale values separated by a distance d in the θ direction, respectively, and L is the grayscale of the image grade. The preprocessed image is scanned, and when the distance d=1 is taken, the GLCM features in the four directions are calculated as shown in Table 1 below.

Table 1

angle Contrast related Angular second moment Inverse differential moment θ=0° 1.2087 0.8910 0.5976 0.9076 θ=45° 1.4424 0.8707 0.5881 0.9049 θ=90° 0.5392 0.9505 0.6218 0.9346 θ=135° 1.4619 0.8690 0.5903 0.9023

When d=1, M _CON1 is the mean value of contrast, S _CON1 is the variance of contrast, and its formula is:

The formulas for the remaining three features are the same. When d=1, the set of four features is:

M ₁ ={M _CON1 ,S _CON1 ,M _COOR1 ,S _COOR1 ,M _ASM1 ,S _ASM1 ,M _HOM1 ,S _HOM1 } (8)

Change the value of the distance d and set the value range to [1, 10], and finally get an 80-dimensional GLCM feature vector F _glcm , whose expression is as follows:

F _glcm = {M ₁ ,M ₂ ,...,M ₉ ,M ₁₀ } (9)

The next step is to calculate the local features in the sleeping position image: sleeping position area A, leg area number N, head area pressure center coordinate point H, head area pressure center coordinate point H and leg area pressure center point H The angle α formed by the line connecting ₁ and the horizontal direction.

First calculate the area of the sleeping posture area, the size of the preprocessed image is 64×32, and the gray value of the background area is 255, then the area A of the sleeping posture area can be calculated according to the number of pixels other than 255 in the image; Edge extraction, after obtaining the sleeping posture outline, take the lower left corner of the preprocessed sleeping posture image as the coordinate origin, take the length direction of the bed as the Y direction, and define the positive Y direction as the direction from the legs to the head, and take the width of the bed as the direction. The direction is the X direction to establish a coordinate system, the X axis is the number of columns, the Y axis is the number of rows, and the area of Y∈[5764], X∈[032] is divided into the head area S _head , then the pressure center coordinate point of the head area Calculated as follows:

where x _min and x _max are the minimum and maximum values of the abscissa of the head contour, respectively, and y _min and y _max are the minimum and maximum values of the ordinate of the head contour. Similarly, the area of Y∈[016], X∈[032] is divided into the leg area S _leg , and the number of connected domains in this area is taken as the number N of the leg area, then the pressure center point of the leg area _The formula for calculating the coordinate H1 is:

Among them, x _min1 and x _max1 are the minimum and maximum values of the abscissa of the leg area, respectively, and y _min1 and y _max1 are the minimum and maximum values of the ordinate of the leg area. When the above areas are divided, the size of the area should be larger than that of the feature area. Size, that is, the corresponding parts of the human body are always in this area under different sleeping positions;

According to the coordinates of H and H ₁ obtained above, the calculation formula of the included angle α can be obtained as follows:

After the above features are extracted, a local feature set F _s ={A,N,H,α} is generated,

It is fused with HOG feature and GLCM feature into a fused feature vector F that reflects six kinds of static pressure sleeping position images, and its expression is:

F={F _hog ,F _glcm ,F _s } (13)

The fused features not only include the overall differences in different sleeping position types, but also reflect the differences in details. Strengthening the characteristics between different sleeping positions can effectively improve the performance of the classification model and improve the recognition accuracy during training.

Step 4: Training of Sleeping Posture Classification Model

Before training, first collect a large number of samples of six sleeping positions through the first step, and process them in the second and third steps to obtain a fusion feature vector dataset of different sleeping position types; The labels of the postures (supine, prone, right trunk, right fetal, left trunk, left fetal) are respectively set to y _i ={0,1,2,3,4,5}. A part of the feature vectors and corresponding labels are combined to generate a training set of sleeping posture samples, and the rest of the samples are used as a test set to test the network performance. The function parameters for classifying sleeping positions are obtained through training.

Then a network structure based on artificial neural network is constructed. The network includes an input layer, a hidden layer, and an output layer. The activation function of the hidden layer is the sigmoid function, and the function of the output layer is softmax. Through the actual test to optimize the number of hidden layer nodes of the network, the results show that the best effect is when the number of nodes is 100. After setting the parameters, the data set obtained before is used to start training.

During training, the samples in the training set are used as the input of the neural network, the cross entropy is used as the loss function to continuously optimize the weight and threshold of the network, and the recognition accuracy of the test set is used to judge the performance of the classification model. The sleeping posture classification model has a recognition accuracy of 99.17%, and it only takes 0.13s to recognize 180 pictures, which is saved as a classification operator and used directly for classification and recognition of sleeping postures (see Figure 5).

Step 5: Real-time monitoring and recognition of sleeping position

Display the pressure data collected in real time on the front-end interface of the system, and repeat the first three steps to obtain the fusion feature vector of the current sleeping position image, and then use the classification model trained in the fourth step to continuously perform the fusion feature vector of the current sleeping position image. Type identification of sleeping postures, generate log records of identification results, and realize long-term monitoring of human sleeping postures (see Figure 6).

The system to which the method is applied (see FIG. 1 ) includes: a large-area pressure sensor 1 , data acquisition devices 3 and 5 , and a host computer terminal 4 . When in use, the pressure sensor 1 is directly laid on the mattress 6, the user 7 lies directly on it, the user lies down, the coverage area of the pressure sensor is larger than the horizontal and vertical width and height of the user, and the user lies in the center, The large-area pressure sensor used in this application can be an entire 64*32 flexible pressure sensor array, and the pressure sensor acquires data through a data acquisition device and transmits it to the host computer terminal. The invention is mainly aimed at adults, especially the elderly. The height of the experimenter is about 1.6-1.8m. When the user is lying down, the head area is located in the area of Y∈[5764] and X∈[032] of the sensor array. The leg region is located in the region of Y ∈ [016], X ∈ [032].

In order to collect the complete sleeping posture of the human body under the condition of high acquisition rate and low noise, the pressure sensor is made of two 32*32 flexible pressure sensors. After processing and identification, the above-mentioned trained classification operators, image preprocessing and feature fusion methods are loaded on the upper computer terminal.

The basic principles of the method for monitoring sleeping posture based on feature fusion and artificial neural network in the present invention are described above in conjunction with specific embodiments. Realize sleep posture monitoring.

What is not described in the present invention applies to the prior art.

The implementation of the present invention has been described in detail above, which is only a preferred implementation process of the present invention, and cannot be considered to limit the protection scope of the claims of the present application. All equivalent changes and improvements made within the scope of the claims of the present application shall fall within the protection scope of the present application.

Claims (5)

1. a sleeping posture monitoring method based on feature fusion and artificial neural network, the steps of the method are: Step 1: Collection of sleeping posture images The human body pressure data of six sleeping positions (supine, prone, right trunk type, right fetal type, left trunk type, and left fetal type) were collected using a sheet-type large-area pressure sensor array, and were reorganized and sorted. After converting into a two-dimensional sleeping position image, the coordinates of the pressure value in the two-dimensional sleeping position image correspond to the position of the sensor unit one-to-one, so as to effectively restore the direction and position of the human body lying on the sensor; Step 2: Preprocessing of sleeping position images Make the difference between the sleeping position image and the pressure data output by the sensor under no-load condition, eliminate the noise brought by the sensor itself, obtain the actual sleeping position pressure image, and perform histogram analysis on it. Sleeping posture segmentation and morphological de-noising methods are used to obtain preprocessed sleeping posture images with more distinct limb features; Step 3: Feature extraction and fusion of images Perform feature extraction on the sleeping posture image after the second step of preprocessing. First, extract the HOG feature of the image, divide the image into units according to the size of m × m pixels, and calculate its gradient size and direction, and then block according to the size of n × n units. Divide, take the block as the window sliding scan image to obtain the HOG feature F _hog ; Then extract the GLCM features of the sleeping posture image after the second step of preprocessing, convert the 256 grayscale image into 8 grayscale levels, calculate the grayscale co-occurrence matrix of different directions θ and different distances d, and solve to get Contrast CON, correlation COOR, angular second-order matrix ASM, and inverse difference moment HOM are the values of four features, and the mean M and variance S of different angles at the same distance are obtained as the final GLCM features, and the formulas for each feature are respectively for:

in:

i and j are the grayscales of the pixels; p(i,j,d,theta) is the frequency of a pair of pixel grayscale values separated by a distance d in the direction of theta, i and j respectively; L is the grayscale of the image Level; suppose there are four directions, and the number of d values is k; When d=1, M _CON1 is the mean value of contrast, and S _CON1 is the variance of contrast, then its specific formula is:

The formulas for the remaining three features are the same. When d=1, the vector set of the four features is: M ₁ ={M _CON1 ,S _CON1 ,M _COOR1 ,S _COOR1 ,M _ASM1 ,S _ASM1 ,M _HOM1 ,S _HOM1 } (8) Change the value of the distance d, and finally get the GLCM feature vector F _glcm whose expression is: F _glcm = {M ₁ , M ₂ ,...,M _k } (9) Next, extract local features from the sleeping posture image after the second step of preprocessing: the local features are the sleeping posture area A, the number of leg areas N, the center coordinate point H of the head area, the center coordinate point H of the head area and the leg area. The angle α formed by the connection line _of the regional center coordinate point H1 and the horizontal direction; The area A of the sleeping posture area is the number of pixels in the sleeping posture area. After preprocessing, the size of the sleeping posture image is 64×32, and the gray value of the background area is 255. According to the number of pixels other than 255 in the image, the sleeping posture can be calculated. The area of the posture area A; Then, the edge is extracted from the image, and after obtaining the sleeping posture outline, the lower left corner of the preprocessed sleeping posture image is taken as the coordinate origin, the length direction of the bed is the Y direction, and the positive Y direction is defined as the direction from the legs to the head , establish a coordinate system with the width direction of the bed as the X direction, the X axis is the number of columns, the Y axis is the number of rows, and the area of Y∈[5764], X∈[032] is divided into the head area S _head , then the head area The formula for calculating the center coordinate point is:

Where x _min , x _max are the minimum and maximum values of the abscissa of the head profile, respectively; y _min , y _max are the minimum and maximum values of the ordinate of the head profile; Divide the area of Y∈[016], X∈[032] into the leg area S _leg , and take the number of connected domains in this area as the number of leg areas N, then the coordinate H ₁ of the pressure center point of the leg area is The calculation formula is:

Among them, x _min1 and x _max1 are the minimum and maximum values of the abscissa of the leg area, respectively; y _min1 and y _max1 are the minimum and maximum values of the ordinate of the leg area. According to the coordinate values of H and H ₁ obtained above, the calculation formula of the included angle α can be obtained as follows:

After the above features are extracted, a local feature set F _s ={A,N,H,α} is generated, and then the local feature set, HOG feature and GLCM feature are fused into a fusion feature vector F that reflects the six static pressure sleeping posture images, Its expression is: F={F _hog ,F _glcm ,F _s } (13) Step 4: Training of Sleeping Posture Classification Model Using the algorithm based on artificial neural network to train the fusion feature vector of six static pressure sleeping position images to obtain the classification model of different sleeping positions; Step 5: Real-time monitoring and recognition of sleeping position Display the pressure data collected in real time on the front-end interface of the system, and repeat the first three steps to obtain the fusion feature vector of the current sleeping position image, and then use the classification model trained in the fourth step to continuously perform the fusion feature vector of the current sleeping position image. Type recognition of sleeping postures, generate log records of the recognition results, and realize long-term monitoring of human sleeping postures. 2. monitoring method according to claim 1 is characterized in that, the concrete process of the 4th step is: at first is to build the network structure based on artificial neural network; Then use the fusion feature vector that the 3rd step extracts as sleeping posture data set , the labels of the six sleeping postures are respectively set as y _i ={0,1,2,3,4,5}, and the feature vector and the corresponding label are combined to obtain the training set of sleeping posture samples, which is used as the input of the neural network , after training, the classification model of different sleeping positions is obtained, and the recognition accuracy rate reaches more than 99%, which is saved as a classification operator and directly used for classification and recognition of sleeping positions. 3. monitoring method according to claim 2 is characterized in that, the network structure of artificial neural network comprises an input layer, a hidden layer, an output layer, the activation function of the hidden layer is a sigmoid function, and the function of the output layer is a softmax ; The number of hidden layer nodes is 100. 4. The monitoring method according to claim 1, characterized in that, during HOG feature extraction, m=2, n=2, and the window sliding step is 1 unit; during GLCM feature extraction, k=10, four directions are: θ ₁ =0°, θ ₂ =45°, θ ₃ =90°, θ ₄ =135°. 5. An application system of the monitoring method according to any one of claims 1-4, the system comprising: a large-area pressure sensor, a data acquisition device and an upper computer terminal; the pressure sensor is directly laid on the mattress, and the The coverage area is larger than the horizontal and vertical width and height of the user, and the user lies in the center on the pressure sensor; the pressure sensor and the acquisition device are connected by cables, and the data acquisition device and the host computer terminal are connected through the USB interface; Load the above-trained classification model, image preprocessing, and feature fusion methods.

Images