CN115841651B - Constructor intelligent monitoring system based on computer vision and deep learning - Google Patents

Abstract

The invention discloses an intelligent monitoring system for construction personnel based on computer vision and deep learning, which includes an intelligent statistical module and an intelligent monitoring module. The intelligent statistical module includes a video acquisition module, a deep learning algorithm analysis module, and an analysis result display module. Cap recognition module, overalls recognition module, state recognition module and alarm reminder module. The present invention can not only realize the intelligent statistics of the number of personnel and vehicles entering and exiting the construction site and the total number of personnel and vehicles in the local area, but also use image recognition technology to identify the clothing and status of construction personnel passing through the entrance of the construction site, and Construction workers who do not meet the standards will be reminded, which can effectively avoid the occurrence of safety accidents caused by construction workers not wearing safety helmets, work clothes or poor mental state, thereby effectively improving the construction safety performance of the construction site .

Description

Intelligent monitoring system for construction workers based on computer vision and deep learning

technical field

The invention relates to the technical field of intelligent statistics, in particular to an intelligent monitoring system for construction personnel based on computer vision and deep learning.

Background technique

At present, in the process of building construction, the construction site is required to implement closed management in principle, and an access control system for entry and exit is set up. At present, most of the access control systems used in construction sites are systems composed of tripod gates, swing gates, wing gates, stainless steel fence gates and other channel systems combined with proximity card readers. After the second-generation ID card is entered into the system, they can enter and exit freely, and personnel without registration and authorization cannot enter the construction site without authorization. At the same time, surveillance cameras are installed at the entrance and exit of the construction site, and video recordings for a period of time are stored in the local server for easy reference.

However, the current access control system has the following defects in actual application:

1) People come in and out frequently at the entrance and exit of the construction site, people flow out of order, and on-site management is difficult; 2) Using paper records to record attendance data makes it difficult to accurately count actual working hours, and the data is easy to be tampered with; 3) There are many people on the construction site and subcontracting Due to the characteristics of many units, types of work and positions, the project management personnel cannot clearly and timely grasp the number of construction workers on the site, the number of personnel of each type of work and the number of subcontractors of each specialty, which is not conducive to improving the efficiency of on-site management; 4) The occurrence of construction labor personnel In the case of wage disputes, it is difficult for the supervisory department to obtain evidence, and it is difficult to defend rights; 5) It is difficult to count the entry and exit of vehicles and personnel in vehicles.

Aiming at the problems in the related technologies, no effective solution has been proposed yet.

Contents of the invention

Aiming at the problems in the related technologies, the present invention proposes an intelligent monitoring system for construction personnel based on computer vision and deep learning to overcome the above-mentioned technical problems in the existing related technologies.

For this reason, the concrete technical scheme that the present invention adopts is as follows:

An intelligent monitoring system for construction workers based on computer vision and deep learning, which includes an intelligent statistics module and an intelligent monitoring module;

Wherein, the intelligent statistical module is used to detect the construction workers passing through the entrance of the construction site by using the trained deep learning algorithm, and track the target through the tracking algorithm, count and count when the target collides with the detection line, and at the same time through the client display in real time;

The intelligent monitoring module is used to use preset image recognition technology to identify the clothing and status of the construction personnel passing through the entrance of the construction site, and remind the construction personnel who do not meet the standards.

Further, the intelligent statistics module includes a video acquisition module, a deep learning algorithm analysis module and an analysis result display module;

The video acquisition module is used to collect real-time monitoring pictures according to the monitoring cameras erected at each entrance and exit of the construction site, and input the real-time monitoring pictures into the POE switch to obtain initial video material after conversion;

The deep learning algorithm analysis module is used to output in real time the video analysis picture corresponding to the initial video material and the personnel statistics data through the trained deep learning algorithm, and output the analysis picture and statistical results to the client;

The analysis result display module is used for displaying video analysis pictures and statistical data of personnel and vehicles on the client side, and is also used for managers to switch analysis pictures and statistical data obtained by different entrance and exit cameras according to requirements.

Further, the deep learning algorithm analysis module includes a detection area setting module, a video analysis module, a data statistics module and a data transmission module;

The detection area setting module is used to control the actual detection range in each screen by adjusting the corresponding detection range according to the screen layout of different entrances and exits, so as to realize the setting of the detection area;

The video analysis module is used to analyze the image in the initial video material through the coordinates of the line collision detection point, so as to realize the analysis and statistics of the personnel and vehicles in the detection area;

The data statistics module is used to realize the judgment and statistics of target entry and exit through whether the target frame hits the line and the color of the target frame's line-crossing area;

The data transmission module is used to output the analysis screen and statistical results to the client through the RTSP server and the HTTP push service.

Further, the setting of the detection area includes the following steps:

According to the shape of the required area, the positions of the endpoints are sequentially determined and expressed by an array, and each element of the array is a binary array representing the endpoint of the graph. The adjustment and setting of the detection area is realized through the adjustment of the array.

Further, the analysis of the image in the initial video material through the coordinates of the line collision detection point to realize the analysis and statistics of the personnel and vehicles in the detection area includes the following steps:

Obtain the position parameters and categories of all objects in the current image;

Determine whether the geometric center of an object in the new frame of image is within the preset offset from the geometric center of an object in the previous frame of image. If so, determine that the two objects are the same object and have the same ID. If not, it is determined that there is a new object in the new frame image, and a new ID is assigned to the object;

A rectangle is used to represent the image of the object range and the object ID is known. Set the parameters of an object range as x ₁ , y ₁ , x ₂ , y ₂ , where x ₁ <x ₂ , y ₁ <y ₂ , and the line collision detection point The coordinates are (check_point_x, check_point_y), check_point_x=x ₁ , check_point_y=int[y ₁ +(y ₂ -y ₁ )*0.6], int refers to the rounding of the operation result;

It is judged whether the object collision line detection point is located in the judgment area, if yes, the statistical calculation is performed on the object, and if not, the statistical calculation is not performed.

Further, the determination and statistics of the target entry and exit through whether the target frame hits the line and the color of the target frame's line-crossing area include the following steps:

It is stipulated that all the pictures captured by the current camera are the detection area, and the blue and yellow bar areas are set as the judgment area, and when the target hits the yellow line up, it is recorded as entering, and when the target hits the blue line down, it is recorded as out;

Obtain the real-time monitoring pictures collected by the monitoring cameras at the entrances and exits of the construction site, and reduce the size of the acquired real-time monitoring pictures;

Judging whether there is a target in the detection area of the reduced real-time monitoring screen, if not, the monitoring screen is regarded as an invalid screen and ignored and cleaned up, if so, it is output after the target is framed;

Detect whether the target frame hits the line and the color of the target frame's line-colliding area, and judge and count the entry and exit of the target according to the color of the target frame's line-colliding area.

Further, the intelligent monitoring module includes a safety helmet identification module, a work clothes identification module, a state identification module and an alarm reminder module;

The safety helmet identification module is used to identify construction personnel who pass through the entrance of the construction site without wearing safety helmets by using image recognition technology;

The work clothes identification module is used to use image recognition technology to identify construction workers who pass through the entrance of the construction site without wearing work clothes;

The state identification module is used to identify the mental state of construction workers passing through the entrance of the construction site by using image recognition technology;

The alarm reminder module is used to remind construction workers who do not wear safety helmets, work clothes and whose mental state does not meet the standards.

Further, the state recognition module includes a construction personnel facial image acquisition module, a fusion feature recognition module and a state recognition result output module;

The facial image acquisition module of the construction personnel is used to obtain the facial images of the construction personnel in the real-time monitoring screen of each entrance and exit of the construction site;

The fusion feature recognition module is used to identify the facial image of the construction personnel by using the facial state recognition algorithm based on independent feature fusion, so as to realize the identification of the mental state of the construction personnel entering the construction site;

The state recognition result output module is used to output the mental state information of the construction workers in the real-time monitoring screen of each entrance and exit of the construction site.

Further, the fusion feature recognition module includes a global state feature extraction module, a local state feature extraction module, a state feature fusion module, and a state feature analysis and recognition module;

The global state feature extraction module is used to extract the global state feature of the facial image by discrete cosine transform, and utilizes the independent component analysis technique to remove the correlation of the global state feature to obtain the independent global state feature;

The local state feature extraction module is used to extract the features of the eye and mouth regions in the image sequence, and perform Gabort wavelet transform and feature fusion on the eye and mouth regions respectively to obtain the dynamic multi-scale features of the two local regions as Local state features of facial images;

The state feature fusion module is used to fuse independent global state features and local state features, and add local detail information to the global features to obtain facial state fusion features;

The state feature analysis and identification module is used to analyze and identify the obtained facial state fusion features through a preset classifier, and obtain the mental state information of the construction personnel, wherein the mental state of the construction personnel includes awake state, mild Fatigue state, moderate fatigue state and severe fatigue state.

Further, the preset classifier is obtained by selecting some features through the AdaBoost algorithm, removing redundant features and performing training, and the calculation formula of the preset classifier is:

Among them, _T represents the final number of algorithm loops, at represents the selected weight of the classifier h _t (X), which is determined by AdaBoost algorithm learning, X=(X ₁ , X ₂ ,...,X _T ) represents the selected Dynamic Gabor features for facial image sequences.

The beneficial effects of the present invention are:

1) By using the trained deep learning algorithm to detect the construction personnel passing the entrance of the construction site, and using the tracking algorithm to track the target, count and count when the target collides with the detection line, so that the construction site personnel and vehicles can be monitored , the number of exits, and the intelligent statistics of the total number of personnel and vehicles in the local area. In addition, the present invention can also use image recognition technology to identify the clothing and status of construction personnel passing through the entrance of the construction site, and remind construction personnel that do not meet the standards. In this way, the occurrence of safety accidents caused by construction personnel not wearing safety helmets, work clothes or poor mental state can be effectively avoided, and the construction safety performance of the construction site can be effectively improved.

2) The present invention can not only achieve a recognition accuracy rate of more than 95% under various lighting conditions, but also ensure that the statistical data is accurate, so that the time difference between the analysis screen and the original screen does not exceed 1 second, and has a high recognition accuracy rate And the advantages of fast recognition speed.

3) Based on the algorithm used in the present invention and the plasticity contained in the UI, different functions can be added according to different needs, such as face recognition, type of work recognition, etc., and the scalability is high; in addition, the hardware equipment required by the present invention is cheap , Improve on-site management efficiency and obtain more economic benefits while saving costs.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required in the embodiments. Obviously, the accompanying drawings in the following description are only some of the present invention. Embodiments, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

Fig. 1 is a structural block diagram of an intelligent monitoring system for construction personnel based on computer vision and deep learning according to an embodiment of the present invention;

2 is a structural block diagram of a deep learning algorithm analysis module in an intelligent monitoring system for construction personnel based on computer vision and deep learning according to an embodiment of the present invention;

3 is a structural block diagram of a state recognition module in an intelligent monitoring system for construction personnel based on computer vision and deep learning according to an embodiment of the present invention;

Fig. 4 is a structural block diagram of a fusion feature recognition module in an intelligent monitoring system for construction workers based on computer vision and deep learning according to an embodiment of the present invention.

In the picture:

1. Intelligent statistics module; 2. Intelligent monitoring module; 11. Video acquisition module; 12. Deep learning algorithm analysis module; 121. Detection area setting module; 122. Video analysis module; 123. Data statistics module; 124. Data transmission module; 13. analysis result display module; 21. safety helmet recognition module; 22. work clothes recognition module; 23. state recognition module; 231. construction personnel facial image acquisition module; module; 2322, local state feature extraction module; 2323, state feature fusion module; 2324, state feature analysis and recognition module; 233, state recognition result output module; 24, alarm reminder module.

Detailed ways

In order to further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention, and are mainly used to illustrate the embodiments, and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments, for reference Those of ordinary skill in the art should be able to understand other possible implementations and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

According to an embodiment of the present invention, an intelligent monitoring system for construction personnel based on computer vision and deep learning is provided.

The present invention will be further described in conjunction with the accompanying drawings and specific embodiments. As shown in FIGS. Monitoring module 2;

The intelligent statistical module 1 is used to detect the construction workers passing through the entrance of the construction site by using the trained deep learning algorithm, and track the target through the tracking algorithm, and count and count when the target collides with the detection line, and at the same time through the client. real-time display;

Wherein, the intelligent statistical module 1 includes a video acquisition module 11, a deep learning algorithm analysis module 12 and an analysis result display module 13;

Described video collection module 11 is used for collecting real-time monitoring picture according to the monitoring camera erected at each entrance and exit of the construction site, and inputting the real-time monitoring picture into the POE switch to obtain initial video material after conversion;

The deep learning algorithm analysis module 12 is used to output in real time the video analysis picture corresponding to the initial video material and the personnel statistics data through the trained deep learning algorithm, and output the analysis picture and statistical results to the client;

Among them, the backbone feature extraction network used in the algorithm of this embodiment is better than the cross-level partial network, which reduces memory consumption, and further widens the network while enhancing the learning ability of the convolutional neural network to ensure the accuracy of the algorithm. In the training strategy of the backbone feature extraction network, in order to ensure the recognition accuracy in complex environments, multiple images are stitched together to simulate objects in complex environments. This algorithm will also change the image to make another decision, improve the weak environment of the decision boundary, and improve the robustness of the system. The algorithm recognizes objects in the following three steps:

First, the image is sampled 5 times with a convolutional layer with a stride of 2 and a convolution kernel of 3×3 to extract the main features of the image and generate five feature layers of different sizes.

Secondly, the multi-scale receptive field fusion of the smaller size feature layer is performed and the maximum pooling is performed, and then the parameters of the processed small size feature map and the larger size map are aggregated through tensor splicing. Therefore, this algorithm is still applicable under different size detection.

Finally, the feature layer is divided into grids of different sizes, the purpose is to detect various targets with large differences in size and avoid target loss. Then generate multi-standard a priori boxes on grids of different sizes, and each a priori box returns the probability that the object is a preset object category, and then use the a priori box with the maximum confidence among multiple a priori boxes as the actual position of the object, Returns the confidence and the actual position of the object for all preset object categories.

Specifically, the confidence interval is set manually, and the confidence is given after the algorithm recognizes the object. In the test, the lowest confidence value is 0.3, and the confidence interval is (0.3, 1). Under this interval, because the types of objects to be detected are relatively single and the natural environment changes little, the recognition accuracy is high. The optimal confidence interval is greatly affected by environmental factors, so there is no optimal confidence interval applicable to different environments, and the confidence interval should be adjusted several times in the actual environment to obtain the optimal confidence interval under the corresponding conditions.

Specifically, the deep learning algorithm analysis module 12 includes a detection area setting module 121, a video analysis module 122, a data statistics module 123 and a data transmission module 124;

The detection area setting module 121 is used to control the actual detection range in each screen by adjusting the corresponding detection range according to the screen layouts of different entrances and exits, so as to realize the detection area (the detection area includes the whole screen range, and the judgment area only includes Blue and yellow bar area, if a person appears in the screen but does not pass through the blue and yellow bar area, the person will be captured but will not be included in the statistics);

In this embodiment, it is found during the on-site test that it is necessary to adjust the detection range of video images in different areas. Generally speaking, all the pictures captured by the camera will be recognized as the detection area by default. However, in practical applications, it is often not necessary for the entire monitoring picture to participate in the recognition. The passage screen mainly includes the turnstile passage and the guard duty room on the right. By default, the algorithm will identify and detect people in the entire screen. If someone walks back and forth in the duty room, it will also be judged by the algorithm as "entering" or "exiting". This misjudgment will give the system statistics The result is a huge error. Therefore, during the landing process of this system on the project site, it is necessary to adjust the detection range of the corresponding background algorithm according to the screen layout of different entrances and exits. By setting the detection area in the algorithm code, the actual detection range in each screen is controlled. For any point on the image, it can be determined by a binary array. If you need to set a detection area, you can determine the positions of the endpoints in sequence according to the shape of the required area. At this time, the detection area is expressed by an array. Each element of the array is a binary array representing the endpoint of the graph, so the array can be used to achieve the purpose of adjusting the geometric area. All people entering the channel screen will be identified, but only those who pass through the detection area in the picture will be included in the statistical data, and those who move outside the detection area will not have an impact on the statistical data.

The video analysis module 122 is used to analyze the image in the initial video material through the coordinates of the line collision detection point, so as to realize the analysis and statistics of personnel and vehicles in the detection area;

After the algorithm in this embodiment is initialized, it can identify and give the object category and range in the image. The algorithm uses a rectangular frame to determine the position of the image. In a plane, only one diagonal line is needed to determine the position of a rectangle. Coordinates, such as the upper right vertex coordinate value (x ₁ , y ₁ ) and the lower left vertex coordinate value (x ₂ , y ₂ ), can determine the position of the rectangle in the image, so the algorithm only needs to return these four parameters with object class. The four parameters returned by the algorithm are used to determine the line collision detection point and draw the rectangle. After obtaining the position parameters and categories of all objects in the current image, if the offset between the geometric center of an object in the new frame image and the geometric center of an object in the previous frame image is within the set offset, the two objects Treated as the same object with the same ID. If there is a new object in the new frame, assign a new ID to the object. After the above processing, an image in which the range of the object is represented by a rectangle and the ID of the object is known is obtained. Now assume that the parameters of an object range are x ₁ , y ₁ , x ₂ , y ₂ , where x ₁ <x ₂ , y ₁ <y ₂ , the coordinates of the line collision detection point are (check_point_x, check_point_y), check_point_x=x ₁ , check_point_y=int[y ₁ +(y ₂ -y ₁ )*0.6], int means to round the operation result. The algorithm will perform statistical calculations only when an object's line collision detection point is within the judgment area, otherwise it will not perform statistical calculations.

The data statistics module 123 is used to realize the judgment and statistics on the entry and exit of the target through whether the target frame hits the line and the color of the target frame's line-crossing area;

In this embodiment, a more intuitive and accurate determination method is adopted when implementing the personnel counting function. That is, the judgment area is preset in the algorithm, and the blue and yellow bar areas are preset as the judgment area. When the target goes up and hits the yellow line, it is recorded as entering, and when the target goes down and hits the blue line, it is recorded as out, so as to calculate the number of pedestrians entering and exiting. When the algorithm is running, the video will be processed. First, reduce the size and then detect whether there is a target in the picture. If there is no target in the picture, the algorithm will treat it as an invalid picture and ignore it and clean it up. When there is a target in the picture, The algorithm will assign a frame to the target and output it, and finally detect whether the target frame hits the line and whether the area hit by the target frame is blue or yellow as the basis for judging the target's entry and exit.

The data transmission module 124 is used to output the analysis screen and statistical results to the client through the RTSP server and the HTTP push service.

After the video material is recognized and analyzed by the deep learning algorithm, the analysis screen needs to be output to the client. In order to minimize the delay between the real-time monitoring screen and the analysis screen displayed in the client, the transmission method used in this embodiment For RTSP server, RTSP (Real Time Streaming Protocol) real-time streaming protocol, when using RTSP, both the client and the server can send requests, that is to say, RTSP can be bidirectional, and the server that provides the service can be processed according to the actual load situation. Switching to avoid excessive load concentration on the same server causing delays. First write the video stream interface that can support the RTSP protocol in the algorithm, and then obtain the RTSP stream according to the camera's IP address, port number, device user name and password and other information. Furthermore, the analysis screen can be transmitted to the client by inputting the flow address into the interface preset by the algorithm.

Similar to outputting real-time analysis screens, the statistically obtained data in the deep learning algorithm also needs to be transmitted to the client in real time. In this embodiment, HTTP is used to realize data transmission. HTTP (Hyper Text Transfer Protocol) is the Hypertext Transfer Protocol, which has the advantages of being simple, flexible and easy to expand. This embodiment first writes the HTTP program in the deep learning algorithm on the server side, then creates a TCP connection from the client to the server and writes the data request message in the client. When the client sends the message, an HTTP program is realized. ask. After receiving the request, the server will organize the response according to the request content and multiplex the TCP connection to report the response content to the client, parse and read the response content in the client, and finally display it on the client user interface in real time.

The analysis result display module 13 is used to display video analysis images and statistical data of personnel and vehicles on the client terminal, and is also used for managers to switch analysis images and statistical data obtained by different entrance and exit cameras according to requirements.

In order to present the real-time video analysis screen more intuitively to the on-site management personnel and facilitate the on-site management personnel to obtain the analysis screens of different entrances and exits according to the needs, this embodiment combines the deep learning algorithm with the Unreal 4 engine, and adds RTSP and HTTP to the algorithm Streaming service, which displays the obtained video analysis screen and people statistics results on the client in real time, and adds other functions to the client.

The functions contained in the client mainly include:

(1) Personnel and vehicle statistics function, including the number of personnel and vehicles entering and exiting, and the total number of personnel and vehicles in the local area;

(2) Angle of view switching function. When multiple cameras are set up at different entrances and exits, the client can switch between different cameras and obtain the personnel and vehicle entry and exit data recorded by the camera in real time.

The intelligent monitoring module 2 is used to identify the clothing and state of the construction personnel passing through the entrance of the construction site by using the preset image recognition technology, and remind the construction personnel who do not meet the standards.

Wherein, the intelligent monitoring module 2 includes a helmet identification module 21, a work clothes identification module 22, a state identification module 23 and an alarm reminder module 24;

The safety helmet identification module 21 is used to identify construction personnel who pass through the entrance of the construction site without wearing safety helmets by using image recognition technology;

The work clothes recognition module 22 is used to use image recognition technology to identify construction workers who pass through the site entrance without wearing work clothes;

The state identification module 23 is used to identify the mental state of construction workers passing through the site entrance by using image recognition technology;

Specifically, the state recognition module 23 includes a construction personnel facial image acquisition module 231, a fusion feature recognition module 232 and a state recognition result output module 233;

The facial image acquisition module 231 of the construction personnel is used to obtain the facial images of the construction personnel in the real-time monitoring screen of each entrance and exit of the construction site;

The fusion feature identification module 232 is used to identify the facial image of the construction personnel using the facial state identification algorithm based on independent feature fusion, so as to identify the mental state of the construction personnel entering the construction site;

The fusion feature identification module 232 includes a global state feature extraction module 2321, a local state feature extraction module 2322, a state feature fusion module 2323 and a state feature analysis and identification module 2324;

The global state feature extraction module 2321 is used to extract the global state feature of the face image by discrete cosine transform (discrete cosinetransform, DCT), and utilize independent component analysis (independent component analysis, ICA) technology to remove the correlation of the global state feature, Get independent global state features;

Discrete cosine transform is a commonly used image data compression method. For an MxN digital image (x, y), its 2D discrete cosine transform is defined as:

In the formula, u=0, 1, 2,..., M-1; v=0, 1, 2,..., N-1;

The characteristics of discrete cosine transform are: when the frequency domain change factors u and v are large, the value of DCT coefficient C(u, v) is very small; while the value of C(u, v) with large value is mainly distributed when u and v are relatively large. The small upper left corner area is also a concentrated area of useful information. In this embodiment, the useful information in this area is extracted as the global fatigue feature of the image.

As an effective method to solve the problem of blind signal separation, independent principal component analysis can successfully separate the independent source signals from the mixed signal through the transformation matrix through ICA. This property can not only reduce the fatigue feature extraction The dimensionality of the fatigue eigenvector can also reduce the high-order correlation among the components in the eigenvector.

In addition to the second-order statistical correlation that can be removed by the PCA method in the facial expression image, the high-order statistical correlation also occupies a large component. Therefore, using the ICA method to remove the high-order correlation in the facial expression image will result in a more discriminative ability. Characteristics. The basic idea of the ICA algorithm is to use a set of basis functions to represent a series of random variables, while assuming that the components are statistically independent or as independent as possible.

In this embodiment, ICA is used to successfully separate independent features from the global DCT features of the facial image sequence through the transformation matrix, which can not only reduce the dimension of the feature vector, but also reduce the high-order correlation between the components in the feature vector characteristics to obtain more discriminative independent global features.

The local state feature extraction module 2322 is used to extract the features of the eye and mouth regions in the image sequence, and perform Gabort wavelet transform and feature fusion on the eye and mouth regions respectively to obtain the dynamic multi-scale features of the two local regions as local state features of facial images;

The performance characteristics of facial fatigue have different scales, there are overall larger scales and subtle smaller scales, so it is difficult to extract all the important features of facial fatigue expressions by single-scale analysis. However, using multi-scale Decomposition, extraction of features on scales and directions that contain more fatigue information for analysis, can better analyze facial visual information effectively. For facial video image sequences, due to the different scales of different facial movements during fatigue, it is necessary to use A multiscale approach to analyzing fatigue information.

Gabor wavelet is a powerful tool for multi-scale analysis. Compared with DCT, Gabor transform obtains the best localization in time domain and frequency domain at the same time. The advantages of insensitivity to illumination, position, etc., are suitable for representing the local features of the face, and because DCT pays more attention to the global information of the image, it often ignores the more important local information in the process of face recognition. Therefore, this implementation In the example, the Gabor wavelet transform is introduced to extract the local multi-scale features of the image, and it is fused with the independent global features, and the local features of the image are added to the global information.

The state feature fusion module 2323 is used to fuse independent global state features and local state features, and add local detail information to the global features to obtain facial state fusion features;

The state feature analysis and identification module 2324 is used to analyze and identify the obtained facial state fusion features through a preset classifier to obtain the mental state information of the construction workers, wherein the mental state of the construction workers includes the awake state (eye Normal opening, active eyeballs, straight head, concentrated attention, and flat eyebrows), mild fatigue (decreased eyeball activity, dull eyes, drooping eyebrows, forehead wrinkling, increased head turning frequency, mental fatigue) Sluggishness), moderate fatigue state (eye closure, yawning, nodding, etc., severe drooping of eyebrows, severe facial muscle deformation) and severe fatigue state (eye closure tends to aggravate, and persistent eye closure occurs, and concentration is lax) .

The preset classifier is obtained by selecting some features through the AdaBoost algorithm, removing redundant features and performing training. The calculation formula of the preset classifier is:

Among them, _T represents the final number of algorithm loops, at represents the selected weight of the classifier h _t (X), which is determined by AdaBoost algorithm learning, X=(X ₁ , X ₂ ,...,X _T ) represents the selected Dynamic Gabor features for facial image sequences.

The state recognition result output module 233 is used to output the mental state information of the construction workers in the real-time monitoring screen of each entrance and exit of the construction site.

The alarm reminding module 24 is used to remind construction workers who do not wear safety helmets, do not wear work clothes and whose mental state does not meet the standards.

To sum up, with the help of the above technical solution of the present invention, the construction personnel passing the entrance of the construction site are detected by using the trained deep learning algorithm, and the tracking algorithm is used to track the target, and when the target collides with the detection line, count and Statistics, so as to realize the intelligent statistics of the number of personnel and vehicles entering and exiting the construction site and the total number of personnel and vehicles in the local area. In addition, the present invention can also use image recognition technology to identify the clothing and status of construction personnel passing through the entrance of the construction site. , and remind the construction workers who do not meet the standards, so as to effectively avoid the occurrence of safety accidents caused by the construction workers not wearing safety helmets, not wearing work clothes or poor mental state, and thus effectively improving the safety of the construction site Construction safety performance.

At the same time, the present invention can not only achieve a recognition accuracy rate of more than 95% under various lighting conditions, but also ensure that the statistical data is accurate, so that the time difference between the analysis screen and the original screen does not exceed 1 second, and has a high recognition accuracy rate. And the advantages of fast recognition speed.

At the same time, based on the algorithm used in the present invention and the plasticity contained in the UI, different functions can be added according to different requirements, such as face recognition, job type recognition, etc., and the scalability is high; in addition, the hardware equipment required by the present invention is cheap , Improve on-site management efficiency and obtain more economic benefits while saving costs.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of the present invention. within the scope of protection.

Claims (8)

1. The constructor intelligent monitoring system based on computer vision and deep learning is characterized by comprising an intelligent statistics module (1) and an intelligent monitoring module (2);

the intelligent statistics module (1) is used for detecting constructors passing through a construction site entrance by using a trained deep learning algorithm, tracking targets by using a tracking algorithm, counting and counting when the targets collide with a detection line, and displaying in real time by a client;

the intelligent monitoring module (2) is used for respectively identifying the dressing and the state of constructors passing through the site entrance by utilizing a preset image identification technology and reminding constructors which do not accord with the standard;

the intelligent statistics module (1) comprises a video acquisition module (11), a deep learning algorithm analysis module (12) and an analysis result display module (13);

the video acquisition module (11) is used for acquiring real-time monitoring pictures according to monitoring cameras erected at various entrances and exits of a construction site, and inputting the real-time monitoring pictures into a POE switch to obtain an initial video material after conversion;

the deep learning algorithm analysis module (12) is used for outputting video analysis pictures and personnel statistics data corresponding to the initial video materials in real time through a trained deep learning algorithm, and outputting the analysis pictures and statistics results to a client;

the analysis result display module (13) is used for displaying video analysis pictures and statistics data of personnel and vehicles on the client side, and is also used for enabling management personnel to switch the analysis pictures and the statistics data obtained by cameras at different entrances and exits according to requirements;

the deep learning algorithm analysis module (12) comprises a detection area setting module (121), a video analysis module (122), a data statistics module (123) and a data transmission module (124);

the detection area setting module (121) is used for controlling the actual detection range in each picture by adjusting the corresponding detection range according to the picture layout of different entrances and exits so as to realize the setting of the detection area;

the video analysis module (122) is used for analyzing images in the initial video material through coordinates of the wire collision detection points, so that analysis and statistics of personnel and vehicles in a detection area are realized;

the data statistics module (123) is used for judging and counting whether the target frame collides with the wire or not and the color of the wire collision area of the target frame;

the data transmission module (124) is used for outputting the analysis picture and the statistical result to the client through the RTSP server and the HTTP push service.

2. The intelligent constructor monitoring system based on computer vision and deep learning as set forth in claim 1, wherein the setting of the detection area comprises the steps of:

the positions of all the endpoints are determined in sequence according to the shape of the required area and expressed by an array, each element of the array is a binary array representing the endpoint of the graph, and the adjustment and the setting of the detection area are realized by adjusting the array.

3. The intelligent monitoring system for constructors based on computer vision and deep learning according to claim 2, wherein the analysis of the image in the initial video material by the coordinates of the collision detection point, the analysis and statistics of the personnel and vehicles in the detection area are realized by the following steps:

acquiring position parameters and categories of all objects in a current image;

judging whether the geometric center of an object in the new frame image and the offset of a geometric center of a certain object in the previous frame image are within a preset offset, if so, judging that the two objects are the same object and have the same ID, if not, judging that the new object exists in the new frame image, and assigning a new ID for the object;

using an image of a known object ID with a rectangle representing the object range, taking the parameter of a certain object range as x ₁ ，y ₁ ，x ₂ ，y ₂ Wherein x is ₁ <x ₂ ，y ₁ <y ₂ The coordinates of the wire-strike detection point are (check_point_x, check_point_y), check_point_x=x ₁ ，check_point_y＝int[y ₁ +(y ₂ -y ₁ )*0.6]Int means rounding the operation result;

and judging whether the object collision detection point is positioned in the judging area, if so, carrying out statistical operation on the object, and if not, not carrying out statistical operation.

4. The intelligent monitoring system for constructors based on computer vision and deep learning according to claim 3, wherein the determining and counting of the goal passing in and out by the goal frame wire collision and the color of the goal frame wire collision area comprises the following steps:

the method comprises the steps that all pictures captured by a current camera are defined as detection areas, a blue and yellow strip-shaped area is set as a judging area, and the pictures are recorded as entering when a target goes up to hit a yellow line, and recorded as exiting when the target goes down to hit a blue line;

acquiring real-time monitoring pictures acquired by monitoring cameras at all entrances and exits of a construction site, and performing size reduction treatment on the acquired real-time monitoring pictures;

judging whether a target appears in the detection area of the reduced real-time monitoring picture, if not, regarding the monitoring picture as an invalid picture and performing neglect cleaning, if so, framing the target and outputting;

detecting whether the target frame collides with the wire or not and the color of the wire collision area of the target frame, and judging and counting the entry and exit of the target according to the color of the wire collision area of the target frame.

5. The intelligent monitoring system for constructors based on computer vision and deep learning according to claim 1, wherein the intelligent monitoring module (2) comprises a safety helmet identification module (21), a work clothes identification module (22), a state identification module (23) and an alarm reminding module (24);

the helmet identification module (21) is used for identifying constructors without wearing the helmet, which pass through the site entrance, by utilizing an image identification technology;

the work clothes identification module (22) is used for identifying constructors who do not wear work clothes and pass through a worksite entrance by utilizing an image identification technology;

the state identification module (23) is used for identifying the mental state of constructors passing through the worksite entrance by utilizing an image identification technology;

the alarm reminding module (24) is used for reminding constructors who do not wear safety helmets, do not wear work clothes and do not accord with the spirit state standard.

6. The intelligent monitoring system for constructors based on computer vision and deep learning according to claim 5, wherein the state recognition module (23) comprises an constructor face image acquisition module (231), a fusion feature recognition module (232) and a state recognition result output module (233);

the constructor face image acquisition module (231) is used for acquiring face images of constructors in real-time monitoring pictures of all entrances and exits of a construction site;

the fusion characteristic recognition module (232) is used for recognizing the facial image of the constructor by utilizing a facial state recognition algorithm based on independent characteristic fusion, so as to recognize the mental state of the constructor entering the construction site;

the state identification result output module (233) is used for outputting the mental state information of constructors in real-time monitoring pictures of all entrances and exits of the construction site.

7. The intelligent constructor monitoring system based on computer vision and deep learning of claim 6, wherein the fusion feature recognition module (232) comprises a global state feature extraction module (2321), a local state feature extraction module (2322), a state feature fusion module (2323) and a state feature analysis recognition module (2324);

the global state feature extraction module (2321) is used for extracting global state features of the facial image through discrete cosine transformation, and removing correlation of the global state features by utilizing an independent component analysis technology to obtain independent global state features;

the local state feature extraction module (2322) is used for extracting features of an eye region and a mouth region in an image sequence, and performing Gabort wavelet transformation and feature fusion on the eye region and the mouth region respectively to obtain dynamic multi-scale features of two local regions as local state features of a facial image;

the state feature fusion module (2323) is used for fusing the independent global state feature and the local state feature, and adding local detail information into the global feature to obtain a face state fusion feature;

the state feature analysis and recognition module (2324) is used for analyzing and recognizing the obtained facial state fusion features through a preset classifier to obtain mental state information of constructors, wherein the mental states of the constructors comprise an awake state, a mild fatigue state, a moderate fatigue state and a severe fatigue state.

8. The intelligent monitoring system for constructors based on computer vision and deep learning according to claim 7, wherein the preset classifier is obtained by selecting part of features, removing redundant features and training through an AdaBoost algorithm, and the calculation formula of the preset classifier is as follows:

wherein T represents the final number of algorithm loops, a _t Representation classifier h _t (X) the selected weights, determined by AdaBoost algorithm learning, x= (X) ₁ ，X ₂ ，…，X _T ) Representing the dynamic Gabor features of the selected facial image sequence.