CN118229085A

CN118229085A - Intelligent park energy management risk visual management system based on attention prediction mechanism

Info

Publication number: CN118229085A
Application number: CN202410435261.XA
Authority: CN
Inventors: 秦章原; 陈昊; 张凌; 叶晓东; 滕彦; 梁晖; 叶盛; 鞠津京; 江毅奋
Original assignee: Shanghai Investigation Design and Research Institute Co Ltd SIDRI
Current assignee: Shanghai Investigation Design and Research Institute Co Ltd SIDRI
Priority date: 2024-04-11
Filing date: 2024-04-11
Publication date: 2024-06-21

Abstract

The application provides an intelligent park energy management risk visual management system based on an attention prediction mechanism, which comprises the following components: the data acquisition management layer is used for acquiring sensing risk data information of the Internet of things system in the target park; the data processing management layer is used for identifying key information in the video stream in real time, integrating the sensing risk data information, extracting key frames and potential risk characteristics and improving the attention to a specific target; the data analysis management layer is used for rapidly positioning and accurately predicting abnormal conditions in the park, evaluating safety risks, environment risks and the like in the park in real time, and formulating preventive measures according to risk evaluation results; and the visual display management layer is used for displaying the risk data information in a visual form according to the demands of the user so as to assist in supervising the potential risks in the park. The intelligent park potential risk detection method and the intelligent park potential risk detection device improve accuracy of intelligent park potential risk detection, enhance space positioning capability, reduce calculation complexity and improve detection performance.

Description

Intelligent park energy management risk visual management system based on attention prediction mechanism

Technical Field

The application belongs to the technical field of data processing, relates to the technical field of energy management risk management of an intelligent park, and particularly relates to an energy management risk visual management system of the intelligent park based on an attention prediction mechanism.

Background

The intelligent park is an industrial park which integrates resources and services inside and outside the park and realizes digital and intelligent management and service by using new generation information technologies such as an Internet of things system, artificial intelligence, edge computing, mobile Internet and the like and an advanced management mode and developing the innovative park management and service industry into a main line. The establishment requirement of the intelligent park is based on park safety, a security system, a park management system, a park service system and a park operation system are built, a high-quality modern park is built, and industrial aggregation and 'people' aggregation are achieved.

According to the establishment requirement of wisdom garden, need realize that the garden can manage comprehensive management of security risk, including risk assessment, risk prevention, risk reply, risk disposition etc., improve garden security management level, effectively strengthen the whole information control dynamics of garden managers and reduce operation cost. At present, the common mode is to use monitoring video in a park as a data source, combine artificial intelligence, computer vision and other technologies, and realize the functions of monitoring energy use, optimizing energy distribution, reducing energy waste and the like in a visual mode so as to improve the energy efficiency of the park. While monitoring and managing various risks in the campus including, but not limited to, security risks, environmental risks, equipment failures, etc. Through visual management, potential risks are discovered and dealt with in time.

With the continuous development of artificial intelligence, the deep learning method is gradually applied to the field of target detection. The current target detection algorithm based on deep learning is divided into two types according to the need of extracting candidate areas: the method is called a double-stage target detection algorithm, mainly comprises RCNN, fast RCNN, FASTER RCNN and other algorithms, and has the advantages of high detection accuracy; the latter is called a single-stage target detection algorithm, and mainly uses YOLO (You Only Look Once) and SSD (Single Shot Multi-Box Detector) algorithms, and has the advantage of high detection speed. Along with the continuous upgrading of the target detection algorithm, the single-stage target detection algorithm ensures the detection speed and simultaneously has good detection precision. In order to further improve the accuracy of detecting network target identification, an improved attention mechanism module is fused, and the receptive field of the feature extraction module is increased by introducing multi-scale convolution.

However, in the current smart park energy management risk visual management technology, although advanced technologies such as the internet of things system, artificial intelligence, edge computing and the like have been widely applied, there still exist some technical problems to be solved urgently, specifically as follows:

(1) In the application of the existing target detection algorithm in the intelligent park, although the accuracy of the double-stage target detection is higher, the real-time performance can be influenced due to the fact that the calculation complexity is too high when a large-scale monitoring video is processed. Although the single-stage target detection speed is high, the recognition accuracy of the early algorithm under a small target or a complex scene still has room for improvement.

(2) At present, the energy use in a park is monitored and optimized in a visual mode, and the problems that fine details, complex energy equipment states and multidimensional energy flow conditions are difficult to accurately identify possibly exist, so that fine energy management and safety early warning cannot be achieved.

(3) Overall risk management capability is limited: for diversified risk sources such as safety risk, environmental risk and equipment failure in an intelligent park, the existing system may lack an efficient and unified risk assessment model and coping strategies, and is difficult to realize full-period and comprehensive risk prevention and control.

(4) Insufficient data fusion and intelligent decision support: the insufficient data integration and analysis depth of each subsystem may cause information island phenomenon, and the advantage of big data cannot be fully exerted, so that timely, accurate and comprehensive decision support is provided for park managers.

(5) Limitations of the application of the attention mechanism: although some research attempts have been made to apply attention mechanisms to target detection to improve feature extraction, it is still a challenge in this field to design an attention module with high adaptability and pertinence for energy management risk prediction in a specific scene of an intelligent park, and to effectively merge the attention module into a target detection network.

Therefore, in the existing intelligent park energy management risk management technology, when a large-scale monitoring video is processed, the real-time performance is poor due to the fact that the computing complexity is too high, and the problems that fine energy management and safety early warning cannot be achieved due to the fact that fine problems are difficult to accurately identify in the monitoring process are solved, meanwhile, the problems of information island and the like caused by insufficient data fusion and intelligent decision support, insufficient integration degree and data dispersion exist.

Disclosure of Invention

In view of the above-mentioned drawbacks of the prior art, an objective of the present application is to provide a smart park energy management risk vision management system based on an attention prediction mechanism, which is used for solving the problems that in the smart park energy management risk management technology, when a large-scale monitoring video is processed, the real-time performance is poor due to too high calculation complexity, and the fine energy management and the safety precaution cannot be realized due to difficulty in accurately identifying the fine problem in the monitoring process, and meanwhile, the problems of insufficient data fusion and intelligent decision support, and information island caused by insufficient integration degree and data dispersion exist.

In a first aspect, the present application provides a smart campus energy management risk vision management system based on an attention prediction mechanism, the system comprising: the system comprises: the system comprises a data acquisition management layer, a data processing management layer, a data analysis management layer and a visual display management layer; the data acquisition management layer is used for acquiring sensing risk data information of an Internet of things system of an operation site in the target intelligent park; the sensing risk data information of the Internet of things system comprises: equipment facility status information, environmental basis parameters; the data processing management layer is connected with the data acquisition management layer and is used for identifying key information in a video stream in real time, integrating sensing risk data information of the Internet of things system, extracting key frames and potential risk features from the video stream and improving the attention to a specific target; the data analysis management layer is connected with the data processing management layer and is used for rapidly positioning and accurately predicting abnormal conditions in a target intelligent park, evaluating safety risks and environmental risks in the park in real time and formulating corresponding preventive measures according to risk evaluation results; the visual display management layer is connected with the data analysis management layer and is used for displaying the risk data information sensed by the Internet of things system in the target intelligent park according to the user demands in a visual form so as to assist in supervising the potential risks in the target intelligent park.

In an embodiment of the present application, the data acquisition management layer includes: the system comprises an Internet of things system and a visual perception system; the system comprises an Internet of things system, a target intelligent park, a target intelligent sensing device, a target intelligent park and a target intelligent park, wherein the Internet of things system is used for storing intelligent sensing devices in the target intelligent park and acquiring device facility state information and environment basic parameters in real time; the visual perception system is used for acquiring monitoring pictures in a target area through the camera probe, constructing a video monitoring network of the whole coverage of the target intelligent park, collecting video stream information of each area of the park and transmitting the video stream information to the safety control center.

In an embodiment of the present application, the data processing management layer includes: an edge computing system and a visual feature extraction system; the edge computing system is connected with the data acquisition management layer and is used for carrying out data analysis and processing on the sensing risk data information of the Internet of things system, identifying key information in a video stream in real time, obtaining video stream information and integrating the equipment facility state information and the environment basic parameters; the visual feature extraction system is connected with the edge computing system and is used for extracting key frames and potential risk features from the video stream to obtain visual feature information, and meanwhile, the attention degree of a specific target is improved by combining a multi-scale attention mechanism.

In an embodiment of the present application, the data analysis management layer includes: the system comprises a prediction evaluation system, a decision scheduling system and an analysis optimization system; the prediction evaluation system is connected with the data processing management layer and is used for rapidly positioning and accurately predicting abnormal conditions in the target intelligent park and evaluating safety risks and environmental risks in the target intelligent park in real time; the decision scheduling system is connected with the prediction evaluation system and is used for making corresponding countermeasures according to the risk evaluation result, sending instructions to corresponding responsibility bodies and outputting an optimized energy allocation strategy; the analysis optimizing system is connected with the decision scheduling system and is used for storing, cleaning and analyzing historical data based on the big data analysis platform, obtaining a risk prediction model through training and performing periodic optimization.

In one embodiment of the present application, the predictive evaluation system includes: the attention prediction module and the risk assessment module; the attention prediction module is used for dynamically focusing on a region with potential safety hazard or high energy consumption based on an attention mechanism model of deep learning, so as to realize quick positioning and accurate prediction of abnormal conditions; the risk assessment module is connected with the attention prediction module and is used for carrying out real-time assessment and early warning on the safety risk, the environment risk and the equipment fault risk in the target intelligent park based on the sensing risk data information and the visual characteristic information of the Internet of things system and based on the sensing risk data information and the video stream information of the Internet of things system.

In one embodiment of the present application, the decision scheduling system includes: an intelligent decision platform and an energy management system; the intelligent decision platform is used for making corresponding preventive measures, optimization schemes and emergency response plans according to risk assessment results, and sending instructions to a responsibility main body corresponding to risk management and an automatic control system; the energy management system is used for optimizing an energy distribution strategy based on a result output by the prediction model.

In one embodiment of the present application, the analysis optimization system includes: the big data analysis platform and the model optimization module; the big data analysis platform is connected with the decision scheduling system and is used for storing, cleaning and analyzing historical data based on the big data analysis platform, finding rules, trends and modes and carrying out risk prediction management and energy management; the model optimization module is connected with the big data analysis platform and is used for obtaining a risk prediction model through training and periodically and iteratively updating parameters of the risk prediction model so as to continuously improve prediction precision and system efficiency.

In an embodiment of the present application, the visual display management layer includes: a visual data service module and a visual service module; the visual display management layer is used for integrating, analyzing and integrating the sensing risk data information and the video stream information of the Internet of things system, supporting visual scene display in the target intelligent park based on the integrated result, and hanging the sensing risk data information and the video stream information of the Internet of things system with the data analysis management layer through the Internet of things system technology; the visual data service module is used for carrying out visual integrated processing on sensing risk data information and video stream information of the Internet of things system; the visual service module is used for linking the risk analysis model with the data based on the result of the visual integration processing so as to generate a visual scene capable of displaying the target intelligent park.

In an embodiment of the present application, the visual data service module includes: any one or more of a data cleaning unit, an information processing unit, a data conversion unit, a spatialization fusion unit, a multi-source data fusion unit, a data association unit, a data query unit, a data light weight unit, and a sharing exchange unit.

In an embodiment of the present application, the visual scene includes: digital twin holographic scenes, three-dimensional visualization scenes, environmental risk management scenes, space management scenes, equipment risk management scenes, energy management scenes and comprehensive security scenes.

In a second aspect, the present application provides a smart park energy management risk visual management method based on an attention prediction mechanism, including the steps of: acquiring sensing risk data information and video stream information of an Internet of things system of a target intelligent park; the sensing risk data information of the Internet of things system comprises: equipment facility status information, environmental basis parameters; performing data processing, identification and analysis on the sensing risk data information and the video stream based on the Internet of things system to acquire video stream information, and integrating and extracting visual features of the sensing risk data information of the Internet of things system; training an attention mechanism model based on visual characteristics to obtain abnormal conditions in a target intelligent park, and positioning, predicting and evaluating to obtain a potential risk evaluation result; corresponding preventive measures, optimization schemes and emergency response schemes are formulated based on the potential risk assessment results, and risk instructions are sent to relevant responsibility main bodies according to user requirements and relevant management rules to alarm and remind; and analyzing big data through the potential risk analysis model, and sending the big data to a target intelligent park risk management system for display so as to realize intelligent management of potential risks of the target intelligent park.

As described above, the intelligent park energy management risk visual management system based on the attention prediction mechanism has the following beneficial effects:

(1) The intelligent garden energy management risk visual management system based on the attention prediction mechanism applies the deep learning and computer vision technology to the real-time monitoring of the garden energy management risk, and can automatically identify the state of energy equipment, environmental change and potential safety hazards by performing intelligent processing and analysis on data sources such as video monitoring and the like, thereby greatly improving the efficiency and accuracy of information acquisition and processing;

(2) The application adopts a self-adaptive attention prediction mechanism, so that the system has the capability of dynamically focusing key information; in massive data, the attention prediction module can simulate the rapid screening process of important information of human brain, and accurately lock a high-risk area or an abnormal energy consumption link, so that more accurate risk early warning and energy management optimization are realized; the predictive management method not only reduces the risk caused by information omission in the traditional mode, but also improves the scientificity and rationality of resource allocation;

(3) The intelligent park management system based on the attention prediction mechanism predicts the problems possibly occurring in the future and provides preventive measure suggestions based on big data analysis and a machine learning algorithm; through a risk assessment model updated in real time, a manager can formulate a coping strategy before the problem occurs, so that the loss caused by sudden accidents is obviously reduced, and meanwhile, the sustainability and toughness of park operation are improved;

(4) The application integrates the front edge technology of a plurality of fields such as an Internet of things system, artificial intelligence, edge calculation, mobile Internet and the like, and constructs a comprehensive and deep multidimensional information sensing, analyzing and deciding system; the cross-domain deep fusion breaks through the limitation of the traditional park management means, and realizes the omnibearing and three-dimensional management from facility equipment to environmental safety and to personnel behaviors;

(5) The system supports function configuration and expansion according to the actual demands of the intelligent park, and can provide personalized solutions for different park scenes; meanwhile, by means of cloud computing and a big data platform, the system can continuously collect, learn and optimize a prediction model of the system so as to adapt to continuously changing environmental conditions and management requirements, and a self-iterative and continuously improved service mode is formed; meanwhile, the application has great flexibility and high reliability, reduces the input cost and effectively improves the economic benefit.

Drawings

Fig. 1 is a schematic diagram of a general architecture for energy management and risk vision management for an energy+smart park according to an embodiment of the present application.

Fig. 2 is a schematic diagram showing the overall structure of a smart campus energy management risk visual management system based on an attention prediction mechanism according to an embodiment of the present application.

Fig. 3 is a flow chart illustrating a method for managing risk vision in a smart campus based on an attention prediction mechanism according to an embodiment of the application.

Fig. 4 is a flowchart of S13 in the smart campus energy management method based on the attention prediction mechanism according to the embodiment of the application.

Fig. 5A is a schematic diagram illustrating a conventional CBAM block diagram in a smart campus energy management risk vision management system based on an attention prediction mechanism according to an embodiment of the present application.

Fig. 5B is a schematic diagram of a conventional CAM flow chart of a smart campus energy management system based on an attention prediction mechanism according to an embodiment of the present application.

Fig. 5C is a schematic diagram illustrating a CAM flow chart after modification of the intelligent campus energy management system based on the attention prediction mechanism according to the embodiment of the present application.

Fig. 6A is a schematic diagram of an improved SAM flow of a smart campus energy management system based on an attention prediction mechanism according to an embodiment of the present application.

Fig. 6B is a schematic diagram of a conventional SAM flow of a smart campus energy management system based on an attention prediction mechanism according to an embodiment of the present application.

Fig. 7A is a schematic diagram of an improved YOLOv s network structure of an application improvement CBAM of the intelligent campus energy management system based on the attention prediction mechanism according to the embodiment of the present application.

Fig. 7B is a schematic diagram of a YOLOv s network structure of a smart campus energy management risk vision management system based on an attention prediction mechanism according to an embodiment of the present application.

Description of element reference numerals

1. Data acquisition management layer

2. Data processing management layer

3. Data analysis management layer

4. Visual display management layer

11. Internet of things system

12. Visual perception system

21. Edge computing system

22. Visual feature extraction system

31. Predictive evaluation system

32. Decision scheduling system

33. Analysis optimization system

41. Visual data service module

42. Visual service module

311. Attention prediction module

312. Risk assessment module

321. Intelligent decision platform

322. Energy management system

331. Big data analysis platform

332. Model optimization module

S11 to S15 steps

Detailed Description

Other advantages and effects of the present application will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present application with reference to specific examples. The application may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present application. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict.

It should be noted that the illustrations provided in the following embodiments merely illustrate the basic concept of the present application by way of illustration, and only the components related to the present application are shown in the drawings and are not drawn according to the number, shape and size of the components in actual implementation, and the form, number and proportion of the components in actual implementation may be arbitrarily changed, and the layout of the components may be more complicated.

The principle and implementation of the intelligent campus energy management system based on the attention prediction mechanism of the present embodiment will be described in detail below with reference to the accompanying drawings, so that those skilled in the art can understand the intelligent campus energy management system based on the attention prediction mechanism of the present embodiment without creative effort.

Referring to fig. 1 and fig. 2, a schematic diagram of a total architecture of energy+smart campus energy management risk vision management according to an embodiment of the present application and a schematic diagram of a total architecture of a smart campus energy management risk vision management system based on an attention prediction mechanism according to an embodiment of the present application are shown respectively. As shown in fig. 1 and 2, the smart campus energy management risk vision management system based on the attention prediction mechanism includes: the system comprises a data acquisition management layer 1, a data processing management layer 2, a data analysis management layer 3 and a visual display management layer 4.

The data acquisition management layer 1 is used for acquiring sensing risk data information of an Internet of things system in a target intelligent park; the data processing management layer 2 is connected with the data acquisition management layer 1 and is used for identifying key information in a video stream in real time, integrating sensing risk data information of the Internet of things system, extracting key frames and potential risk features from the video stream and improving the attention to a specific target; the data analysis management layer 3 is connected with the data processing management layer 2 and is used for rapidly positioning and accurately predicting abnormal conditions in a target intelligent park, evaluating safety risks and environmental risks in the park in real time and formulating corresponding preventive measures according to risk evaluation results; the visual display management layer 4 is connected with the data analysis management layer 3 and is used for displaying the risk data information sensed by the Internet of things system in the target intelligent park according to the user demands in a visual form so as to assist in monitoring the potential risks in the target intelligent park. The risk data information sensing method of the Internet of things system comprises the following steps: different types of data such as equipment and facility state information, environment basic parameters and the like.

The data acquisition management layer 1 comprises: an internet of things system 11 and a visual perception system 12. The internet of things system 11 is configured to store intelligent sensing devices in a target intelligent park, and obtain device facility state information and environment basic parameters in real time; the visual perception system 12 is used for acquiring monitoring pictures in a target area through a camera probe, constructing a video monitoring network of the whole coverage of the target intelligent park, collecting video stream information of each area of the park and transmitting the video stream information to the safety control center.

In this embodiment, the internet of things system 11 is an important part of the data source of the smart park, and is mainly responsible for collecting and controlling information in the target area (i.e. smart park); mainly comprises the following steps: status information of various types of equipment and facilities, environment basic parameters, and weak current systems such as a video telephone system, a high-definition camera system, an access control system, an environment monitoring system, a security system, a fire protection system, an energy consumption statistics system, wired and wireless communication equipment, a sensing RFID (radio frequency identification device) equipment and the like. The internet of things system 11 is mainly implemented through an internet of things sensor network, and is implemented by deploying various intelligent sensors, such as: temperature and humidity sensor, smoke detector, energy consumption monitor etc. collect state information and environmental parameter of the equipment facilities in the garden in real time to realized collecting basic information such as job site, building, vehicle, production place, environment, security protection in the wisdom garden in real time, and to basic information and data transmission to the data processing management layer that gathers, in order to provide the data basis of energy management risk management, thereby can the information such as the surrounding environment state of this wisdom garden and the real-time state of each equipment facilities of real-time feedback.

The visual perception system 12 is used for acquiring monitoring pictures in a target area through a camera probe, constructing a video monitoring network of the whole coverage of the target intelligent park, collecting video stream information of each area of the park and transmitting the video stream information to the safety control center.

In this embodiment, a video stream may be shot in real time for a security risk management scene, an environmental risk management scene, and an equipment failure risk management scene in the smart park, and monitoring images at different moments may be extracted from the shot video stream.

Please continue to refer to fig. 1 and 2.

The data processing management layer 2 includes: an edge computing system 21 and a visual feature extraction system 22. The edge computing system 21 is connected with the data acquisition management layer 1, and is used for performing data analysis and processing on the sensing risk data information of the internet of things, identifying key information in a video stream in real time, obtaining video stream information, and integrating the equipment facility state information and the environment basic parameters; the visual feature extraction system 22 is connected to the edge computing system 21, and is configured to extract key frames and potential risk features from the video stream, so as to obtain visual feature information, and simultaneously combine a multi-scale attention mechanism to improve attention to a specific target.

In this embodiment, in the edge computing system 21, a mode of computing edge nodes is adopted to perform preliminary data analysis and processing at a place close to a data source, so as to reduce the pressure of a cloud server, identify key information in a video stream in real time, and perform preliminary integration on sensing risk data of the internet of things. Further, the visual feature extraction system 22 employs advanced computer vision techniques, such as Convolutional Neural Networks (CNNs), to extract key frames and potentially risky features from the video stream, while combining a multi-scale attention mechanism to enhance attention to a specific target.

For example: the intelligent park fire operation site firstly acquires video stream to the fire site, extracts fire monitoring pictures in real time according to a certain time interval, and performs data preprocessing on the fire monitoring pictures, such as: denoising, normalizing, cleaning and other processes are carried out on the moving fire picture to obtain data in a uniform format; extracting features of the preprocessed image to obtain key frames of video stream information; according to the fire risk assessment task, open fire and smoke on the fire scene are defined, whether the open fire moves or not, and whether the smoke concentration, the spreading area move or not can be detected by adopting a target detection method for selection; extracting visual features of the video by using methods such as a histogram, features and the like; and then fusing the sensing risk data information of the Internet of things system and the open fire and smoke characteristics extracted from the video stream to form a fire risk characteristic vector. The example process considers the factors such as correlation and redundancy among the features, so that the accuracy of subsequent risk assessment and early warning can be improved.

Another example is: in the intelligent park, acquiring data such as environmental parameters in the intelligent park, equipment in the area, environmental parameters in the production area and the like through equipment such as a temperature sensor, a humidity sensor and the like; and simultaneously extracting the field monitoring video picture in the area. These environmental parameters are integrated with information such as field surveillance video pictures to ensure temporal and spatial synchronicity of the two. The integrated data is then preprocessed, for example: and carrying out operations such as denoising, normalization, data cleaning and the like on the integrated data of the environment parameters and the field monitoring video picture so as to improve the quality and consistency of the data. Then, according to the requirements of the environmental factors of the operation site, such as: the characteristics required to be extracted are determined according to factors such as pipeline pressure, medium pressure, air humidity, environmental temperature (summer temperature is lower than a high temperature threshold value, winter temperature is higher than a low temperature threshold value, each threshold value is determined according to production working conditions and chemical properties of products), whether storm, heavy snow, thunderstorm weather and the like occur or not; then, extracting features of the preprocessed data, and extracting on-site video pictures from sensing risk data information and video streams of the internet of things system, for example: and fusing characteristics such as thunder and lightning, rainfall and the like to form an environment risk characteristic vector.

The data analysis management layer 3 includes: a predictive evaluation system 31, a decision scheduling system 32 and an analysis optimization system 33. The prediction evaluation system 31 is connected with the data processing management layer 2, and is used for rapidly positioning and accurately predicting abnormal conditions in the target intelligent park and evaluating safety risks and environmental risks in the target intelligent park in real time; the decision scheduling system 32 is connected with the prediction evaluation system 31, and is configured to formulate corresponding countermeasures according to risk evaluation results, send instructions to corresponding responsibility subjects, and output an optimized energy allocation strategy; the analysis optimizing system 33 is connected with the decision scheduling system 32, and is used for storing, cleaning and analyzing historical data based on a big data analysis platform, obtaining a risk prediction model through training and performing periodic optimization.

In this embodiment, the predictive evaluation system 31 includes: attention prediction module 311, risk assessment module 312. The attention prediction module 311 is configured to dynamically focus on a region with potential safety hazard or high energy consumption based on an attention mechanism model of deep learning, so as to implement rapid positioning and accurate prediction of an abnormal situation.

Specifically, the type equipment facility state data in the target intelligent park and the feature images extracted from the video streams corresponding to the positions of the type equipment facility state data are input into the attention mechanism model, and the execution process of the attention module is improved to finish the process. Wherein the improved attention module comprises: the improved channel attention sub-module and the improved space attention sub-module are sequentially connected in series, and channel attention correction is firstly carried out and then the space attention correction is carried out; and then, broadcasting the channel attention in the space dimension and multiplying the channel attention by the feature map, and broadcasting the space attention weight in the channel dimension and multiplying the result by the previous result, so that the feature map after comprehensive attention correction is finally obtained.

For example: first, the improve channel attention submodule I-CAM (Improved ChannelAttention Module, improve channel attention submodule): firstly, processing the equipment state and the characteristic images thereof through two parallel paths, extracting multi-scale characteristic information by using a plurality of 3X 3 convolution layers with different scales on one path, and generating a channel attention vector through global maximum pooling and global average pooling; the other path does not carry out any convolution operation, and the original feature map is directly subjected to global maximum pooling and global average pooling so as to retain the original channel information. Then, reducing the channel dimension of the four pooling results to 1/4 of the channel number of the original input feature map by using a 1×1 convolution layer; the four channel attention vectors are subjected to a shared MLP learning channel importance weight with a reduction rate r, and a final channel attention weight Mc (F) is obtained through element-by-element summation and Sigmoid activation function; and multiplying the channel attention weight with the input feature map on the corresponding channel to realize the feature weighting of the channel level.

Next, the modified spatial attention sub-module, for the feature map F' after I-SAM (Improved Spatial Attention Module, modified spatial attention sub-module) processing has been applied, also adopts a two-way parallel structure:

a) Also using improved multi-scale convolution followed by maximum pooling and average pooling to extract spatial attention information;

b) The feature map F' is directly max-pooled and average-pooled.

Then, connecting four space attention matrixes, generating space attention weights Ms (F') through a 7×7 convolution layer, and obtaining final space attention weights through a Sigmoid activation function; the spatial attention weights are multiplied in the spatial dimension with the feature map previously subjected to the channel attention process to enhance or suppress features at specific spatial locations.

Finally, the improved channel attention submodule and the improved space attention submodule are integrated, namely: the improved channel attention submodule and the improved space attention submodule are sequentially connected in series, and channel attention correction is firstly carried out, and then space attention correction is carried out; the feature map after full attention correction is finally obtained by broadcasting the channel attention weights in the spatial dimension and multiplying the feature map, and by broadcasting the spatial attention weights in the channel dimension and multiplying the previous results.

For example: according to the above, in the intelligent park fire operation site, the fire risk feature vector is input into the attention mechanism model, the channel attention sub-module is improved to process the fire equipment state and the fire risk feature vector in parallel paths, one channel attention vector is generated, and the other channel attention vector is not subjected to convolution operation and the original channel information is withheld; then, reducing the channel dimension of the four pooling results to 1/4 of the channel number of the original input feature map by using a1×1 convolution layer; the four channel attention vectors are subjected to a shared MLP learning channel importance weight with a reduction rate r, and a final channel attention weight Mc (F) is obtained through element-by-element summation and Sigmoid activation function; and multiplying the channel attention weight with the input feature map on the corresponding channel to realize the feature weighting of the channel level. Further, the spatial attention sub-module is improved by adopting a two-way parallel structure, so that a processed characteristic diagram is obtained. Then, four spatial attention matrices are connected and spatial attention weights are generated by one 7×7 convolution layer; then, the spatial attention weight is obtained through the calculation of an activation function; the spatial attention weights are multiplied in the spatial dimension with the feature map previously subjected to the channel attention process to enhance or suppress features at specific spatial locations. Finally, sequentially connecting the improved channel attention submodule and the improved space attention submodule in series, firstly executing channel attention correction, and then executing space attention correction; and finally obtaining the fire operation characteristic diagram after comprehensive attention correction by broadcasting the channel attention weight in the space dimension and multiplying the channel attention weight with the characteristic diagram and broadcasting the channel attention weight in the channel dimension and multiplying the channel attention weight with the previous result.

Similarly, the environmental risk feature vector in the intelligent park can also be calculated through the above process, so as to obtain a corrected environmental risk feature map.

In this embodiment, the risk assessment module 312 is connected to the attention prediction module 311, and is configured to perform real-time assessment and early warning on the security risk, the environmental risk, and the equipment failure risk in the target smart park based on the internet of things sensing risk data information and visual feature information and based on the internet of things sensing risk data information and video stream information.

Specifically, the improvement CBAM is integrated into a YOLOv s target detection network, redundant information is filtered, key target characteristics are highlighted, and detection speed and accuracy of various energy management risk events (such as abnormal energy consumption, equipment failure, environmental risk and the like) in a park are improved.

Specifically, at the feature extraction section of YOLOv s, the above-described modified CBAM module is inserted before entering the pre-probe; by introducing the improvement CBAM, the key target information can be screened out more effectively, and the influence of redundant information is reduced, so that the accuracy of the detection network is improved. The data flow direction is as follows: original image-feature extraction network-refinement CBAM module-predict head-output object box and class probabilities.

For example: inputting the corrected live fire operation characteristic diagram into a YOLOv s target detection network at a live fire operation site in an intelligent park, and inserting the improved CBAM module before entering a pre-measuring head; outputting target detection frames, such as: open flame target frames, smoke target frames, etc., and calculating the probability of each occurrence. By introducing the improvement CBAM, key target information can be screened out more effectively, and the influence of redundant information is reduced, so that the accuracy of a detection network for open fire, smoke and the like in a live fire operation site is improved.

Another example is: inputting the corrected environmental risk feature map into YOLOv s target detection network in the intelligent park, and inserting the improved CBAM module before entering the pre-measurement head; outputting target detection frames, such as: and (5) calculating the probability of each occurrence of the lightning target frame, the field device running state target frame and the like. By introducing the improvement CBAM, key target information can be screened out more effectively, and the influence of redundant information is reduced, so that the accuracy of a detection network for environmental disasters and the like in the field is improved.

The decision scheduling system 32 comprises: an intelligent decision platform 321 and an energy management system 322. The intelligent decision platform 321 is configured to formulate a corresponding preventive measure, an optimization scheme, an emergency response plan according to the risk assessment result, and send an instruction to a responsibility main body corresponding to risk management and an automation control system; the energy management system 322 is configured to optimize an energy allocation strategy based on the result output by the prediction model.

In this embodiment, the occurrence of the fire scene can be determined by the obtained detection result, for example, the occurrence of an open fire movement event in the fire scene; and judging the corresponding reasons according to the detected target frame and the corresponding phenomena, pushing the corresponding emergency treatment measures, and simultaneously sending fire alarm information to each relevant responsible person in a mode of monitoring system alarm, APP notification, short message notification and the like so as to meet the requirement of response management of each level.

The analysis optimization system 33 includes: big data analysis platform 331, model optimization module 332. The big data analysis platform 331 is connected to the decision scheduling system 32, and is configured to store, clean and analyze historical data based on the big data analysis platform, discover rules, trends and modes, and perform risk prediction management and energy management; the model optimization module 332 is connected to the big data analysis platform 331, and is configured to obtain a risk prediction model through training and perform periodic iterative updating of parameters of the risk prediction model, so as to continuously improve prediction accuracy and system performance.

In this embodiment, in a large number of data processing processes, the attention mechanism model and the target detection model are processed, the on-site situation is evaluated through the processes of analysis, identification, judgment and the like, and the corresponding processing scheme is pushed according to the evaluation result, and the alarm information is sent to the relevant responsible personnel. Through a large amount of data processing and model training, the model is optimized, corresponding preventive measures, emergency plans and the like are optimized according to risk assessment results, and instructions are pushed to relevant responsible personnel or an automatic control system.

The visual display management layer 4 includes: a visual data service module 41 and a visualization service module 42. The visual display management layer 41 is configured to integrate, analyze and integrate the risk data information and the video stream information of the internet of things, support visual scene display in the target intelligent park based on the integrated result, and link the risk data information and the video stream information of the internet of things with the data analysis management layer through the internet of things technology.

The visual data service module 41 is configured to perform visual integration processing on the sensing risk data information and the video stream information of the internet of things; the visualization service module 42 is configured to link the risk analysis model with data based on a result of the visualization integration process, so as to generate a visualization scene capable of displaying the target smart park.

In this embodiment, the visual data service module 41 includes, but is not limited to: the system comprises a data cleaning unit, an information processing unit, a data conversion unit, a spatialization fusion unit, a multi-source data fusion unit, a data association unit, a data query unit, a data light-weight unit, a sharing exchange unit and the like.

Specifically, the visual data service module 41 first collects real-time status information data of the energy management device from each internet of things sensor and each camera probe, such as: sensing risk data such as motor temperature, air humidity, pressure, illumination intensity and the like, and fusing real-time video stream information of a park in real time; after integration, these data are pre-processed, such as: data cleaning, data screening, data conversion into a unified format, and data compression, so as to reduce the burden of storage and transmission. Then, the preprocessed data is subjected to visual integrated processing, and can be displayed in a chart, a curve, a three-dimensional scene and the like; and simultaneously, decoding and playing the real-time video stream information, and fusing the real-time video stream information and the real-time video stream information to form an intuitive visual interface.

The visual scene comprises: digital twin holographic scenes, three-dimensional visualization scenes, environmental risk management scenes, space management scenes, equipment risk management scenes, energy management scenes and comprehensive security scenes.

In this embodiment, on the basis of the above-mentioned visual integration processing, risk identification and early warning are also required. Namely: and analyzing the sensing risk data through a machine learning algorithm, predicting possible dangers, and sending out early warning when the risks reach a certain threshold value.

For example: the three-dimensional visualization is used for displaying the three-dimensional scene of the intelligent park, and can visually check the real-time scene of each position in the intelligent park. Through view functions such as rotation, sectioning and roaming of the three-dimensional graphic engine, information such as areas, floors and rooms recorded in the modeling process of the model is displayed at the design position of the three-dimensional graphic engine in the three-dimensional space of the intelligent park in cooperation with space objects and equipment asset objects stored in the service database. The visualization function mainly displays objects, and some operation contents are predefined according to different display requirements. The model is based on three-dimensional visualization, and related data of intelligent park operation is required to be displayed in a platform. The data linkage mode can be presented in various forms, and generally, the basic data of intelligent park operation are displayed in a three-dimensional scene, such as: the access control data can be realized through the association with a specific access control card punching machine, the data presentation is realized, when people in a certain position need to check the access data, the displayed data table and the displayed statistical curve are led out from the equipment position in a lead mode and then are suspended in the space around the model, and meanwhile, the specific components are colored or flash according to the requirement; visual representation of weather data (rain, snow, sand storm, haze and the like), power consumption water of the whole park, operation data of power supply equipment and the like, wherein a three-dimensional view displays a proper angle and depth of field as a background for data display; the inspection personnel positioning is used for dynamically adding the inspection personnel positioning information into the scene according to personnel positioning information by using the function of the three-dimensional graphic engine in the three-dimensional scene, and simultaneously displaying the information data of the personnel in combination with the first display mode.

Various application scenarios in the intelligent park are described below as examples.

The visual presentation throughout the intelligent park has the following:

(1) By superposing the optimal path calculation result, parking space path guidance, building load optimization calculation and the like on the three-dimensional model, the method is presented in a three-dimensional scene in a reasonable and visual animation effect mode, for example, the path calculation result is simulated by a directional flowing-shaped dotted line, the calculated building power load change is represented by the depth of thermodynamic diagram color, and the like.

The three-dimensional scene includes: digital twin holographic scenes, space management scenes, equipment management scenes, energy management scenes and comprehensive security scenes.

Specifically, the digital twin holographic scene mainly includes: a three-dimensional model of the whole park, outdoor underground pipelines, GIS base maps (satellite maps and street maps) around the park, important equipment running states, environmental parameters and the like.

(2) The whole model of the intelligent park is displayed, local details (such as hiding the equipment model in the whole and independently loading equipment in the local view) can be checked according to the requirements; predefined viewing angles for each area (building element), automatically switching viewing angles when the focus of interest is switched; and simulating the state condition of the current park according to the time and weather environment data acquired in real time to display.

(3) The content of the space management scene is mainly that corresponding content is displayed in the three-dimensional scene according to the space information of the intelligent park recorded in the system.

For example, if the user wants to view the operation and maintenance building, the three-dimensional graph only shows the operation and maintenance building, and other buildings are hidden; when the equipment is checked to produce the third building of the furnace, building components above and below the third building can be completely hidden; while displaying information of the current spatial pipeline.

(4) The content of the device management scene is mainly to locate and highlight the device according to the device ID, and to display an attribute detail window.

The interaction operation of the device management scene mainly comprises the following steps: the equipment displays and uses the basic three-dimensional visual function, and can set warning color for the equipment when the warning information is displayed. Such as: if the running condition of the power supply motor is to be checked, the motor at the target position can be directly checked, and whether the motor runs or not and the running real-time data such as the information of motor power, running time, temperature and the like can be intuitively checked.

(5) The content of the comprehensive security scene is mainly that emergency event simulation in a park is subjected to three-dimensional display, such as escape routes, rescue material positions and the like; displaying the position and the patrol route of security personnel; the invasive position of the invasive person is located.

The interactive operation of the comprehensive security scene mainly comprises the following steps: the escape route near the area with higher risk is displayed in the three-dimensional view, route display is performed in the space, floors and rooms are horizontally sectioned, and other unwanted areas are transparent for convenient viewing; the high-brightness identification of the rescue material positions such as fire fighting water boxes, fire extinguishers, sandbags and the like in the park is realized, and the display function is triggered by one key when a dangerous situation is encountered, so that the check is facilitated.

In summary, the intelligent garden energy management risk visual management system based on the attention prediction mechanism applies the deep learning and computer visual technology to the real-time monitoring of the garden energy management risk, and can automatically identify the state of energy equipment, environmental change and potential safety hazards by performing intelligent processing and analysis on data sources such as video monitoring and the like, thereby greatly improving the efficiency and accuracy of information acquisition and processing; the system has the capability of dynamically focusing key information by adopting a self-adaptive attention prediction mechanism; in massive data, the attention prediction module can simulate the rapid screening process of important information of human brain, and accurately lock a high-risk area or an abnormal energy consumption link, so that more accurate risk early warning and energy management optimization are realized; the predictive management method not only reduces the risk caused by information omission in the traditional mode, but also improves the scientificity and rationality of resource allocation; through the risk assessment model updated in real time, a manager can formulate a coping strategy before the problem occurs, so that the loss caused by sudden accidents is obviously reduced, and meanwhile, the sustainability and toughness of park operation are improved. Meanwhile, the intelligent park management system based on the attention prediction mechanism integrates the front edge technologies of a plurality of fields such as an Internet of things system, artificial intelligence, edge calculation, mobile Internet and the like, and a comprehensive and deep multidimensional information sensing, analyzing and deciding system is constructed; the cross-domain deep fusion breaks through the limitation of the traditional park management means, and realizes the omnibearing and three-dimensional management from facility equipment to environmental safety and to personnel behaviors; meanwhile, the method has great flexibility and high reliability, reduces the input cost and effectively improves the economic benefit.

It should be noted that, it should be understood that the division of the modules of the above system is merely a division of a logic function, and may be fully or partially integrated into a physical entity or may be physically separated. And these modules may all be implemented in software in the form of calls by the processing element; or can be realized in hardware; the method can also be realized in a form of calling software by a processing element, and the method can be realized in a form of hardware by a part of modules. For example, the x module may be a processing element that is set up separately, may be implemented in a chip of the system, or may be stored in a memory of the system in the form of program code, and the function of the x module may be called and executed by a processing element of the system. The implementation of the other modules is similar. In addition, all or part of the modules can be integrated together or can be independently implemented. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in a software form.

The above modules may be one or more integrated circuits configured to implement the above methods, for example: one or more Application SPECIFIC INTEGRATED Circuits (ASIC), or one or more microprocessors (DIGITAL SIGNAL processor, DSP), or one or more field programmable gate arrays (FieldProgrammable GATE ARRAY, FPGA), etc. For another example, when a module above is implemented in the form of a processing element scheduler code, the processing element may be a general-purpose processor, such as a central processing unit (Central Processing Unit, CPU) or other processor that may invoke the program code. For another example, the modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

Referring to fig. 3, a flow chart of a smart campus energy management risk visual management method based on an attention prediction mechanism according to an embodiment of the application is shown in an embodiment. As shown in fig. 3, the present embodiment provides a visual management method for risk management of intelligent park based on an attention prediction mechanism.

The intelligent park energy management risk visual management method based on the attention prediction mechanism specifically comprises the following steps of:

S11, acquiring sensing risk data information and video stream information of the Internet of things of the target intelligent park. The sensing risk data information of the Internet of things comprises: equipment status information, environmental basis parameters.

In the embodiment, intelligent sensing equipment is deployed in a target intelligent park, and equipment state information and environment basic parameters of each equipment are acquired in real time through an Internet of things sensor acquisition device; meanwhile, a monitoring picture in a target area is obtained through a camera probe, a video monitoring network of the whole coverage of the target intelligent park is built, the video monitoring network of the whole coverage is built through a high-definition camera and an AI intelligent camera, video stream data of each area of the park is captured and transmitted, and the video stream data is transmitted to a safety control center.

Specifically, raw data is collected from various sensors (e.g., temperature sensor, humidity sensor, pressure sensor, etc.) in the smart campus; and acquiring real-time video data from the camera or the video stream source.

And S12, carrying out data processing, identification and analysis on the basis of the sensing risk data information of the Internet of things and the video stream so as to acquire video stream information, and integrating and extracting visual features of the sensing risk data information of the Internet of things.

In this embodiment, data preprocessing is performed on sensing risk data information of the internet of things, original data is cleaned, noise, errors or redundant data are removed, and data standardization and normalization are performed. Integrating the data to ensure that the sensing data of the Internet of things and the video stream data are synchronous in time so as to facilitate subsequent data analysis and fusion; and carrying out spatial alignment on the sensing data of the Internet of things and a specific position or object in the video stream, and ensuring the relevance between the data.

Further, processing the video stream by adopting computer vision technology (such as frame extraction, background subtraction, target tracking and the like) to extract key frames or target objects; visual features are then extracted from the key frames or target objects, such as: color, texture, shape, movement pattern, etc.

For example: in the field operation process, the camera probe can realize that personnel in an operation area enter an in-out tracking shooting function, target pedestrians are detected and identified through a target detection algorithm, pedestrian target frames are extracted, then the pedestrian target frames are tracked, and information such as the moving route of the target pedestrians is obtained, so that on-site monitoring events such as guardianship personnel and operation personnel, events such as illegal operation of the operation personnel and the like are monitored more accurately, related personnel can be reminded and managed, and the time efficiency and accuracy of management of the operation personnel in an intelligent park are improved.

It should be noted that the algorithms adopted by different monitoring targets are different, and the selection needs to be performed according to the requirements of users.

And S13, training an attention mechanism model based on visual characteristics to obtain abnormal conditions in the target intelligent park, and positioning, predicting and evaluating to obtain a potential risk evaluation result. Referring to fig. 4, a flowchart of S13 in the intelligent campus energy management risk visual management method based on the attention prediction mechanism according to the embodiment of the application is shown. As shown in fig. 4, the method comprises the following steps:

When a human observes an object, its vision system tends to focus on the most important object, while ignoring portions of the line of sight that are not important to identify the object. The attention mechanism, similar to the human visual system, has been shown to facilitate various computer vision tasks by weighting the input information so that the computer can focus on important parts or features. In the field of image processing and object recognition, attention mechanisms are widely used to improve the performance of algorithms. Attention mechanisms are therefore introduced aimed at improving the feature extraction capabilities of the model for the target. From the design structure, the system is mainly divided into a self-attention model, a space attention model, a channel attention model and a multi-head attention model.

The conventional mixed domain attention mechanism CBAM, the attention sub-module would directly perform global maximum pooling and global average pooling of the channel domain and the spatial domain on the feature map. In this way, although the weights of the channel domain and the space domain can be simply extracted, the utilization rate of the module to the information in the feature map is low, so that the accuracy of the detection network is affected.

Therefore, the application adopts the sequential series structure of the traditional mixed domain attention mechanism CBAM, improves the channel attention submodule and the space attention submodule thereof, and constructs an improved CBAM attention mechanism module.

S131, based on a deep learning attention mechanism model, dynamic focusing is performed on a region with potential safety hazard or high energy consumption, and rapid positioning and accurate prediction of abnormal conditions are realized.

Referring to fig. 5A, fig. 5B, fig. 5C, fig. 6A, and fig. 6B, a schematic diagram of a conventional CBAM block diagram in an attention prediction mechanism-based smart campus risk vision management system according to an embodiment of the present application, a conventional CAM flow schematic diagram of an attention prediction mechanism-based smart campus risk vision management system according to an embodiment of the present application, an improved SAM flow schematic diagram of an attention prediction mechanism-based smart campus risk vision management system according to an embodiment of the present application, and a conventional SAM flow schematic diagram of an attention prediction mechanism-based smart campus risk vision management system according to an embodiment of the present application are shown.

In this embodiment, a deep learning-based attention mechanism model, such as an improved CBAM module, is adopted to dynamically focus on potential safety hazards or high-energy consumption areas, so as to realize rapid positioning and accurate prediction of abnormal conditions. The improvement CBAM module includes: the improved channel attention sub-module and the improved space attention sub-module are formed by two parts.

(1) Improved channel attention submodule

Multiscale convolution is embedded in the attention mechanism to improve network performance and reduce the number of parameters, limiting the convolution kernel size to 3 x 3. For convenience, a path is reserved without operation to increase the effect of the network. In addition, in order to improve the performance of the current convolutional network, the pooling operation is indispensable, so that adding a parallel pooling path can also produce good effects. To prevent the pooling layer output from merging with the convolution layer output from causing the feature map dimension to increase, a1 x 1 convolution is first used to reduce the computation. As the four layers are output, the channel number of the original characteristic diagram is reduced to 1/4 of the original channel number.

For an input feature map F epsilon R ^c×h×w, the channel attention submodule carries out two-way parallel processing on the original feature map, the first part carries out multi-scale convolution operation to generate a new feature map F epsilon R ^c×h×w, and then carries out maximum pooling and average pooling; the second part performs maximum pooling and average pooling directly without any operation, resulting in four channel attention vectors: Representing the average pooling feature and the maximum pooling feature, respectively. The importance of each channel information is learned by using a shared MLP (multi-layerperceptron ), and finally the four channel attention vectors are combined by summing element by element and then activated by a Sigmoid function to obtain the final channel attention weight Mc (F). In short, the channel attention weight calculation formula is:

Mc(F)＝σ(MLP(maxpool(F))+MLP(avgpool(F))+MLP(maxpool(*F))+MLP(avgpool(*F)))

wherein Mc (F) represents element-wise summation; sigma represents a Sigmoid activation function.

And weighting the channel weight Mc (F) and the input F according to the corresponding channel to obtain a channel attention result. The shared network MLP is a multi-layer sensor with a hidden layer sized to reduce the number of parametersWhere r is the reduction rate. Thus, the two-layer convolution can learn the feature importance degree on each channel while reducing the convolution parameter number.

In this implementation, the input feature map F is first processed through two parallel paths: extracting multi-scale characteristic information by using a plurality of 3X 3 convolution layers with different scales on one path, and generating a channel attention vector through global maximum pooling and global average pooling; the other path does not carry out any convolution operation, and the original feature map is directly subjected to global maximum pooling and global average pooling so as to retain the original channel information. The channel dimensions of the four pooling results are reduced to 1/4 of the original input feature map channel number using a1 x1 convolutional layer. The four channel attention vectors are passed through a shared MLP learning channel importance weight with a reduction rate r, and the final channel attention weight Mc (F) is obtained by element-wise summation with Sigmoid activation function. The channel attention weight is multiplied by the input feature map on the corresponding channel to achieve feature weighting at the channel level. The intelligent park system has the advantages that the whole field element monitoring process in each area position in the intelligent park is realized, the event trace in the whole park field management process is reserved, the alarm details of the whole management process are recorded, and the early warning information is analyzed and counted, so that the risk management capability, safety and fusion of the intelligent park system are improved.

(2) Improved spatial attention submodule

The result of channel attention is further extracted with spatial weights. The spatial attention sub-module also performs the same two-way parallel processing on the input feature map F 'e R ^c×h×w as the channel attention sub-module, the first part uses the same improved multi-scale convolution to generate a new feature map F' e R ^c×h×w, then applies maximum pooling and average pooling along the channel axis, and the second part does not perform any operation, directly applies maximum pooling and average pooling, and will obtain 4 spatial attention matrices: Representing the average pooling feature and the maximum pooling feature in the channel, respectively, and connecting them together to generate an efficient feature profile. Features are presented and connected to generate an efficient feature map. And (3) generating a spatial attention matrix by applying a convolution layer on the characteristic description graph, and finally activating by using a Sigmoid function to obtain a final spatial attention weight Ms (F'). In short, the spatial attention weight calculation formula is:

wherein σ represents a Sigmoid activation function; where ∈7x7 denotes a convolution operation with a convolution kernel size of 7x7; ms (F') represents a spatial attention weight.

In the implementation process, as for the feature map F' to which the I-CAM processing has been applied, a two-way parallel structure is adopted as well: also using improved multi-scale convolution followed by maximum pooling and average pooling to extract spatial attention information; the feature map F' is directly max-pooled and average-pooled. The four spatial attention matrices are connected, and the spatial attention weights Ms (F') are generated through a 7×7 convolution layer, and then the final spatial attention weights are obtained through a Sigmoid activation function. The spatial attention weights are multiplied in the spatial dimension with the feature map previously subjected to the channel attention process to enhance or suppress features at specific spatial locations.

(3) Improved CBAM attention module integration

The two improvement submodules are connected in series in sequence, the improvement channel attention submodule is used for correcting, and then the result is subjected to spatial attention submodule correction. The overall improved attention process can be summarized as follows:

Wherein, Representing the multiplication of the corresponding pixel points; in multiplication, attention weights are broadcast into the channel dimensions. In the multiplication process, the attention weights are correspondingly broadcast into the feature map, the channel attention weights are broadcast along the space dimension, and the space attention weights are broadcast along the channel dimension; ms (F') represents a spatial attention weight; f _refined denotes the final weighted output.

Compared with CBAM, the method only carries out the maximum pooling and average pooling operation on the original feature map, and a multi-scale convolution operation using convolution and splicing is added in the improved CBAM structure to generate a new feature map, and the two operations are processed in parallel. Introducing multi-scale convolution can promote the receptive fields of the channel attention weight and the space attention weight obtained by operation, emphasize target information and filter other redundant information.

In the implementation process, the improved channel attention submodule and the improved space attention submodule are sequentially connected in series, and channel attention correction is firstly carried out, and then space attention correction is carried out; the feature map after full attention correction is finally obtained by broadcasting the channel attention weights in the spatial dimension and multiplying the feature map, and by broadcasting the spatial attention weights in the channel dimension and multiplying the previous results.

In summary, the method can capture richer characteristic information by introducing multi-scale convolution and parallel pooling paths, and particularly can more accurately identify targets such as energy equipment, environmental conditions, potential safety hazards and the like in complex scenes; the improved channel attention module improves the extraction capability of the related features of key energy management through learning the importance of the features of different channels, and reduces misjudgment and missed detection. Likewise, by means of the improved spatial attention sub-module, the system can better understand the importance of the features at each spatial location, thereby having a stronger focusing and response capability to risk events in a specific area, such as precisely identifying the location of potential safety hazards or the area where high energy consumption equipment is located. Furthermore, the MLP structure with the reduction rate r and the simplified convolution operation are used in the improvement process, so that the calculation complexity and the parameter number of the model are reduced, and the vision management system has better operation efficiency and resource utilization efficiency while ensuring the performance.

And S132, based on the Internet of things sensing risk data information and the video stream information, carrying out real-time evaluation and early warning on the security risk, the environment risk and the equipment fault risk in the target intelligent park. Referring to fig. 7A and fig. 7B, a schematic diagram of an improved YOLOv s network structure of an application improvement CBAM of the intelligent campus energy management system based on the attention prediction mechanism according to the embodiment of the present application and a schematic diagram of a YOLOv s network structure of the intelligent campus energy management system based on the attention prediction mechanism according to the embodiment of the present application are shown respectively.

In this embodiment, the features to be extracted are defined and selected according to specific risk assessment and early warning tasks. These features include, but are not limited to: physical quantities such as temperature, humidity and pressure in the sensing data of the Internet of things, and visual characteristics such as target detection, movement track, color and texture in the video stream.

For the sensing data of the Internet of things, the characteristics can be extracted by using methods such as statistical analysis, time sequence analysis and the like. For example, statistics of mean, variance, trend, etc. of the sensed data may be calculated, or the data fitted using a time series model to extract features. For video streaming information, computer vision and image processing techniques may be used to extract features. For example, an object in a video may be identified using an object detection algorithm, a motion trajectory of the object may be calculated using a light flow method, and visual features of the video may be extracted using a color histogram, texture features, or the like.

And fusing the features extracted from the sensing data of the internet of things and the video stream information to form a comprehensive feature vector or feature matrix. The fusion process can consider the factors such as the correlation and redundancy among the features so as to improve the accuracy of subsequent risk assessment and early warning. And selecting and optimizing the extracted features according to actual application requirements and evaluation results. Features which contribute to risk assessment and early warning tasks greatly are selected, and redundant and irrelevant features are removed at the same time, so that the complexity of subsequent processing is reduced, and the generalization capability of a model is improved.

Specifically, in the original YOLOv s, the feature enhancement section iteratively fuses the feature maps and also uses multiple sequential convolution operations. Although the method can combine the characteristic information with different scales, a large amount of redundant information is generated in the process, and the detection precision of the network is reduced. Meanwhile, a plurality of continuous convolution operations are used for the high-dimensional feature map, so that the running parameters and the calculated amount of the network are increased, and the detection performance of the network is also affected.

The attention mechanism is added in the target detection network, so that important information in the characteristics can be obviously enhanced, and the method has an important effect on object detection. Thus, improvement CBAM was introduced into YOLOv s. Before the input prediction part predicts, the improved attention module is used for processing the input prediction part to extract more comprehensive and more important target information, other redundant information is filtered, and the accuracy of the detection network is increased.

According to the application, the improved CBAM module is integrated into a YOLOv s target detection network, redundant information is filtered, key target characteristics are highlighted, and the detection speed and accuracy of various energy management risk events (such as abnormal energy consumption, equipment failure and environmental risk) in a park are improved.

In summary, the application enhances the receptive field of the feature map through the operation of multi-scale convolution and splicing, so that the model has a larger visual field and a deeper understanding level when processing different types of risk factors, and is beneficial to improving the comprehensive judgment capability and decision support quality of the system.

And S14, corresponding preventive measures, optimization schemes and emergency response schemes are formulated based on the potential risk assessment results, and risk instructions are sent to relevant responsibility main bodies according to user requirements and relevant management rules to alarm and remind.

In the embodiment, according to the risk assessment result, corresponding preventive measures, optimization schemes or emergency response plans are formulated, and instructions are pushed to relevant responsible persons or an automatic control system; an energy management system: and the energy distribution strategy is optimized by utilizing the result output by the prediction model, so that the invalid consumption is reduced, and the energy utilization efficiency is improved.

For example: according to the obtained detection result, for example, an open fire movement event occurs in a fire operation site, the occurrence of a fire scene at the moment can be judged; and judging the corresponding reasons according to the detected target frame and the corresponding phenomena, pushing the corresponding emergency treatment measures, and simultaneously sending fire alarm information to each relevant responsible person in a mode of monitoring system alarm, APP notification, short message notification and the like so as to meet the requirement of response management of each level.

And S15, carrying out big data analysis through the potential risk analysis model, and sending the big data to a target intelligent park risk management system for display so as to realize intelligent management of potential risks of the target intelligent park.

In the embodiment, the data obtained by analyzing the potential risk analysis model is stored, cleaned and analyzed through the big data analysis platform, massive historical data are found, rules, trends and modes are found, and the risk prediction model and the energy management strategy are continuously optimized; and then according to actual application effect feedback, periodically and iteratively updating the prediction model parameters, and continuously improving the prediction precision and the system efficiency.

In summary, the intelligent park energy management risk visual management system based on the attention prediction mechanism provided by the application has the following steps

The beneficial effects are that:

According to the intelligent park energy management risk visual management system based on the attention prediction mechanism, which is provided by the application, the deep learning and computer vision technology is applied to real-time monitoring of park energy management risk, and the intelligent processing and analysis of data sources such as video monitoring can automatically identify the state of energy equipment, environmental change and potential safety hazards, so that the efficiency and accuracy of information acquisition and processing are greatly improved; the system has the capability of dynamically focusing key information by adopting a self-adaptive attention prediction mechanism; in massive data, the attention prediction module can simulate the rapid screening process of important information of human brain, and accurately lock a high-risk area or an abnormal energy consumption link, so that more accurate risk early warning and energy management optimization are realized; the predictive management method not only reduces the risk caused by information omission in the traditional mode, but also improves the scientificity and rationality of resource allocation; through the risk assessment model updated in real time, a manager can formulate a coping strategy before the problem occurs, so that the loss caused by sudden accidents is obviously reduced, and meanwhile, the sustainability and toughness of park operation are improved. Meanwhile, the intelligent park management system based on the attention prediction mechanism integrates the front edge technologies of a plurality of fields such as an Internet of things system, artificial intelligence, edge calculation, mobile Internet and the like, and a comprehensive and deep multidimensional information sensing, analyzing and deciding system is constructed; the cross-domain deep fusion breaks through the limitation of the traditional park management means, and realizes the omnibearing and three-dimensional management from facility equipment to environmental safety and to personnel behaviors; meanwhile, the method has great flexibility and high reliability, reduces the input cost and effectively improves the economic benefit.

The descriptions of the processes or structures corresponding to the drawings have emphasis, and the descriptions of other processes or structures may be referred to for the parts of a certain process or structure that are not described in detail.

The above embodiments are merely illustrative of the principles of the present application and its effectiveness, and are not intended to limit the application. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the application. Accordingly, it is intended that all equivalent modifications and variations of the application be covered by the claims, which are within the ordinary skill of the art, be within the spirit and scope of the present disclosure.

Claims

1. An intelligent campus energy management risk vision management system based on an attention prediction mechanism, the system comprising: the system comprises a data acquisition management layer, a data processing management layer, a data analysis management layer and a visual display management layer;

the data acquisition management layer is used for acquiring sensing risk data information of an Internet of things system of an operation site in the target intelligent park; the sensing risk data information of the Internet of things system comprises: equipment facility status information, environmental basis parameters;

the data processing management layer is connected with the data acquisition management layer and is used for identifying key information in a video stream in real time, integrating sensing risk data information of the Internet of things system, extracting key frames and potential risk features from the video stream and improving the attention to a specific target;

The data analysis management layer is connected with the data processing management layer and is used for rapidly positioning and accurately predicting abnormal conditions in a target intelligent park, evaluating safety risks and environmental risks in the park in real time and formulating corresponding preventive measures according to risk evaluation results;

The visual display management layer is connected with the data analysis management layer and is used for displaying the risk data information sensed by the Internet of things system in the target intelligent park according to the user demands in a visual form so as to assist in supervising the potential risks in the target intelligent park.

2. The attention prediction mechanism based smart campus energy management risk vision management system of claim 1, wherein the data collection management layer comprises: the system comprises an Internet of things system and a visual perception system;

The system comprises an Internet of things system, a target intelligent park, a target intelligent sensing device, a target intelligent park and a target intelligent park, wherein the Internet of things system is used for storing intelligent sensing devices in the target intelligent park and acquiring device facility state information and environment basic parameters in real time;

The visual perception system is used for acquiring monitoring pictures in a target area through the camera probe, constructing a video monitoring network of the whole coverage of the target intelligent park, collecting video stream information of each area of the park and transmitting the video stream information to the safety control center.

3. The attention prediction mechanism based smart campus energy management risk vision management system of claim 1, wherein the data processing management layer comprises: an edge computing system and a visual feature extraction system;

The edge computing system is connected with the data acquisition management layer and is used for carrying out data analysis and processing on the sensing risk data information of the Internet of things system, identifying key information in a video stream in real time, obtaining video stream information and integrating the equipment facility state information and the environment basic parameters;

the visual feature extraction system is connected with the edge computing system and is used for extracting key frames and potential risk features from the video stream to obtain visual feature information, and meanwhile, the attention degree of a specific target is improved by combining a multi-scale attention mechanism.

4. The attention prediction mechanism based smart campus energy management risk vision management system of claim 1, wherein the data analysis management layer comprises: the system comprises a prediction evaluation system, a decision scheduling system and an analysis optimization system;

The prediction evaluation system is connected with the data processing management layer and is used for rapidly positioning and accurately predicting abnormal conditions in the target intelligent park and evaluating safety risks and environmental risks in the target intelligent park in real time;

The decision scheduling system is connected with the prediction evaluation system and is used for making corresponding countermeasures according to the risk evaluation result, sending instructions to corresponding responsibility bodies and outputting an optimized energy allocation strategy;

The analysis optimizing system is connected with the decision scheduling system and is used for storing, cleaning and analyzing historical data based on the big data analysis platform, obtaining a risk prediction model through training and performing periodic optimization.

5. The attention prediction mechanism based smart campus energy management system of claim 4 wherein the predictive assessment system comprises: the attention prediction module and the risk assessment module;

The attention prediction module is used for dynamically focusing on a region with potential safety hazard or high energy consumption based on an attention mechanism model of deep learning, so as to realize quick positioning and accurate prediction of abnormal conditions;

The risk assessment module is connected with the attention prediction module and is used for carrying out real-time assessment and early warning on the safety risk, the environment risk and the equipment fault risk in the target intelligent park based on the sensing risk data information and the visual characteristic information of the Internet of things system and based on the sensing risk data information and the video stream information of the Internet of things system.

6. The attention prediction mechanism based smart campus energy management risk vision management system of claim 4 wherein the decision scheduling system comprises: an intelligent decision platform and an energy management system;

The intelligent decision platform is used for making corresponding preventive measures, optimization schemes and emergency response plans according to risk assessment results, and sending instructions to a responsibility main body corresponding to risk management and an automatic control system;

the energy management system is used for optimizing an energy distribution strategy based on a result output by the prediction model.

7. The attention prediction mechanism based smart campus energy management risk vision management system of claim 4, wherein the analysis optimization system comprises: the big data analysis platform and the model optimization module;

The big data analysis platform is connected with the decision scheduling system and is used for storing, cleaning and analyzing historical data based on the big data analysis platform, finding rules, trends and modes and carrying out risk prediction management and energy management;

The model optimization module is connected with the big data analysis platform and is used for obtaining a risk prediction model through training and periodically and iteratively updating parameters of the risk prediction model so as to continuously improve prediction precision and system efficiency.

8. The attention prediction mechanism based smart campus energy management risk vision management system of claim 1, wherein the vision presentation management layer comprises: a visual data service module and a visual service module;

The visual display management layer is used for integrating, analyzing and integrating the sensing risk data information and the video stream information of the Internet of things system, supporting visual scene display in the target intelligent park based on the integrated result, and hanging the sensing risk data information and the video stream information of the Internet of things system with the data analysis management layer through the Internet of things system technology;

Wherein,

The visual data service module is used for carrying out visual integrated processing on sensing risk data information and video stream information of the Internet of things system;

the visual service module is used for linking the risk analysis model with the data based on the result of the visual integration processing so as to generate a visual scene capable of displaying the target intelligent park.

9. The attention prediction mechanism based smart campus energy management risk vision management system of claim 8, wherein the vision data service module comprises: any one or more of a data cleaning unit, an information processing unit, a data conversion unit, a spatialization fusion unit, a multi-source data fusion unit, a data association unit, a data query unit, a data light weight unit, and a sharing exchange unit.

10. The attention prediction mechanism based smart campus energy management risk vision management system of claim 8 wherein the visualization scenario comprises: digital twin holographic scenes, three-dimensional visualization scenes, environmental risk management scenes, space management scenes, equipment risk management scenes, energy management scenes and comprehensive security scenes.

11. An intelligent park energy management risk visual management method based on an attention prediction mechanism is characterized by comprising the following steps of:

acquiring sensing risk data information and video stream information of an Internet of things system of a target intelligent park; the sensing risk data information of the Internet of things system comprises: equipment facility status information, environmental basis parameters;

Performing data processing, identification and analysis on the sensing risk data information and the video stream based on the Internet of things system to acquire video stream information, and integrating and extracting visual features of the sensing risk data information of the Internet of things system;

Training an attention mechanism model based on visual characteristics to obtain abnormal conditions in a target intelligent park, and positioning, predicting and evaluating to obtain a potential risk evaluation result;

Corresponding preventive measures, optimization schemes and emergency response schemes are formulated based on the potential risk assessment results, and risk instructions are sent to relevant responsibility main bodies according to user requirements and relevant management rules to alarm and remind;

and analyzing big data through the potential risk analysis model, and sending the big data to a target intelligent park risk management system for display so as to realize intelligent management of potential risks of the target intelligent park.