CN114399734A - Forest fire early warning method based on visual information - Google Patents

Abstract

The invention discloses a forest fire early warning method based on visual information, which comprises the following steps: inputting images in a data set to be detected into a forest fire detection model for training; arranging monitoring cameras in a forest, and numbering all the monitoring cameras; and inputting video data acquired by the monitoring camera into a forest fire detection model for classification, sending a smoke alarm if the classification result is that smoke is detected, providing the serial number of the monitoring camera of the video source, and finishing forest fire early warning. The invention utilizes the attention mechanism to make the network more sensitive to small-scale smoke and better extract the characteristics of early forest fire smoke. The invention integrates a characteristic-level and decision-level classification detection module, improves the discrimination capability of the smoke and similar objects, particularly the discrimination of the smoke and the fog. According to the invention, a negative sample of fog is added into the data set, so that the network is suitable for detecting the smoke in the foggy environment.

Description

A forest fire warning method based on visual information

technical field

The invention belongs to the field of forest fire alarm prediction, in particular to a forest fire early warning method based on visual information.

Background technique

There are an average of 220,000 forest fires in the world every year, affecting an area of 10 million hectares, causing countless property losses and casualties of precious animals. It is imminent to prevent forest fires. In the early stage of a fire, smoke is always generated before an open fire, so timely detection of smoke can effectively prevent the fire from worsening. Most of the traditional methods are based on sensors, but due to the limited detection distance of sensors, they are only suitable for indoor places, and for large areas forests are not effective.

For forest fires, the easiest way to achieve is to detect smoke through surveillance cameras. Traditional image-based forest smoke detection is based on the shape, color, texture, and motion characteristics of the smoke. Suspected smoke candidate area. Then design artificial features, extract the smoke feature vector of the candidate area, and finally input the extracted features into the classifier to obtain the final detection result. However, forest fire surveillance cameras have a wide field of view and generally obtain long-distance fire smoke video, so the smoke area is often small. The traditional image-based detection algorithm has poor detection effect and serious missed detection, and cannot be applied in multi-scene tasks.

In recent years, deep learning has developed rapidly. A large number of experiments have proved that the smoke detection algorithm based on deep learning is more effective and practical. Through neural network and loss function, a large number of smoke data sets are trained, and an efficient smoke detection model is obtained. At first, researchers used AlexNet for feature extraction to obtain better feature maps than traditional algorithms, but the accuracy and false negative rate did not meet the requirements. With the continuous development of convolutional neural networks, the feature extraction of images is becoming more and more accurate, such as VGG, RCNN, YOLO series networks have been able to meet many application requirements, but as the basic network of target detection, they are directly used for smoke detection effect and No, the main reason is that the smoke in the early stage of the fire accounts for a small proportion of the entire field of view, and the pixel information is very small, so it is difficult for the neural network to identify; secondly, because the real environment is more complex, it is a key problem to identify fire smoke in foggy conditions; There are few datasets in the specific field of fire smoke detection, most of the public datasets are unlabeled, and the samples are single and unrepresentative. The network trained by such datasets cannot effectively detect early smoke and fires, which is very important for researchers. cause great inconvenience.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a forest fire warning method based on visual information, so as to solve the problems existing in the above-mentioned prior art.

On the one hand, in order to achieve the above object, the present invention provides a forest fire warning method based on visual information, including:

constructing a data set to be tested and a forest fire detection model, and inputting the images in the data set to be tested into the forest fire detection model for training, to obtain the trained forest fire detection model;

arranging surveillance cameras in the forest and numbering all said surveillance cameras;

Input the video data collected by the surveillance camera into the trained forest fire detection model for classification, if the classification result is that smoke is detected, a smoke alarm is issued, and the number of the surveillance camera from which the video is sourced is provided to complete the forest fire warning.

Optionally, the process of constructing the data set to be tested includes:

Collect forest smoke pictures and fog-only forest pictures from web crawling;

The smoke with the forest as the background is artificially created, and the camera is used for long-distance shooting, and the captured video is saved as a picture in a unified format frame by frame, as a forest smoke picture;

Randomly add fog interference to some of the forest smoke pictures to obtain forest smoke pictures with fog interference;

The data set to be tested is synthesized by combining the forest smoke picture with fog interference and the fog-only forest picture.

Optionally, the process of constructing the data set to be tested further includes:

Annotate the forest smoke picture with fog interference, and obtain the classification information of the smoke and the coordinate size information of the real frame.

Optionally, the process of constructing the forest fire detection model includes:

The YOLOv5 network model is used to construct the forest fire detection model, and the forest fire detection model performs feature extraction through four convolution layers.

Optionally, the process of inputting the images in the data set to be tested into the forest fire detection model for training includes:

Preprocessing the images in the data set to be tested;

Feature extraction is performed on the pre-processed image through the attention mechanism, and the feature map is extracted;

Feature fusion and decision classification are performed on the feature map to obtain a final feature map, and detection is performed based on the final feature map.

Optionally, the process of preprocessing the images in the data set to be tested includes:

Perform mosaic data enhancement, adaptive anchor frame calculation and adaptive image scaling on the images in the data set to be tested;

The mosaic data enhancement splices the images in the data set to be tested by means of random scaling, random cropping, and random arrangement.

Optionally, feature extraction is performed on the preprocessed image through the attention mechanism, and the process of extracting the feature map includes:

Find the smoke area in the preprocessed image through spatial attention, and reduce the weight of other background areas;

The feature map is extracted by emphasizing the feature channel representing smoke through channel attention, and reducing the weight of other channels.

Optionally, the process of performing feature fusion and decision classification on the feature map includes:

Obtain the data of the second, third and fourth convolutional layers of the forest fire detection model and fuse them to obtain fusion features;

Classify the fused features to obtain a final feature map.

The technical effect of the present invention is:

(1) The present invention embeds the CBAM attention mechanism in the neck part of the YOLOv5 network, which makes the network more sensitive to small-scale smoke and better extracts the characteristics of early forest fire smoke.

(2) The present invention integrates the feature-level and decision-level classification and detection modules into the Backbone part of YOLOv5, which improves the discrimination ability between smoke and similar objects, especially the characteristics of smoke and fog.

(3) The present invention adds a negative sample of fog in the data set, so that the network can be adapted to the detection of smoke in a foggy environment.

Description of drawings

The accompanying drawings constituting a part of the present application are used to provide further understanding of the present application, and the schematic embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the attached image:

Fig. 1 is the YOLOv5s network structure diagram in the embodiment of the present invention;

Fig. 2 is the YOLOv5 structure diagram after the fusion CBAM in the embodiment of the present invention;

3 is a structural diagram of a CBAM in an embodiment of the present invention;

4 is a structural diagram of a fusion classification detection module in an embodiment of the present invention;

FIG. 5 is a structural diagram of a YOLOv5+ fusion classification detection module in an embodiment of the present invention.

Detailed ways

It should be noted that the embodiments in the present application and the features of the embodiments may be combined with each other in the case of no conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

It should be noted that the steps shown in the flowcharts of the accompanying drawings may be executed in a computer system, such as a set of computer-executable instructions, and, although a logical sequence is shown in the flowcharts, in some cases, Steps shown or described may be performed in an order different from that herein.

Example 1

An embodiment of the present invention provides a forest fire warning method based on visual information, including:

(1) Collect forest smoke pictures crawled from the web and convert them into a fixed format.

(2) The smoke with the forest as the background is artificially created, and the camera is used for long-distance shooting, and the shot video is saved as a picture in a unified format frame by frame.

(3) Add fog interference to some forest smoke pictures through the program.

(4) Combine the above processed pictures together with the pictures with only fog into a data set, and manually label all the images containing smoke through the labeling software to obtain the classification information of the smoke and the coordinate size information of the real frame.

(5) Build an improved YOLOv5 network model on the edge server. The improved YOLOv5 network is trained with the created dataset.

(6) Arrange surveillance cameras in various areas of the forest, and assign each surveillance number. If it is a mountain forest, a mountain can be monitored by 3 cameras, and monitoring is arranged at a distance of about 3 kilometers from the mountainside. Each camera has a horizontal viewing angle of 120°, and a mountain can be monitored in all directions; The forest watchtower is arranged for monitoring, and a watchtower also uses 3 cameras to form a 360° monitoring of the surroundings.

(7) The video data from the surveillance camera is received at the edge server and sent to the trained network model for classification.

(8) If the classification result is that smoke is detected, a smoke alarm is issued, and the surveillance camera number of the video source is provided, and then step 6 is repeated; if the classification result does not detect smoke, step 6 is repeated.

The model in the embodiment of the present invention adopts the YOLOv5 network model. YOLOv5 is a single-stage end-to-end target detection framework, with four versions: s, x, m, and l. The model changes from small to large, and the target detection is accurate. The speed of detection is gradually improved, but the detection speed is also gradually reduced. The YOLOv5 model has been continuously updated. Specifically, the present invention uses the s version of YOLOv5-5.0, because the detection of forest fire smoke requires rapid response to prevent it in a short enough time. The fire spread. The structure of YOLOv5s is shown in Figure 1 and consists of four parts: the input part, the Backbone part, the Neck part and the Detect part.

The input part mainly includes parts including mosaic data enhancement, adaptive anchor box calculation and adaptive image scaling. Mosaic data enhancement is stitched by random scaling, random cropping, and random arrangement, which has a good effect on small-scale detection; setting an appropriate anchor frame will get a higher intersection ratio, which helps to improve the accuracy of the model In the training process, the prior box can better learn to adapt to the shape of different objects. Adaptive anchor box design For different data sets, there will be anchor boxes with initial set length and width. In the network training, the network outputs the predicted frame based on the initial anchor frame, and then compares it with the real frame, calculates the difference between the two, and then reversely updates and iterates the network parameters. Adaptive image scaling is that, in actual use, many images have different aspect ratios. Therefore, after scaling and filling, the size of the black borders at both ends are different, and if there is more filling, there will be information redundancy, which will affect the inference speed. Therefore, the least black borders are added to the original image adaptively, and the black borders at both ends of the image height are reduced. During inference, the amount of calculation will also be reduced, that is, the target detection speed will be improved.

The Backbone part uses the CSP1 network structure for feature extraction, which is mainly used to solve the problem of gradient replication in larger convolutional layers. In addition, the YOLOv5 series also adds a Focus structure that YOLOv4 does not have.

The Neck part is used for feature fusion, and the feature pyramid network (FPN) and perceptual adversial network (PAN) structures are used to realize the upsampling and downsampling process. FPN is top-down, and it uses upsampling to transfer and fuse information to obtain predicted feature maps. PAN uses a bottom-up feature pyramid. FPN uses top-down feature convolution, using interpolation to do 2 upsampling from 19*19 to 38*38 on the higher feature layer, and the feature layer is connected horizontally by 1x1 convolution, changing the bottom layer of the lower feature layer number of channels. Different from YOLOv4, Yolov4's Neck uses ordinary convolution operations, while Yolov5's Neck structure adopts the CSP2 structure designed by reference to CSPNet to enhance the ability of network feature fusion.

The Detect part mainly includes the Bounding box loss function and NMS non-maximum suppression. The Bounding box loss function is GIOU_loss, and its formula is:

Among them, C is the smallest frame that can frame the real frame and the predicted frame at the same time. The area of C minus the area of the predicted frame and the real frame, and then compared with the area of C, can reflect the distance between the real frame and the predicted frame, effectively Solved the problem that when the predicted frame and the real frame do not intersect, the distance information between the predicted frame and the real frame is lost, and the IOU is equal to 0. NMS non-maximum suppression can select the one with the highest IOU in many prediction boxes, making the model more accurate.

In the early stage of the forest fire, the smoke is still relatively small, and the proportion of the entire monitoring field of view is also very small. Compared with the background, the features finally extracted by the multi-layer convolutional neural network are very small, resulting in weak feature expression ability, which is not suitable for small The detection of the target drops, whether it is a normal environment or a foggy environment, it is a challenge. In order to solve this problem, the present invention integrates spatial attention and channel attention into the YOLOv5 network, which helps the network to learn the characteristics of smoke and reduce the interference of the background. The network structure after fusion of CBAM is shown in Figure 2. The CBAM module is They are placed in front of the convolutional layers of the detection heads of three different scales, which can work on targets of various scales.

The structure of the CBAM module is shown in Figure 3. It is connected by spatial attention and channel attention. Spatial attention will find the area with the highest probability of containing objects in the picture, that is, the area with smoke, and give this area a higher weight. , while reducing the weight of other background regions; during the feature extraction process through convolution, the original image will be turned into a multi-channel feature map, and the channel attention will emphasize the feature channels representing smoke and give them higher weights, while other The channel weight becomes smaller. This joint attention mechanism can pay more attention to the location of smoke, reduce background interference, and improve the accuracy of small-scale smoke detection.

The existence of fog brings great difficulties to smoke detection, not only because it looks similar to smoke, resulting in a higher false positive rate, but also the fog will affect the quality of photos monitored by the terminal, making feature extraction more difficult. To obtain more discriminative features, real-time classification is achieved. This paper integrates a feature-level and decision-level fusion classification and detection module into YOLOv5, which can make the network more capable of distinguishing smoke and fog and reduce the false positive rate. The fusion classification and detection module is shown in Figure 4. The feature fusion part obtains three sets of more discriminative features through feature-level fusion similar to the Feature Pyramid Network (FPN) structure, including detailed edge texture information and high-level semantic information of smoke. , to improve the discrimination between smoke and similar objects.

Based on the three groups of classification layers, the convolution layer, the dropout layer and the global pooling layer, the decision layer fusion is carried out. This module has three inputs and one output. In this paper, the second, third, and fourth Conv modules of YOLOv5 are used as input, and the output is used as the input of the SPP module; the first half is feature fusion, and the information of different scales of convolutional layers is taken out separately, that is, strengthening In order to detect targets of different scales, the detection of targets with uneven distribution and blurred boundary contours such as smoke can also be strengthened. The extracted three different scale features are input into the decision fusion module behind. This module passes through the convolution layer and dropout layer. And the weighted fusion of the classifier composed of the global pooling layer, because the features of these three scales contain different target information, they have different importance in classification. The YOLOv5 network after adding the fusion classification detection module is shown in Figure 5.

The above are only the preferred specific embodiments of the present application, but the protection scope of the present application is not limited to this. Substitutions should be covered within the protection scope of this application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims (8)

1. A forest fire early warning method based on visual information is characterized by comprising the following steps:

establishing a data set to be tested and a forest fire detection model, inputting images in the data set to be tested into the forest fire detection model for training, and obtaining the trained forest fire detection model;

arranging monitoring cameras in a forest, and numbering all the monitoring cameras;

and inputting the video data acquired by the monitoring camera into the trained forest fire detection model for classification, sending a smoke alarm if the classification result is smoke detection, providing the serial number of the monitoring camera with a video source, and finishing forest fire early warning.

2. The method of claim 1, wherein constructing the dataset under test comprises:

collecting forest smoke pictures and forest pictures with only fog, which are crawled from a network;

artificially manufacturing smoke with a forest as a background, remotely shooting by a camera, and storing shot videos into pictures with a uniform format frame by frame as forest smoke pictures;

randomly adding fog interference to part of the forest smoke picture to obtain a forest smoke picture with fog interference;

and synthesizing the forest smoke picture with fog interference and the forest picture with fog only into the data set to be detected.

3. The method of claim 2, wherein constructing the dataset under test further comprises:

and marking the forest smoke picture with the fog interference to obtain the classification information of the smoke and the coordinate size information of the real frame.

4. The method of claim 1, wherein constructing the forest fire detection model comprises:

and constructing the forest fire detection model by adopting a YOLOv5 network model.

5. The method of claim 1, wherein inputting the images in the dataset under test into the forest fire detection model for training comprises:

preprocessing the image in the data set to be detected;

extracting the characteristics of the image after finishing the preprocessing by an attention mechanism, and extracting a characteristic graph;

and carrying out feature fusion and decision classification on the feature graph to obtain a final feature graph, and carrying out detection based on the final feature graph.

6. The method of claim 5, wherein preprocessing the image in the dataset under test comprises:

performing mosaic data enhancement, adaptive anchor frame calculation and adaptive image scaling on the image in the data set to be detected;

and splicing the images in the data set to be detected by the mosaic data enhancement in a random scaling, random cutting and random arrangement mode.

7. The method according to claim 5, wherein the extracting the feature map by performing feature extraction on the image after finishing the preprocessing through an attention mechanism comprises:

finding out a smoke area in the image after finishing the preprocessing by spatial attention, and reducing the weight of other background areas;

and extracting the feature map by emphasizing the feature channel representing the smoke through channel attention and reducing the weight of other channels.

8. The method according to claim 4 or 5, wherein the process of performing feature fusion and decision classification on the feature map comprises:

acquiring data of a second convolution layer, a third convolution layer and a fourth convolution layer of the forest fire detection model, fusing the data to acquire fusion characteristics;

and classifying the fusion characteristics to obtain a final characteristic diagram.

Images