CN110443172A - A kind of object detection method and system based on super-resolution and model compression - Google Patents

Abstract

The present invention provides a target detection method and system based on super-resolution and model compression. The method includes: using an image with high resolution as a training label, using a corresponding image with low resolution as a training sample, and training super-resolution use the trained super-resolution model to process the low-resolution images to be processed to generate corresponding high-resolution images; improve the Faster‑RCNN network according to the preset method, and improve the high-resolution images generated by the super-resolution model. The target detection model is trained using the improved Faster‑RCNN network; the trained target detection model is compressed by the preset model compression method, so that it can be deployed on smart terminals. The solution of the present invention can realize real-time target detection for low-resolution satellite images captured by satellites, and has good detection accuracy.

Description

A target detection method and system based on super-resolution and model compression

technical field

The invention relates to the technical field of target detection, in particular to a target detection method and system based on super-resolution and model compression.

Background technique

Target detection technology is a field of computer vision that has experienced rapid technological development in recent years. The epoch-making significance in this field is the RCNN series of target detection networks. Among the RCNN series, the Faster-RCNN network has the best performance and the fastest speed. However, for target detection on a large number of low-resolution satellite images captured by satellites, the detection accuracy of existing target detection methods is not ideal. ; In addition, although there are many network models with high detection accuracy at this stage, such as Faster-RCNN, Fast-RCNN and so on. However, these networks are slower than networks such as YOLOv3 without model compression, so they cannot truly achieve real-time target detection.

SUMMARY OF THE INVENTION

The purpose of the present invention is to solve the problem of target detection on a large number of low-resolution satellite images captured by satellites, and the existing target detection methods cannot achieve real-time accurate detection; In the case of compression, the network speed is slower than that of YOLOv3, so it cannot really realize the problem of real-time target detection.

In order to solve the above technical problems, the present invention provides a target detection method based on super-resolution and model compression, and the method includes:

Use high-resolution images as training labels, and use corresponding low-resolution images as training samples to train the super-resolution model;

Use the trained super-resolution model to process the low-resolution images to be processed to generate corresponding high-resolution images;

Improve the Faster-RCNN network according to the preset method, and use the improved Faster-RCNN network to train the target detection model for the high-resolution images generated by the super-resolution model;

A preset model compression method is used to compress the trained target detection model so that it can be deployed on smart terminals.

Further, the Faster-RCNN network is improved according to a preset method, including:

Change the base network of Faster-RCNN network from VGG16 to ResNeXt101.

Further, the improvement of the Faster-RCNN network according to the preset method also includes:

Replace the convolutional layers in the Faster-RCNN network with worm-eaten convolutional layers.

Further, the improvement of the Faster-RCNN network according to the preset method also includes:

The FPN network is used on the base network.

Further, using the preset model compression method to compress the trained target detection model so that it can be deployed on the intelligent terminal, specifically:

The gamma parameter of the BN layer behind the convolutional layer in the improved Faster-RCNN network is used for model pruning, and the trained target detection model is compressed so that it can be deployed on smart terminals.

Correspondingly, in order to solve the above technical problems, the present invention also provides a target detection system based on super-resolution and model compression, and the system includes:

The super-resolution model training module is used to use high-resolution images as training labels, and use corresponding low-resolution images as training samples to train super-resolution models;

The high-resolution image generation module is used to process the low-resolution images to be processed by using the trained super-resolution model to generate corresponding high-resolution images;

The target detection model training module is used to improve the Faster-RCNN network according to the preset method, and use the improved Faster-RCNN network to train the target detection model for the high-resolution images generated by the super-resolution model;

The model compression module is used to compress the trained target detection model by using a preset model compression method, so that it can be deployed on the intelligent terminal.

Further, the target detection model training module is specifically used for:

Change the base network of Faster-RCNN network from VGG16 to ResNeXt101.

Further, the target detection model training module is also used for:

Replace the convolutional layers in the Faster-RCNN network with worm-eaten convolutional layers.

Further, the target detection model training module is also used for:

The FPN network is used on the base network.

Further, the model compression module is specifically used for:

The gamma parameter of the BN layer behind the convolutional layer in the improved Faster-RCNN network is used for model pruning, and the trained target detection model is compressed so that it can be deployed on smart terminals.

The beneficial effects of the above-mentioned technical solutions of the present invention are as follows:

The present invention trains a super-resolution model by using an image with high resolution as a training label and a corresponding image with low resolution as a training sample; and uses the trained super-resolution model to process the low-resolution image to be processed. , generate corresponding high-resolution images; improve the Faster-RCNN network according to the preset method, and use the improved Faster-RCNN network to train the target detection model for the high-resolution images generated by the super-resolution model; use the preset The model compression method compresses the trained target detection model so that it can be deployed on smart terminals. Therefore, the effect of real-time target detection on the low-resolution satellite image captured by the satellite is realized, and the detection accuracy is good.

Description of drawings

1 is a schematic flowchart of a target detection method based on super-resolution and model compression of the present invention;

Figure 2 is a schematic diagram of the overall architecture of the RDN network;

Fig. 3 is the structural representation of RDB module;

Figure 4 is a schematic diagram of the structure comparison between ResNeXt50 and ResNet50;

Figure 5 is a schematic diagram of the block structure comparison between ResNet and ResNeXt;

FIG. 6 is a schematic structural diagram of an FPN network;

FIG. 7 is a schematic diagram of the steps of model pruning.

Detailed ways

In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

first embodiment

This embodiment provides a target detection method based on super-resolution and model compression, and the method involves image super-resolution technology, target detection technology and model compression technology. Among them, the high-precision target detection technology is fundamental, and the image super-resolution and model compression technology are used to make the method of this embodiment perform better in solving the target detection task.

Super-resolution technology refers to a technology that reconstructs a corresponding high-resolution image from an observed low-resolution image. If the resolution of the picture is too low, it will greatly affect the target detection and semantic segmentation in computer vision, because the picture with low resolution will lose a lot of picture information, so this embodiment uses the super-resolution technology to first Convert images to high-resolution images.

Model compression technology is a technology that has been widely used in recent years to deploy deep learning technology on smart devices. It is widely used in smart phones, smart wearable devices and other terminals where hardware conditions are not particularly strong. From the original MobileNet, SqueezeNet and model pruning methods to today's automatic model pruning, model retraining and other methods, intelligent terminals can deploy artificial intelligence technology on a large scale.

Specifically, as shown in FIG. 1 , the target detection method based on super-resolution and model compression in this embodiment includes:

S101, using an image with a high resolution as a training label, and using a corresponding image with a low resolution as a training sample, to train a super-resolution model;

S102, using the trained super-resolution model to process the low-resolution image to be processed to generate a corresponding high-resolution image;

It should be noted that the functions of the above S101 and S102 are to use super-resolution technology to make the original blurred satellite images clear; the super-resolution model in S101 uses the RDN network, and its network architecture is shown in Figure 2 It is shown that the RDN network has fewer parameters and is suitable for use as a model for generating high-resolution images in real time; the network adopts the structure of RDB (Residual dense block) in order to utilize as many features as possible and achieve training convergence as soon as possible. The structure of the RDB is shown in Figure 3.

S103, improve the Faster-RCNN network according to the preset method, and use the improved Faster-RCNN network to train the target detection model for the high-resolution images generated by the super-resolution model;

It should be noted that the above steps are to use the improved Faster-RCNN network for target detection. At the beginning of the proposal, Faster-RCNN used the VGG16 network as the basic network to extract features from the original image, and then input the extracted feature map into the RPN network, which used a 3*3 window on the feature map. Sliding and applying 9 kinds of anchors to the original image, the final training retains a certain number of ROIs as the input of the next step.

The first improvement of this embodiment is to change the basic network from VGG16 to ResNeXt101. The ResNeXt101 network has 101 convolutional layers, and each convolutional layer divides the feature map into 32 equal parts along the channel direction, which enables better feature extraction for the original image; To a certain extent, it solves the problem of gradient disappearance; finally, compared with ResNet101, ResNeXt101 does not bring additional parameters, which has no effect on the speed of training and derivation process. Figure 4 is a schematic diagram of the structure comparison between ResNeXt50 and ResNet50.

As can be seen from Figure 4, although the number of channels of the output feature map in each convolutional layer has doubled, the parameters of the ResNeXt network have not increased, but have decreased to a certain extent. Figure 5 is a schematic diagram of the block structure comparison between ResNeXt and ResNet.

The second improvement made to the Faster-RCNN network in this embodiment is to replace the convolutional layer with an Atrous Convolution layer. Using this convolutional method can increase the receptive field of the feature map. The idea of using worm-eaten convolution comes from the InceptionV3 network. This semantic segmentation network uses worm-eaten convolution to increase the receptive field of feature maps during the convolution process. And increasing the receptive field of the feature map is also very important for the target detection task, so in this embodiment, the ordinary convolutional layers are replaced by worm-eaten convolutional layers.

The third improvement made to the Faster-RCNN network in this embodiment is that the FPN (Feature Pyramid Network) network is used on the basic network. , while the original network only uses the features of the last layer. After using FPN, the original Faster-RCNN network's 9 region proposals for one anchor are boosted to 15.

S104, using a preset model compression method to compress the trained target detection model, so that it can be deployed on the intelligent terminal.

It should be noted that the target detection network such as Faster-RCNN has many model parameters, the speed is slow, and the hardware requirements are higher; while the target detection network model such as YOLOv3 has fewer parameters, faster speed, and can achieve real-time Object detection task, but the accuracy of the model is relatively low. Therefore, model compression is an effective method that can balance detection accuracy and detection speed.

The theoretical basis of this method is that not all neurons are activated when human beings are thinking about problems. Therefore, in the deep learning model, the present invention removes some unnecessary connections to reduce the number of parameters of the model and speed up the deep learning model. speed.

In the field of model compression, the most commonly used method is model pruning, which determines which connections to prune according to the performance of the model after pruning. The basic network adopted in this embodiment is ResNet101, and a BN (Batch Normalization) layer is connected behind each convolutional layer. Therefore, the gamma parameter of the BN layer can be used for model pruning. In order to use the gamma parameter of the BN layer, it is necessary to add the L1 regular penalty to the gamma parameter to train the model, and the new loss function becomes:

L=∑ _(x,y) l(f(x,W),y)+μ∑ _γ∈τ g(γ).

Sort all the gammas in the network, remove the channel with small gamma according to the artificially given pruning ratio, and finally perform fine-tuning. This process is repeated many times to obtain better results. The specific process is shown in Figure 7. Show.

In this embodiment, the super-resolution model is trained by using images with high resolution as training labels and corresponding images with low resolution as training samples; processing to generate corresponding high-resolution images; improve the Faster-RCNN network according to the preset method, and use the improved Faster-RCNN network to train the target detection model for the high-resolution images generated by the super-resolution model; use the preset The model compression method compresses the trained target detection model so that it can be deployed on smart terminals. The effect of real-time target detection on low-resolution satellite images captured by satellites is realized, and the detection accuracy is good.

Second Embodiment

This embodiment provides a target detection system based on super-resolution and model compression. The target detection system based on super-resolution and model compression includes:

The super-resolution model training module is used to use high-resolution images as training labels, and use corresponding low-resolution images as training samples to train super-resolution models;

The high-resolution image generation module is used to process the low-resolution images to be processed by using the trained super-resolution model to generate corresponding high-resolution images;

The target detection model training module is used to improve the Faster-RCNN network according to the preset method, and use the improved Faster-RCNN network to train the target detection model for the high-resolution images generated by the super-resolution model;

The model compression module is used to compress the trained target detection model by using a preset model compression method, so that it can be deployed on the intelligent terminal.

The target detection system based on super-resolution and model compression in this embodiment corresponds to the target detection method based on super-resolution and model compression in the first embodiment. Each process step in the above method corresponds to each other; therefore, it is not repeated here.

In addition, it should be noted that those skilled in the art should understand that the embodiments of the embodiments of the present invention may be provided as methods, apparatuses, or computer program products. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media having computer-usable program code embodied therein, including but not limited to disk storage, CD-ROM, optical storage, and the like.

Embodiments of the present invention are described with reference to flowcharts and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the present invention. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, embedded processor or other programmable data processing terminal to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing terminal produce Means implementing the functions specified in one or more of the flowcharts and/or one or more blocks of the block diagrams.

These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal equipment to operate in a particular manner, such that the instructions stored in the computer readable memory result in an article of manufacture comprising instruction means, the The instruction means implement the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams. These computer program instructions can also be loaded on a computer or other programmable data processing terminal equipment, so that a series of operational steps are performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby executing on the computer or other programmable terminal equipment The instructions executed on the above provide steps for implementing the functions specified in the flowchart or blocks and/or the block or blocks of the block diagrams.

Although the preferred embodiments of the present invention have been described, additional changes and modifications to these embodiments may occur to those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications that fall within the scope of the embodiments of the present invention.

It should also be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or terminal device comprising a series of elements includes not only those elements, but also other elements not expressly listed or inherent to such process, method, article or terminal equipment. Without further limitation, an element defined by the phrase "comprises a..." does not preclude the presence of additional identical elements in the process, method, article or terminal device comprising said element.

The above are the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications can be made. These improvements and modifications It should also be regarded as the protection scope of the present invention.

Claims (10)

1. a kind of object detection method based on super-resolution and model compression characterized by comprising

Label using the image of high resolution as training, the sample using the low image of corresponding resolution ratio as training, Training super-resolution model；

Low-resolution image to be processed is handled using trained super-resolution model, generates corresponding high-resolution Image；

Faster-RCNN network is improved according to presetting method, to the high-definition picture that super-resolution model generates, is made With improved Faster-RCNN network training target detection model；

Trained target detection model is compressed using preset model compression method, enables it to be deployed to intelligence In terminal.

2. the object detection method based on super-resolution and model compression as described in claim 1, which is characterized in that described right Faster-RCNN network is improved according to presetting method, comprising:

The basic network of Faster-RCNN network is changed to ResNeXt101 by VGG16.

3. the object detection method based on super-resolution and model compression as claimed in claim 2, which is characterized in that described right Faster-RCNN network is improved according to presetting method, further includes:

Convolutional layer in Faster-RCNN network is replaced with into worm-eaten convolutional layer.

4. the object detection method based on super-resolution and model compression as claimed in claim 3, which is characterized in that described right Faster-RCNN network is improved according to presetting method, further includes:

FPN network is used on the basic network.

5. the object detection method based on super-resolution and model compression as claimed in claim 4, which is characterized in that described to adopt Trained target detection model is compressed with preset model compression method, enables it to be deployed to intelligent terminal On, specifically:

Model beta pruning is carried out using BN layers behind the convolutional layer in improved Faster-RCNN network of gamma parameters, to instruction The target detection model perfected is compressed, and enables it to be deployed on intelligent terminal.

6. a kind of object detection system based on super-resolution and model compression characterized by comprising

Super-resolution model training module, the label for the image using high resolution as training, using corresponding resolution Sample of the low image of rate as training, training super-resolution model；

High-definition picture generation module, for using trained super-resolution model to low-resolution image to be processed into Row processing, generates corresponding high-definition picture；

Target detection model training module, for being improved to Faster-RCNN network according to presetting method, to super-resolution The high-definition picture that model generates, uses improved Faster-RCNN network training target detection model；

Model compression module is made for being compressed using preset model compression method to trained target detection model Obtaining it can be deployed on intelligent terminal.

7. the object detection system based on super-resolution and model compression as claimed in claim 6, which is characterized in that the mesh Mark detection model training module is specifically used for:

The basic network of Faster-RCNN network is changed to ResNeXt101 by VGG16.

8. the object detection system based on super-resolution and model compression as claimed in claim 7, which is characterized in that the mesh Mark detection model training module is also used to:

Convolutional layer in Faster-RCNN network is replaced with into worm-eaten convolutional layer.

9. the object detection system based on super-resolution and model compression as claimed in claim 8, which is characterized in that the mesh Mark detection model training module is also used to:

FPN network is used on the basic network.

10. the object detection system based on super-resolution and model compression as claimed in claim 9, which is characterized in that described Model compression module is specifically used for:

Model beta pruning is carried out using BN layers behind the convolutional layer in improved Faster-RCNN network of gamma parameters, to instruction The target detection model perfected is compressed, and enables it to be deployed on intelligent terminal.