Disclosure of Invention
The invention provides a light-weight arbitrary-scale double-time-phase image change detection method for overcoming the defect that the conventional detection method cannot realize change detection of multi-scale remote sensing images.
The primary objective of the present invention is to solve the above technical problems, and the technical solution of the present invention is as follows:
a light-weight arbitrary-scale double-time-phase image change detection method comprises the following steps:
s1: acquiring a change detection data set based on the existing high-resolution remote sensing image; simultaneously acquiring double-temporal images of a region to be detected and constructing a multi-resolution change detection data set;
s2: constructing a light-weight arbitrary scale change detection model;
s3: pre-training a light-weight random scale change detection model by using the change detection data set based on the high-resolution remote sensing image;
s4: training the whole model constructed in the step S2 by using a multi-resolution change detection data set to obtain a trained light-weight arbitrary scale change detection model;
s5: and inputting the double time phase images with different resolutions of the region to be detected into the trained model to obtain a change result graph of the region to be detected.
Further, the specific process of acquiring the existing change detection dataset based on the high-resolution remote sensing image in step S1 is as follows:
s101: determining a data source of a change detection data set of the high-resolution remote sensing image;
s102: and selecting a data set meeting preset conditions according to the determined data source for downloading.
Further, the specific steps of obtaining the image of the area to be detected and constructing the multi-resolution change detection data set are as follows:
selecting a sample region in a region to be detected, and acquiring a clear and cloudless double-time-phase image with the same resolution in the sample region;
preprocessing and geographically registering the double-time images according to the acquired data description document of the double-time images;
marking the area with changed land coverage on the image of the sample area after geographic registration by visual interpretation, converting label vector data into raster data after marking is finished, collecting samples from the image of the sample area after geographic registration and a change mark, and constructing a change detection data set with the same resolution;
and (3) carrying out double-thrice down-sampling on the image of the later time phase of the double-time-phase image in the change detection data set with the same resolution ratio by N times to simulate a low-resolution image, so as to obtain a multi-resolution change detection data set.
Further, the lightweight arbitrary scale change detection model includes: the super-resolution module is used for resampling the low-resolution image to the size of a high-resolution image; and the change detection module adopts a weight-sharing feature extractor to extract multi-level features and carries out change detection.
Further, the pre-training of the light-weight arbitrary scale change detection model by using the change detection data set based on the high-resolution remote sensing image is a specific process of pre-training the change detection module of the light-weight arbitrary scale change detection model by using the change detection data set based on the high-resolution remote sensing image, which comprises the following steps:
adjusting training parameters, the training parameters including: training times, batch size, an optimizer, initial learning rate setting and a loss function;
and training a change detection module of the model based on the high-resolution remote sensing image change detection data set, and storing parameters of the model with the optimal precision.
Further, the specific steps of training the whole model constructed in step S2 by using the multiresolution change detection data set to obtain a trained lightweight arbitrary scale change detection model are as follows:
loading parameters of a pre-trained change detection module for initializing the model;
adjusting training parameters, wherein the training parameters comprise training times, batch size, an optimizer, initial learning rate setting and a loss function, training the whole model by using a multi-resolution change detection data set with N times of resolution difference, the super-resolution module is used for resampling a low-resolution image to a high-resolution image, and the super-resolution module and the change detection module set different initial learning rates;
and obtaining the light-weight arbitrary scale change detection model.
Further, the specific process of step S5 is: inputting the double time phase images with different resolutions of the region to be detected into the trained model to obtain a change result graph of the region to be detected, and combining the change result graphs to obtain a complete region change detection graph.
The invention also provides a light-weight arbitrary-scale double-time-phase image change detection system, which comprises: a memory and a processor, wherein the memory includes a lightweight arbitrary-scale double-temporal image change detection method program, and the lightweight arbitrary-scale double-temporal image change detection method program, when executed by the processor, implements the steps of:
s1: acquiring a change detection data set based on the existing high-resolution remote sensing image; simultaneously acquiring an image of a region to be detected and constructing a multi-resolution change detection data set;
s2: constructing a light-weight arbitrary scale change detection model;
s3: pre-training a light-weight random scale change detection model by using the change detection data set based on the high-resolution remote sensing image;
s4: training the whole model constructed in the step S2 by using a multi-resolution change detection data set to obtain a trained light-weight arbitrary scale change detection model;
s5: and inputting the double time phase images with different resolutions of the region to be detected into the trained model to obtain a change result graph of the region to be detected.
Further, the specific process of acquiring the existing change detection dataset based on the high-resolution remote sensing image in step S1 is as follows:
s101: determining a data source of a change detection data set of the high-resolution remote sensing image;
s102: and selecting a data set meeting preset conditions according to the determined data source for downloading.
The invention also provides a computer readable storage medium, which includes a lightweight arbitrary-scale double-temporal-phase image change detection method program, and when the lightweight arbitrary-scale double-temporal-phase image change detection method program is executed by a processor, the steps of the lightweight arbitrary-scale double-temporal-phase image change detection method are realized.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that:
the invention provides a light-weight arbitrary scale change detection method, which comprises the steps of constructing a light-weight arbitrary scale change detection model, overcoming the resolution difference between double-time-phase images by using a super-resolution module in the model, recovering a low-resolution image to the size of a high-resolution image, extracting multi-level features in the double-time-phase images by using a change detection module, and further realizing the change detection of the double-time images.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited by the specific embodiments disclosed below.
Example 1
As shown in fig. 1, a lightweight arbitrary-scale two-time phase image change detection method includes the following steps:
s1: acquiring a change detection data set based on the existing high-resolution remote sensing image;
the specific process of step S1 is:
s101: determining a data source of a change detection data set of the high-resolution remote sensing image;
s102: and selecting a data set meeting preset conditions according to the determined data source for downloading.
It should be noted that, in the embodiment of the present invention, data sources may be arranged through existing data sites, such as ieee xplore, Github, and the like, and a data set with a relatively large data volume, a better data quality, and an image type closer to an image of a detection task is selected for downloading for pre-training of a subsequent model. In one embodiment, a CCD change detection data set comprising 16000 256-256 image pairs comprising 10000 training samples, 3000 verification samples and 3000 test samples with an image resolution of 0.03-1 m can be used. The method not only provides the change information of common objects such as buildings, roads, forests and the like, but also provides the change information of a plurality of detailed objects such as automobiles, tanks and the like.
Acquiring a double-temporal image of a region to be detected and constructing a multi-resolution change detection data set;
the method comprises the following specific steps:
selecting a sample region in a region to be detected, and acquiring a clear and cloudless double-time-phase image with the same resolution in the sample region;
preprocessing and geographically registering the double-time images according to the acquired data description document of the double-time images;
marking the area with changed land coverage on the image of the sample area after geographic registration by visual interpretation, converting label vector data into raster data after marking is finished, collecting samples from the image of the sample area after geographic registration and a change mark, and constructing a change detection data set with the same resolution;
and (3) carrying out double-thrice down-sampling on the image of the later time phase of the double-time-phase image in the change detection data set with the same resolution ratio by N times to simulate a low-resolution image, so as to obtain a multi-resolution change detection data set.
It should be noted that, firstly, remote sensing image data of an area to be predicted is acquired on an open website, if the resolution of the obtained image of the area to be predicted, which meets the time and area range required by the detection task, is inconsistent due to the limitation of the cloud amount and the re-returning period, that is, there is a resolution difference of N times, therefore, a sufficient sample area is selected in the area to be predicted, and a double-time-phase image with the same resolution and a clear and non-cloud sample area is downloaded on the website. Then, preprocessing and geographic registration are carried out on the double-time images according to the acquired data description documents of the double-time images;
the processing dimension of the pretreatment comprises: DN value of data, projection coordinate system, radiometric calibration, radiometric correction, mosaic, clipping, histogram matching, etc. Marking the area with changed land coverage on the image of the sample area after geographic registration by visual interpretation, converting label vector data into raster data after marking is finished, collecting samples from the image of the sample area after geographic registration and a change mark, and constructing a change detection data set with the same resolution; and (3) carrying out double-thrice down-sampling on the image of the later time phase of the double-time-phase image in the change detection data set with the same resolution ratio by N times to simulate a low-resolution image, so as to obtain a multi-resolution change detection data set. It should be noted that the constructed multiresolution change detection data set is used for model training, and simultaneously, the double time phase images of the to-be-detected region with the resolution difference of N times in the multiresolution change detection data set are cut for later use for subsequent detection.
In a specific embodiment, the multiresolution change detection dataset is constructed using a google earth image-based change detection dataset comprising 19 pairs of images with a spatial resolution of 0.55 meters, ranging in size from 1006 x 1168 to 4936 x 5224. All images were taken between 2006 and 2019. The data set provides a labeling of building changes between the image pairs. We first cut the dataset into 1118 pairs of 256 × 256 sized non-overlapping samples, as per 6: 2: and 2, dividing the proportion into a training set, a verification set and a test set, and randomly rotating the training set to realize data amplification to avoid overfitting, so that a double-time phase change detection data set with the same resolution ratio is obtained.
In order to verify the validity of the proposed model, a multi-resolution change detection data set needs to be further constructed. Therefore, for the obtained double time phase change detection data set, double-cubic downsampling can be adopted to process the image of the second time phase to obtain various low-resolution images with different scales, so that the problem of multi-scale change detection in the practical application process is simulated. In a specific implementation process, the down-sampling can be performed by setting three resolution differences of 3.0, 4.0 and 5.0.
S2: constructing a light-weight arbitrary scale change detection model;
it should be noted that, in the embodiment of the present invention, the lightweight arbitrary scale change detection model includes: the super-resolution module is used for resampling the low-resolution image to the size of a high-resolution image; more specifically, the super-resolution module includes: the method comprises two parts, namely a lightweight feature extractor and an over-scale reconstruction module (OSM). The lightweight feature extractor consists of a 3 x 3 convolutional layer and 3 sense Group modules. Each Dense Group comprises 3 residual modules which are fused with an SE attention mechanism, and the effective multiplexing of information is forced through a recursive structure, so that more effective characteristics are obtained. Based on the extracted features, the over-scale reconstruction module obtains an over-scale feature map by using two convolution layers of 3 multiplied by 3 and 5 Pixel Shuffle layers, and then obtains a high-resolution image with a target size through bicubic interpolation.
The over-scale reconstruction module is used for generating over-scale features so as to realize super-resolution of any scale and obtain a high-quality super-resolution image; the change detection module adopts a weight-sharing feature extractor to extract multi-level features and detect changes, and more specifically, the change detection module constructs a feature extractor based on ResNet-18, and is used for extracting multi-level features of an input image and learning a change map based on a feature vector of a depth measurement learning double-temporal image.
S3: pre-training a light-weight random scale change detection model by using the change detection data set based on the high-resolution remote sensing image;
the specific process of pre-training the change detection module of the lightweight arbitrary scale change detection model by using the change detection data set based on the high-resolution remote sensing image is as follows:
adjusting training parameters, the training parameters including: training times, batch size, an optimizer, initial learning rate setting and a loss function;
and training a change detection module of the model based on the high-resolution remote sensing image change detection data set, and storing parameters of the model with the optimal precision.
It should be noted that, in a specific implementation process, as it takes much time and labor cost to construct a new multi-resolution data set, the number of samples of the region to be predicted is often less, and direct training is likely to cause model overfitting. Therefore, the change detection module of the constructed model is pre-trained by using the change detection data set based on the high-resolution remote sensing image, so that the change information in the existing data set is fully utilized. In the pre-training, the following training parameters may be set, the number of training times is 200, the optimizer is Adam, the initial learning rate is 0.0001, and the training batch size is 8.
S4: training the whole model constructed in the step S2 by using a multi-resolution change detection data set to obtain a trained light-weight arbitrary scale change detection model;
the specific training process is as follows:
loading parameters of a pre-trained change detection module for initializing the model;
adjusting training parameters, wherein the training parameters comprise training times, batch size, an optimizer, initial learning rate setting and a loss function, training the whole model by using a multi-resolution change detection data set with N times of resolution difference, the super-resolution module is used for resampling a low-resolution image to a high-resolution image, and the super-resolution module and the change detection module set different initial learning rates;
and obtaining the light-weight arbitrary scale change detection model.
In a specific implementation, the training parameters are set as follows: the set training times is 100, the optimizer is Adam, the training batch size is 8, and the two modules are optimized by different loss functions separately to achieve a faster convergence effect. The initial learning rate of the super-resolution module is 0.001, and the change detection module is used for fine adjustment by adopting the learning rate of 0.0001 due to the fact that the pre-training parameters are loaded.
S5: and inputting the double time phase images with different resolutions of the region to be detected into the trained model to obtain a change result graph of the region to be detected.
It should be noted that, in the embodiment of the present invention, the two time-phase images with different resolutions of the region to be detected are input into the trained model to obtain the change result graph of the region to be detected, and the change result graphs are combined to obtain the complete region change detection graph. Fig. 2 is a schematic diagram showing a change detection with a resolution difference of 3.0 times, where (a) represents a time T1 image, (b) represents a time T2 image, (c) represents a time T2 low-resolution image, (d) represents a true value image, (e) represents a super-resolution image, and (f) represents a detection result.
Fig. 3 is a schematic diagram showing a change detection with a resolution difference of 4.0 times, in which (a) shows a time T1 image, (b) shows a time T2 image, (c) shows a time T2 low-resolution image, (d) shows a true value map, (e) shows a super-resolution image, and (f) shows a detection result.
Fig. 4 is a schematic diagram showing a change detection with a 5.0-fold difference in resolution, in which (a) shows a time T1 image, (b) shows a time T2 image, (c) shows a time T2 low-resolution image, (d) shows a true value map, (e) shows a super-resolution image, and (f) shows a detection result.
The invention also provides a light-weight arbitrary-scale double-time-phase image change detection system, which comprises: a memory and a processor, wherein the memory includes a lightweight arbitrary-scale double-temporal image change detection method program, and the lightweight arbitrary-scale double-temporal image change detection method program, when executed by the processor, implements the steps of:
s1: acquiring a change detection data set based on the existing high-resolution remote sensing image; simultaneously acquiring double-temporal images of a region to be detected and constructing a multi-resolution change detection data set;
s2: constructing a light-weight arbitrary scale change detection model;
s3: pre-training a light-weight random scale change detection model by using the change detection data set based on the high-resolution remote sensing image;
s4: training the whole model constructed in the step S3 by using a multi-resolution change detection data set to obtain a trained light-weight arbitrary scale change detection model;
s5: and inputting the double time phase images with different resolutions of the region to be detected into the trained model to obtain a change result graph of the region to be detected.
Further, the specific process of acquiring the existing change detection dataset based on the high-resolution remote sensing image in step S1 is as follows:
s101: determining a data source of a change detection data set of the high-resolution remote sensing image;
s102: and selecting a data set meeting preset conditions according to the determined data source for downloading.
The invention also provides a computer readable storage medium, which includes a lightweight arbitrary-scale double-temporal-phase image change detection method program, and when the lightweight arbitrary-scale double-temporal-phase image change detection method program is executed by a processor, the steps of the lightweight arbitrary-scale double-temporal-phase image change detection method are realized.
It should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.