CN111507454B - Improved cross cortical neural network model for remote sensing image fusion - Google Patents

Abstract

Due to the limitation of the optical sensor itself, the captured multi-hyperspectral images inevitably need to sacrifice their spatial resolution while obtaining high spectral resolution. The invention proposes an improved cross-cortical neural network model, which can fuse high spatial resolution detail information into multi-hyperspectral remote sensing images, thereby obtaining a fusion image with both high spatial resolution and spectral resolution. The comparative experimental results show that the method of the present invention is superior to the classical remote sensing image fusion method, and has less spectral distortion and detail distortion.

Description

An improved cross-cortical neural network model for remote sensing image fusion

technical field

The invention relates to the technical field of remote sensing image processing, in particular to a fusion method of multi-spectral remote sensing images.

Background technique

Multispectral and hyperspectral remote sensing images are important data sources for remote sensing image classification and interpretation. However, due to the limitations of the sensor's own signal-to-noise ratio and communication downlink, the optical remote sensing sensor has to be designed at the beginning of the spatial resolution and spectral resolution. This compromise makes the interpretation and monitoring of complex targets with rich spectral information very difficult, which greatly limits the practical application of multi-hyperspectral images. Therefore, it is necessary to use remote sensing image fusion technology to The spatial resolution image and multi-hyperspectral image are fused, so that the fusion result has high spatial resolution and spectral resolution at the same time.

Classical remote sensing image fusion algorithms include Gram-Schmidt fusion method, Brovey transform fusion method, principal component analysis PCA fusion method and IHS fusion method, etc. The present invention proposes an improved cross-cortical neural network model and applies it to multi-hyperspectral Compared with the traditional remote sensing image fusion method, this algorithm can fuse the high spatial resolution detail features better, and at the same time, it can greatly reduce the spectral distortion of the multi-hyperspectral images in the fusion result.

SUMMARY OF THE INVENTION

In order to make up for the deficiencies of the prior art, the purpose of the present invention is to provide an improved cross-cortical neural network model to solve the fusion problem of multispectral and hyperspectral remote sensing images, and the fused images can have high spatial resolution and spectral resolution at the same time. At the same time, it can better maintain the spatial details and greatly reduce the spectral distortion in the fusion process.

In order to achieve the above purpose, the present invention proposes an improved cross-cortical neural network model, and its neuron mathematical expression is:

E _ij [n]=gE _ij [n-1]+hY _ij [n-1]

Among them, ij is the current neuron, kl is the neighborhood neuron, n is the current iteration number, W and α are the neighborhood connection strength matrix and connection coefficient, respectively, S is the multi-hyperspectral image, D is the high spatial resolution detail image, g and h are the decay coefficient and normalization constant, respectively, E is the activity threshold, Y is the output pulse, F is the output fusion result, once F _ij is greater than the activity threshold E _ij in the current iteration , the neuron ij is fired at the nth In the next iteration, an output pulse Y _ij is generated, and the final fusion result F is obtained when all neurons in the neural network are fired.

In order to adapt to the remote sensing image fusion algorithm, each pixel of the remote sensing image corresponds to each neuron in the neural network model of the present invention. Before using the neural network model to process the multi-hyperspectral and high spatial resolution images, the input Perform a normalization operation on the images, normalize their pixel values to [0,1], and perform a histogram matching operation on the normalized images to obtain the normalized multi-hyperspectral image S and high spatial resolution image H. The spatially resolved image H performs Gaussian smoothing filtering to obtain the smoothed image HL , where the distribution parameter σ of the Gaussian filter is:

where M is the filter length, R is the spatial scale factor between the multi-hyperspectral image and the high spatial resolution image, and G is the modulation transfer function of the multi-hyperspectral image sensor, from which the detailed image D = H - HL can be obtained. , after obtaining the standardized multi-hyperspectral image S and detail image D , it is used as the input of the improved cross-cortical neural network model for iterative calculation.

The initial value of each variable of the neural network is set as, Y [0]= F [0]=0, E [0]=1, n =1; α is calculated as follows:

where Std and Con represent standard deviation and covariance calculations, respectively.

Each iteration of the neural network adds one to the current number of iterations n until all neurons are excited to obtain the output F , and perform inverse normalization on F , that is, expand the range of pixel values in F to obtain a fusion of multi-hyperspectral images As a result, if the total number of multi-hyperspectral image channels is K, the above fusion processing is performed on these K channels respectively, and the final fusion result of multi-hyperspectral image and high spatial resolution image with K channels can be obtained.

The beneficial effects of the present invention are: 1. The traditional cross-cortical neural network model only allows one external excitation input, and the improved model has two external excitation inputs S and D , which is beneficial to the more convenient application of the cross-cortical neural network principle to image fusion 2. The model of the present invention can be applied to the fusion of remote sensing images with different scales due to the consideration of the detail injection operation; 3. The use of the model of the present invention can better maintain the detailed characteristics of high spatial resolution images, and at the same time can greatly Reduces spectral distortion of fusion results.

Description of drawings

FIG. 1 is a flowchart of the remote sensing image fusion method of the present invention.

FIG. 2 is a structural diagram of an improved cross-cortical neural network model of the present invention.

FIG. 3 is an input image and a fusion result according to an embodiment of the present invention.

Detailed ways

In order to facilitate the understanding of the technical means realized by the present invention, the achieved objects and effects, the present invention is further described below.

The flowchart of the remote sensing image fusion method of the present invention is shown in Figure 1. The overall process is to first perform normalization processing in the [0,1] interval on the input high spatial resolution images and multi-hyperspectral images; Resolve the image to extract details, and send the detailed image and the high spatial resolution image into the model of the present invention as input, wherein, the improved cross-cortical neural network model structure of the present invention is shown in Figure 2, and the network parameters are set as the neighborhood connection strength matrix W = [0.5,1,0.5;1,0,1;0.5,1,0.5], attenuation coefficient g = 0.65, normalization constant h = 20.

The neuron ij corresponds to the image pixel one-to-one. When all neurons in the neural network are excited, the final fusion result F is obtained, and the K channels of the multi-hyperspectral spectrum are respectively performed to obtain the final fusion of K independent channels. result.

The input high spatial resolution grayscale image, multi-hyperspectral image and fusion results are shown in Figure 3, where Figure 3(a) is the input high spatial resolution panchromatic grayscale image, and Figure 3(b) is the input Multi-hyperspectral images, the input images were collected from Quickbird high-resolution sensors, and the spatial resolutions were 0.7m and 2.8m, respectively. Figure 3(c) is the fusion result. It can be seen from Figure 3 that this remote sensing fusion method simultaneously obtained high spatial and High spectral resolution with good detail and spectral features.

Table 1 provides the evaluation and comparison results of classical remote sensing image fusion methods such as the method of the present invention and other Gram-Schmidt fusion methods, Brovey transform fusion methods, principal component analysis PCA fusion methods and IHS fusion methods, and the comparison and evaluation index adopts spectral angle matching degree SAM, relative global error ERGAS and Q index index, the mathematical expression of the evaluation index is as follows:

Among them, <> represents the inner product operation, RMSE represents the root mean square operation, σ and μ represent the covariance and mean of the image, respectively, and the spectral angle matching degree SAM in the evaluation index is a measure of the spectral distortion of the remote sensing image. The better the effect, the relative global error ERGAS represents the degree of detail distortion between the fusion result and the high spatial resolution image. The smaller the value, the better the fusion effect. The Q index is a comprehensive evaluation of the spectral distortion and spatial detail preservation of the fusion image. The larger the value, the better the fusion quality.

It can be seen from the evaluation index calculation results in Table 1 that the Q index index of the method of the present invention is higher than other classical Gram-Schmidt fusion methods, Brovey transform fusion methods, principal component analysis PCA fusion methods and IHS fusion methods, etc. Method Spectral distortion index SAM and detail distortion index ERGAS are both smaller than other classical algorithms. It can be seen that the method of the present invention is much better than the classical method in the preservation of spectral distortion and spatial details.

Claims (2)

1. a method for constructing an improved cross-cortical neural network model for remote sensing image fusion, is characterized in that, the described improved cross-cortical neural network model is specifically:

E _ij [n]=gE _ij [n-1]+hY _ij [n-1]

Among them, ij is the current neuron, kl is the neighborhood neuron, n is the current iteration number, W and α are the neighborhood connection strength matrix and connection coefficient, respectively, S is the multi-hyperspectral image, and D is the detail of high spatial resolution Image, g and h are the attenuation coefficient and normalization constant, respectively, E is the activity threshold, Y is the output pulse, and F is the output fusion result. 2. an application method for the improved cross-cortical neural network model for remote sensing image fusion, is characterized in that, comprises the following steps: Step 1: Normalize the input multi-hyperspectral image and high spatial resolution image to [0, 1], and perform a histogram matching operation to obtain S ^k , where k=1, ..., K is the spectral channel number; Step 2: Perform Gaussian low-pass filtering that satisfies the modulation transfer function on the high spatial resolution image to obtain a detailed image D; Step 3: Execute the improved cross-cortical neural network model of claim 1 for each k channel until all neurons are excited to obtain the output F ^k ; Step 4: Perform inverse normalization on the output F ^k pixel values to obtain the final fusion result.

Images