CN106251368B - The fusion method of SAR image and multispectral image based on BEMD - Google Patents

Abstract

A kind of fusion method of SAR image and multispectral image of the present invention, comprising: IHS is carried out to multispectral image and converts to obtain I, H, S component；BEMD is carried out respectively to SAR image and I component to convert to obtain the IMF component and residual components of SAR image and I component；The IMF component and residual components for adjusting SAR image and I component are consistent the IMF component number of the two, and corresponding residual components are added in extra IMF component；The IMF component of SAR image adjusted and I component and IMF component and residual components are merged with residual components respectively；BEMD inverse transformation is carried out according to fusion results, obtains fused I component；IHS inverse transformation is carried out to fused I component, H component and S component, obtains fused image.The fusion method of SAR image and multispectral image based on BEMD of the invention saves preferable texture information, and reliable guarantee can be provided for the specific subdivision of similar atural object, is improved to non-linear, non-stationary signal analysis and processing capacity.

Description

Fusion method of SAR image and multispectral image based on BEMD

technical field

The invention relates to the field of image processing, in particular to a fusion method of a BEMD-based SAR image and a multispectral image.

Background technique

With the rapid development of sensor technology and remote sensing technology, multi-platform, multi-layer, multi-temporal, multi-sensor, multi-spectral and multi-resolution remote sensing images constitute multi-regional and multi-source remote sensing information. Different sensors have different imaging mechanisms and work in different wavelength ranges, so the obtained images often reflect the characteristics of different aspects of ground objects. The information provided by a single sensor may be incomplete, inconsistent, or even inaccurate. In order to use the increasingly complex multi-source remote sensing image information more effectively, the related image fusion technology emerges as the times require.

Synthetic Aperture Radar (SAR) is a microwave sensor that realizes remote sensing imaging by actively emitting electromagnetic waves and receiving the returned electromagnetic wave information. Compared with optical sensors, synthetic aperture radar has all-weather, all-weather, high-resolution imaging capabilities, and it has a long range of action, a wide imaging range, and strong penetrating ability, which solves the problem of optical remote sensing sensors affected by time and weather. question.

At present, the commonly used fusion methods are: IHS transform, PCA transform, Brovey transform and wavelet transform. Among them, IHS transform, PCA transform and Brovey transform usually lead to serious spectral distortion, while wavelet transform solves the problem of spectral distortion. However, the above methods are mainly used for the fusion of multi-spectral images and panchromatic images. When the optical image and high-resolution SAR image are fused in this paper, a large amount of texture information will be lost when the above method is used, and it is difficult to classify subtle objects. It is very difficult to extract.

Hereinafter, a more detailed description of the above-mentioned prior art in the field of remote sensing image fusion is made in order to better understand the present invention.

Traditional remote sensing image fusion generally uses optical panchromatic images and multispectral images for fusion. Traditional fusion methods include: IHS transform, principal component transform (PCA), ratio fusion method, principal component analysis (K-L transform), weighted fusion method , wavelet transform, etc.

Take IHS transformation as an example: transform a RGB color space that describes the color attributes of objects, synthesized from remote sensing data of different bands obtained by optical, thermal infrared and radar (microwave), into an RGB color space with intensity (Intensity), chromaticity H (Hue). ), saturation S (Saturation) to describe the IHS color space of the image.

The fusion algorithm of IHS transform is as follows:

Positive formula:

H=arctg(v ₁ /v ₂ )

In the formula: I represents brightness, H represents chroma, S represents saturation, and v1 and v2 are intermediate variables introduced to calculate I and H.

Reverse formula:

Take wavelet transform as an example: the process of wavelet transform fusion is as follows:

The wavelet transform is defined as:

The transformation kernel function is:

in, is a basic wavelet, also known as mother wavelet, or wavelet basis, which is a bandpass function centered at t=0, and the time domain average value

The wavelet transform is to process low-frequency information on a large spatial scale and high-frequency information on a small spatial scale.

At present, the multi-scale analysis methods used in the fusion method of SAR image and optical image mainly include: two-dimensional discrete wavelet transform, translation-invariant discrete wavelet transform, and curvelet transform representing anisotropy. Those skilled in the art should understand that the wavelet transform is essentially a linear transform, and has limited processing capability for nonlinear and non-stationary signals; moreover, the wavelet transform is non-adaptive, and its fusion effect depends on the selection of the wavelet base.

Therefore, there is a need for a method for fusing high-resolution SAR images with optical images, which can overcome the shortcomings of existing methods that are relatively weak in processing nonlinear and non-stationary signals, and at the same time can maintain the spectrum in multi-source images. information, edge information and texture information.

The meanings of the abbreviations of some key terms in this specification are shown here, including: EMD One-dimensional Empirical Mode Decomposition (EMD); Two-dimensional Empirical Mode Decomposition (BEMD); Intrinsic ModeFunction (IMF); Synthetic Aperture Radar (SyntheticApertureRadar, SAR).

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a fusion method of synthetic aperture radar SAR image and multi-spectral image which can overcome the above-mentioned defects.

In a first aspect, the present invention provides a method for fusing a synthetic aperture radar SAR image and a multispectral image, characterized in that: performing IHS transformation on the multispectral image to obtain the I component, the H component and the S component of the multispectral image. components; perform two-dimensional empirical mode decomposition BEMD transformation on the SAR image and the I component, respectively, to obtain the eigenmode function IMF component and residual component of the SAR image, and the IMF component and residual component of the I component ; Adjust the IMF component and residual component of the SAR image, and the IMF component and residual component of the I component, so that the IMF component of the SAR image is consistent with the number of IMF components of the I component, and the The redundant IMF component is added to the corresponding residual component; the IMF component of the adjusted SAR image and the IMF component of the I component, and the residual component of the adjusted SAR image and the residual component of the I component are respectively fused; according to the fusion result Perform inverse BEMD transformation to obtain the fused I component; perform inverse IHS transformation on the fused I component, the H component and the S component to obtain a fused image.

Preferably, the step of BEMD transformation includes: initializing the image to be subjected to BEMD transformation; identifying extreme value points in the initialized image; when the extreme value points satisfy a predetermined threshold, The points are fitted to obtain the average envelope surface of the image; the local trend of the image is extracted according to the average envelope surface; in the case that the local trend satisfies a predetermined IMF condition, the local trend is determined as an IMF component; A residual component is calculated according to the IMF component; in the case that the residual component satisfies a predetermined monotonic condition, the residual component is determined as the final residual component.

Preferably, in the case that the local trend does not satisfy the IMF determination condition, use the local trend to replace the initialized image, and continue to find the extreme point in it; in the case that the residual component does not satisfy the monotonic condition, use The residual component replaces the initialized image, and continues to search for extreme points in it.

Preferably, the IMF determination condition is determined according to the relationship between the extreme point and the zero point in the image in which the extreme point is identified and the value of the average envelope surface; the monotonic condition is determined by two successively determined IMF components are calculated.

Preferably, before the step of performing IHS transformation on the multispectral image, the method further includes: denoising the SAR image; unifying the pixel resolution of the denoised SAR image and the multispectral image by resampling; After sampling, the SAR image and the multispectral image are registered.

Preferably, before the step of fusing, the method further includes: performing Laplace high-pass filtering on the IMF component of the adjusted SAR image and the IMF component of the I component, and performing Laplace high-pass filtering on the residual component of the adjusted SAR image and the IMF component of the I component. The residual component of the I component is subjected to Laplace low-pass filtering; the IMF component in the SAR image and the weight of each pixel in the residual component are respectively calculated based on the filtering result; wherein, the fusion is performed based on the weight.

The fusion method of SAR image and multispectral image based on BEMD of the present invention preserves better texture information, can provide reliable guarantee for the specific subdivision of similar ground objects, and improves the analysis and processing ability of nonlinear and non-stationary signals .

Description of drawings

FIG. 1 is a flowchart of a fusion method of BEMD-based SAR and multispectral imagery according to an embodiment of the present invention;

Fig. 2 is the flow chart of the BEMD transformation in Fig. 1;

Fig. 3 is a fusion result diagram of multi-source remote sensing images in Guilin area based on wavelet transform;

FIG. 4 is a result diagram of fusion results of multi-source remote sensing images in Guilin area based on the fusion method of the present invention.

Detailed ways

The technical solutions of the present invention will be further described in detail below through the accompanying drawings and embodiments.

Aiming at the defect that the existing remote sensing image fusion transformation method is relatively weak in processing nonlinear and non-stationary signals, the present invention proposes a new remote sensing image fusion transformation method. BEMD transform is suitable for the analysis of nonlinear and non-stationary signals, and has a high signal-to-noise ratio; in addition, BEMD realizes signal decomposition according to the time initial reading characteristics of the signal itself, and the final fusion effect does not depend on the selection of any basis function, and has complete self-determination. adaptability. Using the BEMD transform method, the present invention realizes the fusion of SAR image and high-resolution optical image.

First, the following steps are generally described in the image fusion method adopted in the present invention.

1) Preprocessing the object images fused by multi-source remote sensing images.

For example, the SAR image obtained by the TerraSAR-X satellite and the multispectral image obtained by the GF-1 camera can be selected for image fusion. Preferably, in order to ensure the fusion effect, preprocessing such as geo-scaling, geometric correction, image registration, filtering and denoising, etc. is performed on the two kinds of image data respectively. Since the two images have different spatial resolutions, the pixel resolutions of the images need to be unified before image registration; in addition, since the SAR images have an incident angle during acquisition, ghosting needs to be eliminated in the preprocessing process.

2) Aiming at the calculation cycle in the image fusion process, reasonably select the image pixels and the size of the image window. The image pixel size can be selected as a multiple of 8, preferably a square is the best; and the image should not be too large, for example, the image window size can be selected as 4000*4000.

3) BEMD decomposition of multispectral images and SAR images.

Multispectral images contain three bands R, G, and B, while the three components of IHS color space are relatively independent and can better reflect color information. Therefore, the multispectral image is first subjected to IHS transformation and then to BEMD decomposition, that is, the three bands are decomposed respectively. In the experiment, the decomposition can be defined to obtain four layers of components, which are three IMF components and one residual component. The first IMF component is the highest frequency information, the frequency information of the second and third components decreases sequentially, and the residual component is mainly luminance information.

The SAR image has only one band, that is, the gray band, so the gray band of the SAR image is also decomposed by BEMD.

4) Fusion of the decomposed SAR image grayscale band with the decomposed I component of the multispectral image, and then combined with the H and S components, and then converted into R, G, and B, and finally the fused image is obtained.

Below, the fusion method of SAR image and optical image based on BEMD high resolution according to the present invention will be described in detail.

FIG. 1 is a flowchart of a method for fusing a BEMD-based SAR image and a multispectral image according to an embodiment of the present invention.

The first step: image preprocessing, the process mainly includes image resampling, SAR image denoising, and image registration.

First, the pixel resolution of the two images can be unified by downsampling.

Secondly, because SAR images are easily affected by noise, the nonlinear and non-stationary signals of the image become more and more obvious with the increase of the spatial resolution of the image, and the inherent noise also becomes more prominent. Therefore, it is necessary to denoise the SAR image before sampling, for example, the improved Goldstein filter can be used to denoise the SAR image.

Finally, for the denoised, sampled SAR image and the sampled multispectral image, image point pairs with the same name are collected for image registration.

Step 2: Fusion of SAR image and multispectral image, the specific operation steps are as follows:

1. Perform IHS transformation on the multispectral image (that is, the color space conversion in Figure 1) to obtain the I, H, and S components of the multispectral image.

H=arctg(v ₁ /v ₂ )

Note: v1 and v2 are intermediate variables.

2. Perform BEMD transformation on the preprocessed SAR image and the I component of the transformed multispectral image respectively.

The IMF components and residual components R of the SAR images are obtained through the respective BEMD transformations, and the IMF components and the residual components R of the multispectral images are obtained. After being decomposed by BEMD, the SAR image can be expressed as The I component of the multispectral image can be expressed as Among them, I and R are the IMF component and residual component, respectively; M, N are the number of IMFs that can be decomposed by the I component of the SAR and multispectral images, respectively.

3. Process the image decomposition amount of SAR image and multi-spectral image so that they have the same number of decomposition layers.

Because BEMD is fully adaptive, the number of IMFs obtained by decomposing the I component of SAR image and multispectral image by BEMD may be different, resulting in inconsistency in the subsequent fusion process. Therefore, it is necessary to process the decomposition amount of the images so that the number of decomposition layers of the images to be fused is consistent. Specifically, the number of IMFs to be obtained after decomposition can be set in the BEMD transformation, so that the number of IMFs finally obtained by the two images is the same. For example, set n=min{M,N}, take n=M as an example, the IMFs components after the nth IMF of the I component of the multispectral image will be added to the residual component, so that the number of IMFs is n. At this time, the I component of the multispectral image can be expressed as:

in,

4. Perform Laplace filtering on the components obtained by BEMD transformation of SAR images and multispectral images to detect edge information in the images. For example, Laplacian high-pass filtering is performed on the IMF components of SAR images and multispectral images, and Laplacian low-pass filtering is performed on the residual components of the two. The edge information in the image mainly corresponds to the high-frequency information, that is, the IMF components obtained by BEMD respectively. Correspondingly, the low-frequency information in it is the residual component.

5. Based on the pixel window determined by the Laplacian operator, detect whether the window area is smooth or not. Through detection, it is judged whether information of the same frequency is obtained after filtering in the previous step. If it is judged to be smooth, it means that all high-frequency information is obtained by high-pass filtering, and all low-frequency information is obtained by low-pass filtering. In other words, smoothing means that the filtered information has no jumps in frequency, ie no edges.

6. In the case of judging that the area in the pixel window is smooth, it means that the frequency information is the same or similar, and it can be inferred that the pixel window is the same feature, then the average weight can be calculated; otherwise, the weight needs to be calculated separately. For example, the weight of each pixel in each component of the SAR image can be calculated according to a fusion rule based on regional features, and the weights of the IMF component and the residual component of the SAR image can be denoted as α _n and β, respectively.

7. The IMF component and residual components The fusion of different scales is carried out respectively, and the fusion formula is as follows:

R ^fuse =βR ^SF +(1-β)R ^MI

6. Perform inverse BEMD transformation, and use the fusion result as a new I component I ^fuse , where the inverse transformation formula is as follows:

7. Inversely transform the color space between the new I component I ^fuse and the H and S components of the multispectral image, that is, inversely transform the I ^fuse H ^M S ^M to the RGB space, to obtain a fused image.

FIG. 2 is a flowchart of the BEMD transformation in FIG. 1 .

In step 201, the image to be BEMD transformed is initialized.

In step 202, the extremum points of the initialized image are identified, specifically, all local maxima and minima are found.

In step 203, it is judged whether the number of extreme points satisfies a predetermined condition. If so, the flow goes to 204; otherwise, the flow goes to step 207.

In step 204, select an effective interpolation algorithm to fit the maximum value point and the minimum value point of the signal, respectively, to obtain the upper and lower envelope surfaces, and calculate the average envelope surface.

In step 205, the local trend of the image is extracted according to the average envelope;

In step 206, it is judged whether the local trend satisfies the IMF condition. If satisfied, the flow proceeds to step 207 ; otherwise, the local trend is used to replace the initialized image in step 202 , and the flow returns to step 202 .

The IMF judgment condition is:

1) For the entire dataset, |extreme point-zero point|≤1;

2) For any point on the dataset, the average envelope value determined by the local maxima and local minima is zero.

At step 207, the local trend is set as the IMF component.

In step 208, a residual component is calculated from the IMF component.

In step 209, it is judged whether the residual component calculated in step 207 satisfies the monotonic condition. If satisfied, the process ends, and the residual component is determined as the final output residual component; otherwise, the residual component is used to replace the initialization image in step 202 , and the process returns to step 202 .

The criterion for determining whether the residual component is monotonic is:

1) does not contain any IMF; or

2) The residual component is less than SD, 0.2<SD<0.3,

Usually, SD is set in the range of 0.2-0.3, that is, when SD satisfies 0.2<SD<0.3, the screening process ends.

FIG. 3 is a graph of fusion results of multi-source remote sensing images in Guilin based on wavelet transform; FIG. 4 is a graph of fusion results of multi-source remote sensing images in Guilin based on the fusion method of the present invention. It can be clearly seen from the comparison between Fig. 3 and Fig. 4 that the fusion method according to the present invention preserves better texture information, can provide reliable guarantee for the specific subdivision of similar ground objects, and improves the accuracy of nonlinear and non-stationary signals. Analytical and processing capabilities.

The present invention fuses multi-source remote sensing images based on BEMD transformation, and as the basis for remote sensing image classification, it can ensure the spectral information and edge information contained in multi-source remote sensing images while processing linear or nonlinear, stationary or non-stationary signals. And texture information is not lost. Traditional SAR and optical image fusion methods mainly use linear transformations such as two-dimensional discrete wavelet transform and translation-invariant wavelet transform. These methods largely abandon the processing of nonlinear and non-stationary signals. The fusion scheme provided by the present invention is more suitable for the analysis of nonlinear and non-stationary signals, and has a higher signal-to-noise ratio. In addition, BEMD transform is superior to wavelet transform in that it is different from wavelet transform which depends on the selection of wavelet base in signal processing. It relies on the time scale characteristics of the signal itself to achieve signal decomposition and has complete adaptability.

Professionals should be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. Interchangeability, the above description has generally described the components and steps of each example in terms of function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be implemented in hardware, a software module executed by a processor, or a combination of the two. A software module can be placed in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other in the technical field. in any other known form of storage medium.

The specific embodiments described above further describe the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (5)

1. A method for fusing a Synthetic Aperture Radar (SAR) image and a multispectral image based on BEMD is characterized by comprising the following steps:

IHS transformation is carried out on the multispectral image to obtain an I component, an H component and an S component of the multispectral image;

respectively carrying out two-dimensional empirical mode decomposition (BEMD) transformation on the SAR image and the I component to obtain an Intrinsic Mode Function (IMF) component and a residual component of the SAR image, and the IMF component and the residual component of the I component;

adjusting the IMF component and the residual component of the SAR image and the IMF component and the residual component of the I component to keep the number of the IMF component of the SAR image consistent with that of the IMF component of the I component, and adding the redundant IMF components into the corresponding residual components;

performing laplacian high-pass filtering on the IMF component of the adjusted SAR image and the IMF component of the I component, and performing laplacian low-pass filtering on the residual component of the adjusted SAR image and the residual component of the I component;

determining whether or not a detection window region is smooth based on a result of the filtering;

if smooth, calculate its average weight;

if not smooth, respectively calculating the weight of each pixel in the IMF component and the residual component in the SAR image;

wherein the fusing is performed based on the weights;

respectively fusing the IMF component and the IMF component of the I component of the adjusted SAR image and the residual component of the I component of the adjusted SAR image;

performing BEMD inverse transformation according to the fusion result to obtain a fused I component;

performing IHS inverse transformation on the fused I component, the H component and the S component to obtain a fused image;

when linear or nonlinear, stationary or non-stationary signals are processed, spectral information, edge information and texture information contained in the multi-source remote sensing image are guaranteed not to be lost.

2. The method of claim 1 for fusing a BEMD-based synthetic aperture radar SAR image with a multispectral image, wherein said step of BEMD transforming comprises:

initializing an image to be subjected to BEMD conversion;

identifying extreme points in the initialized image;

under the condition that the extreme point meets a preset threshold value, fitting the extreme point to obtain an average envelope surface of the image;

extracting the local trend of the image according to the average envelope surface;

determining the local trend as an IMF component if the local trend satisfies a predetermined IMF condition;

computing a residual component from the IMF component;

and in the case that the residual component satisfies a predetermined monotone condition, determining the residual component as a final residual component.

3. The method for fusion of a synthetic aperture radar SAR image with a multispectral image based on claim 2, characterized in that:

under the condition that the local trend does not meet the IMF judgment condition, replacing the initialized image with the local trend, and continuously searching for an extreme point;

and under the condition that the residual component does not meet the monotone condition, replacing the initialized image with the residual component, and continuously searching for an extreme point in the initialized image.

4. The method for fusion of a synthetic aperture radar SAR image with a multispectral image based on claim 2, characterized in that:

the IMF determination condition is determined according to a relationship between an extreme point and a zero point in an image in which the extreme point is identified, and a value of the average envelope surface;

the monotonic condition is calculated by two sequentially determined IMF components.

5. The method of fusing a BEMD-based synthetic aperture radar SAR image with a multispectral image according to claim 1, further comprising, before the step of IHS transforming the multispectral image:

denoising the SAR image;

unifying the pixel resolution of the denoised SAR image and the multispectral image through resampling;

and carrying out image registration on the SAR image and the multispectral image after resampling.