CN115829895A - Multi-focus image fusion method and system based on structural similarity and region segmentation - Google Patents
Multi-focus image fusion method and system based on structural similarity and region segmentation Download PDFInfo
- Publication number
- CN115829895A CN115829895A CN202211579054.9A CN202211579054A CN115829895A CN 115829895 A CN115829895 A CN 115829895A CN 202211579054 A CN202211579054 A CN 202211579054A CN 115829895 A CN115829895 A CN 115829895A
- Authority
- CN
- China
- Prior art keywords
- fusion
- image
- region segmentation
- fused
- structural similarity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a multi-focus image fusion method and a system based on structural similarity and region segmentation, wherein the method comprises the following steps: weighting a poly set source image based on a fusion strategy of visual saliency detection, and then enhancing a pre-fusion image through a multi-scale detail enhancement strategy of logarithmic energy difference to obtain a pre-fusion image with enhanced details; segmenting the pre-fused image after the details are enhanced based on the structural similarity index to obtain a region segmentation decision diagram; and performing fusion processing on the region segmentation decision graph according to a pixel selection rule to obtain a final fusion image. The system comprises: the device comprises a pre-fusion module, an enhancement module, a segmentation module and a fusion module. By using the method and the device, the pixels with different focusing attributes can be accurately distinguished and integrated together to generate the full-focus image. The multi-focus image fusion method and system based on structural similarity and region segmentation can be widely applied to the technical field of image fusion.
Description
Technical Field
The invention relates to the technical field of image fusion, in particular to a multi-focus image fusion method and system based on structural similarity and region segmentation.
Background
Because of the limitation of an optical lens in a common digital camera, it is difficult to obtain a clear image with all scenes focused by a single camera, some clear texture and detail information are generally distributed in a focusing region, and more features of related scenes or objects can be displayed, so in order to obtain complete information, so that all objects in a scene are focused and clear, a specific multi-focus image fusion method is generally used to extract focusing pixel information on different source images of the same scene, and finally the fusion of the focusing scenes is realized by integrating the pixel information, for example, in an algorithm based on deep learning, the fusion of different images is realized by using a Convolution Neural Network (CNN) based algorithm, and the CNN can realize the feature extraction of input data by the superposition of a convolution layer and a pooling layer, and finally, a full connection layer is connected to realize classification; the multi-focus image fusion algorithm based on the transform domain firstly decomposes a source image into different sub-bands, then different fusion methods are designed according to the characteristics of the different sub-bands to fuse the different sub-bands, and finally a fusion result is reconstructed, but due to multi-layer and multi-direction decomposition, the method based on the transform domain can fully extract pixel information under different scales, and has strong scene information storage capacity, however, because the pixel information of the fusion result cannot be completely from a focus area of a single source image, pixel value distortion may occur to a certain degree, and multi-layer and multi-direction decomposition also brings higher computational complexity; the algorithm based on the spatial domain can directly process the pixels on the source image, and determines the focus decision diagram by extracting the significant information of the focus area on the source image to realize the fusion of the multi-focus image, however, due to the complexity of the image, sometimes the value of the focus measurement is the largest in the out-of-focus area, so the obtained initial decision diagram has many defects, which often occurs around the boundary of the focus and out-of-focus areas, which also causes the fusion result obtained by many algorithms to have blurring or artificial artifacts at the boundary, even a large amount of pixel information of the out-of-focus area is still remained, and many methods need post-processing algorithms to make the decision diagram achieve the desired effect, such as consistency verification and small area removal, therefore, most of the focused small area can be inevitably deleted, so that the focus area is incomplete to generate suboptimal fusion result.
Disclosure of Invention
In order to solve the above technical problems, an object of the present invention is to provide a multi-focus image fusion method and system based on structural similarity and region segmentation, which can accurately distinguish pixels with different focus attributes and integrate them together to generate a full-focus image.
The first technical scheme adopted by the invention is as follows: a multi-focus image fusion method based on structural similarity and region segmentation comprises the following steps:
weighting a multi-set source image based on a fusion strategy of visual saliency detection to obtain a pre-fusion image;
enhancing the pre-fusion image through a multi-scale detail enhancement strategy of Gaussian filtering and logarithmic energy difference to obtain a pre-fusion image with enhanced details;
segmenting the pre-fused image after the details are enhanced based on the structural similarity index to obtain a region segmentation decision diagram;
and performing fusion processing on the region segmentation decision graph according to a pixel selection rule to obtain a final fusion image.
Further, the fusion strategy based on visual saliency detection performs weighting processing on the multiple sets of source images to obtain a pre-fused image, which specifically includes:
acquiring a first poly set source image and a second poly set source image;
performing visual saliency detection on the first multi-set source image and the second multi-set source image to obtain corresponding saliency values;
carrying out normalization processing on the significance value to obtain a normalized significance value;
and based on the normalized significance value, carrying out weighted average processing on the first multi-set source image and the second multi-set source image to obtain a pre-fusion image.
Further, the step of enhancing the pre-fusion image through a multi-scale detail enhancement strategy of gaussian filtering and logarithmic energy difference to obtain a pre-fusion image after detail enhancement specifically includes:
performing convolution operation on the pre-fusion image through a Gaussian function to obtain detail information of the pre-fusion image;
acquiring a high-frequency component to be fused of the pre-fused image based on the detail information of the pre-fused image;
obtaining the energy of the high-frequency component to be fused by a log-energy high-frequency component fusion algorithm, and carrying out logarithm processing to obtain a log-energy value of the high-frequency component to be fused;
calculating the difference value between log-energy of different high-frequency components to be fused, and introducing a preset threshold value for judgment;
selecting a corresponding high-frequency component fusion rule according to the judgment result to perform fusion processing on the high-frequency component to be fused to obtain a fused high-frequency component;
and adding the fused high-frequency component to the pre-fused image to obtain the pre-fused image with enhanced details.
Further, the expression of the high frequency component fusion rule is as follows:
in the above formula, FH (x, y) represents the fused high frequency component, HM (x, y) represents the decision diagram of the high frequency component, λ represents the preset threshold, DV represents the difference value between log-energy of different high frequency components to be fused,
representing different high-frequency components to be fused,
representing the weighting coefficients.
Further, the step of segmenting the pre-fusion image after the detail enhancement based on the structural similarity index to obtain a region segmentation decision map specifically includes:
acquiring a multi-set source image, and calculating the structural similarity of the multi-set source image and the pre-fusion image after the detail enhancement based on the focusing area of the image to obtain a similarity score map;
storing the similarity score chart into a matrix to generate a score chart;
based on the multi-channel signal of the image, carrying out iterative filtering processing on the score map through a recursive filter to obtain a score map;
carrying out decision processing on the fractional graph through a preset decision rule to obtain an initial two-region segmentation decision graph;
carrying out consistency detection and judgment on central pixels and peripheral pixels of the initial two-region segmentation decision map to obtain a two-region segmentation decision map, wherein the two-region segmentation decision map comprises a full focus region and a full defocus region;
analyzing the fractional graph based on a pixel difference method of RF to obtain a three-region segmentation decision graph, wherein the three-region segmentation decision graph comprises a full focus region, a full defocus region and an uncertain region;
and integrating the two-region segmentation decision diagram and the three-region segmentation decision diagram to obtain a region segmentation decision diagram.
Further, the expression of the preset decision rule is as follows:
in the above formula, IMP (x, y) represents the initial two-region segmentation decision diagram, B 1 (x，y)，B 2 (x，y)，...，B T (x, y) represents the corresponding score map.
Further, the expression of the pixel selection rule in the RF-based pixel difference method is as follows:
in the above equation, RDM (x, y) represents a three-region segmentation decision map, and β represents a parameter for controlling accuracy.
Further, the step of performing fusion processing on the region segmentation decision graph according to the pixel selection rule to obtain a final fusion image specifically includes:
comprehensively processing the two-region segmentation decision graph and the three-region segmentation decision graph through a preset pixel selection rule to obtain a final decision graph;
and (4) performing fusion processing on the final decision diagram and the pre-fusion image by considering the effects of generating complete focusing and vision to obtain a final fusion image.
Further, an expression of the preset pixel selection rule is specifically as follows:
in the above formula, FDM (x, y) represents a final decision diagram, OMP (x, y) represents a two-region partition decision diagram, and RDM (x, y) represents a three-region partition decision diagram.
The second technical scheme adopted by the invention is as follows: a multi-focus image fusion system based on structural similarity and region segmentation comprises:
the pre-fusion module is used for carrying out weighting processing on the multi-set source image based on a fusion strategy of visual saliency detection to obtain a pre-fusion image;
the enhancement module is used for enhancing the pre-fusion image through a multi-scale detail enhancement strategy of Gaussian filtering and logarithmic energy difference to obtain a pre-fusion image after detail enhancement;
the segmentation module is used for carrying out segmentation processing on the pre-fused image after the details are enhanced based on the structural similarity index to obtain a region segmentation decision diagram;
and the fusion module is used for carrying out fusion processing on the region segmentation decision graph according to the pixel selection rule to obtain a final fusion image.
The method and the system have the beneficial effects that: according to the method, a multi-set source image is subjected to weighted fusion processing through a fusion strategy of visual significance detection to obtain a pre-fusion image, wherein the source image is regarded as a combination of a full-focus area, a full-defocus area and an uncertain area, the pre-fusion image is further subjected to enhancement processing through a log-energy-based high-frequency component fusion rule, the definition of the source image can be effectively evaluated through calculating log-energy of different high-frequency components to be fused so as to reflect significant information on the source image, the source image is subjected to structural similarity index segmentation through a non-boundary area decision diagram and a boundary area decision diagram, the generated two-area segmentation decision diagram and the three-area segmentation decision diagram are used for accurately distinguishing the three areas, fusion among different focus images can be effectively realized, pixels with different focus attributes can be accurately distinguished, and are integrated to generate the full-focus image.
Drawings
FIG. 1 is a flowchart illustrating the steps of a multi-focus image fusion method based on structural similarity and region segmentation according to the present invention;
FIG. 2 is a block diagram of a multi-focus image fusion system based on structural similarity and region segmentation according to the present invention;
FIG. 3 is a schematic diagram of two multi-aggregate source images for simulation experiments according to the present invention;
FIG. 4 is a difference graph of the fusion result obtained by the prior art method and the method of the present invention and the same source image;
FIG. 5 is a block diagram of the structure of the multi-cluster image fusion method based on structure similarity and region segmentation according to the present invention.
Detailed Description
The invention is described in further detail below with reference to the figures and the specific embodiments. The step numbers in the following embodiments are provided only for convenience of illustration, the order between the steps is not limited at all, and the execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.
Referring to fig. 1 and 5, the present invention provides a multi-focus image fusion method based on structural similarity and region segmentation, the method comprising the steps of:
s1, weighting a poly set source image based on a fusion strategy of visual saliency detection to obtain a pre-fusion image;
specifically, a corresponding multi-set source image I is obtained 1 And I 2 The invention obtains the pre-fusion image by using a fusion strategy based on visual saliency detection, the visual saliency detection is a method capable of effectively detecting the salient structure of the image, the fusion strategy can effectively avoid the loss of contrast and provide good visual effect for the fusion image, the algorithm defines the pixel saliency based on the contrast of the pixel and all other pixels, and I p Expressed as the intensity value of a pixel p in the image I, the saliency value S (p) of the pixel p can be defined as,
S(p)＝|I p -I 1 |+|I p -I 2 |+…+|I p -I N |
in the above equation, N represents the total number of pixels in I, and if two pixels have the same intensity value, their significance is equal, so the above equation can be rewritten as follows:
in the above formula, j represents the pixel intensity, M j Represents the number of pixels with intensity equal to j, L represents the number of gray levels (256 in the present invention), and then S (p) is normalized to [0,1]；
Let S t Representing a source image I t Where T = (1,2, … T), where T represents the number of source images, taking the value 2 in the present invention, using S t The pre-fusion image PF can be obtained by the following weighted average rule, and the specific formula is as follows:
PF＝W F ×I 1 +(1-W F )×I 2
wherein the weight value W F It is possible to define the following,
in order to further enhance the contrast, brightness and structural information of the pre-fusion result and facilitate significant feature extraction, the invention provides a multi-scale detail enhancement strategy based on Gaussian filtering and logarithmic energy difference.
S2, enhancing the pre-fusion image through a multi-scale detail enhancement strategy of Gaussian filtering and logarithmic energy difference to obtain a pre-fusion image with enhanced details;
specifically, the detailed information at different scales is firstly obtained,
D m，t (x，y)＝I t (x，y)-I t (x，y)*G m，σ
in the above formula, D m，t (x, y) denotes a source image I t (x, y) high frequency components at scale m, representing the convolution operator, G m，σ A gaussian function representing a window size of M x M with a standard deviation of σ, where M ∈ M is a scale, and M is set to 5 in the present invention;
thereafter, the high frequency component to be fused is obtained by the following rule
However, due to the diversity and complexity of images, various images cannot be effectively processed by using a single fusion strategy, so that in order to enable the algorithm of the present invention to integrate the advantages of different fusion rules to obtain the optimal fusion performance, the present invention introduces a log-energy-based high frequency component fusion rule, selects a proper fusion strategy by analyzing the log-energy difference between different high frequency components to be fused, and can effectively evaluate the definition of a source image by calculating the log-energy of different high frequency components to be fused so as to reflect the significant information on the source image, the algorithm can be roughly divided into two steps, wherein the first step is to calculate the energy (i.e. the square of a pixel value) of the high frequency components to be fused, and the second step is to calculate the logarithm of the energy, and the calculation method is defined as follows:
calculating the difference value DV between log-energy of different high-frequency components to be fused, wherein the calculation formula is as follows:
DV＝|E 1 -E 2 |
further, a suitable fusion rule is selected by judging the relationship between DV and a threshold λ, and the expression of the fusion rule is as follows:
in the above formula, HM (x, y) represents a decision diagram of the high frequency component, FH (x, y) represents a fused high frequency component,
and
are respectively as
And
the weighting coefficient of (2);
wherein
And, the threshold λ is set to be in the present invention
Further, HM (x, y) can be obtained by the following formula, the expression of which is specifically shown below:
finally, the obtained fusion high-frequency component is added into the pre-fusion image to realize the enhancement of detail information, and the pre-fusion image EPF with enhanced details is obtained, wherein the expression of the pre-fusion image EPF is as follows:
EPF(x，y)＝IF(x，y)+FH(x，y)
s3, segmenting the pre-fused image after the details are enhanced based on the structural similarity index to obtain a region segmentation decision diagram;
specifically, in the multi-focus fusion task, a source image can be roughly divided into three regions, namely a full focus region, a full defocus region and an uncertain region, wherein the uncertain region comprises pixels of the focus and defocus regions of the image and usually exists around a focus boundary of the image, so that each pixel needs to be classified finely in order to obtain a good fusion effect;
structural Similarity (SSIM), a structural similarity index that measures image quality, is widely used in image processing to evaluate fusion performance, and separates the task of similarity measurement into three comparisons: the method comprises the steps of obtaining an integral similarity measurement by combining the three components, namely brightness, contrast and structure, calculating SSIM values between source images in different focusing areas and a pre-fused image after detail enhancement in the algorithm of the invention, carrying out structural similarity evaluation, and finally aggregating the result into a matrix, thereby obtaining a score map reflecting the brightness, the contrast and the structural information of the source images;
s31, obtaining a two-region segmentation decision diagram;
s311, acquiring a similarity score chart;
firstly, a score map SCM is obtained, and the expression of the score map SCM is as follows:
SCM t ＝ssim(I t ，EPF)
in the above formula, SSIM (-) is the SSIM operator, SCM t For a source image I t Similarity score plot with EPF. The SSIM between each image is calculated as,
in the above equation, (i, j) represents an image pair (a multi-cluster source image and a pre-fusion image after the detail enhancement processing), μ i Representing the mean value of the pixels, σ, of the source image i i Representing the standard deviation, σ, of the source image i ij Representing the covariance of the image pair (i, j), c 1 And c 2 Represents a constant; to prevent the denominator from being 0;
the range of SSIM is between [0,1], the higher SSIM indicates that the structures of two images are more similar, and finally, the calculation result is stored in a matrix to generate a score map;
then, the fractional graph obtained by the recursive filter processing is specifically expressed as follows:
B t ＝RF(SCM t ，σ s ，σ r ，N)
in the above formula, σ s 、σ r And N are set to 3, 0.25 and 3, respectively, in the present invention.
S312, acquiring a corresponding score map;
the recursive filter is an edge-preserving filter, and can reduce the influence of noise without removing significant information such as edges and textures in an image, and in the recursive filter, the distance between each point of a high-dimensional signal can be kept in a low-dimensional space, so that the high-dimensional signal can be processed by a kernel of the low-dimensional filter;
for a multi-channel signal I, the one-dimensional domain transform is implemented by the following expression, which is shown below:
in the above formula, u ∈ Ω represents a point of the signal in the original domain Ω, ct (u) maintains the geodesic distance from the origin to u in the transform domain, σ s Is the standard deviation, σ, of the filter in the spatial domain r C represents the number of channels of the signal for the standard deviation of the filter in the signal range.
For a discrete one-dimensional signal I n, the recursive filter is represented as follows,
J[n]＝(1-a d )I[n]+a d J[n-1]
in the above equation, J [ n ] represents the filtered signal at position n, and a ∈ [0,1] represents the feedback coefficient, which can be expressed by the following equation:
in the above formula, the first and second carbon atoms are,
representing the standard deviation of the filter used for the first iteration and N the total number of iterations, the invention can see sigma in each iteration i D denotes the neighboring sample x in the transform domain n And x n-1 The distance between the two can be expressed by the following formula:
the two-dimensional filtering of the image can be decomposed into one-dimensional filtering performed IN each dimension, so that when each row and each column can be regarded as a discrete signal, recursive filtering is performed, and a method of multiple iterations is adopted to implement, and different parameters are combined, so that the invention can write the recursive filter operation into RF (IN, sigma) s ，σ r N), where IN is the input image.
S313, a decision graph of two-region segmentation;
finally, an initial two-region segmentation decision map IMP is generated through the following rules,
therefore, the consistency verification technology is introduced to optimize the obtained initial decision diagram, and the consistency of the central pixel and the surrounding pixels in the fixed window is analyzed to judge whether the pixel is in the focused or out-of-focus area, which is specifically shown as follows:
in the above equation, OMP (x, y) represents the final two-region segmentation decision diagram, δ represents a square domain window centered at (x, y), and is set to 23 in the present invention;
the two-region segmentation decision map OMP obtained above only roughly divides the source image into two regions (the all-in-focus and all-out-of-focus regions).
S32, obtaining a three-region segmentation decision diagram;
to further perform three-region segmentation on the source image, we propose an RF-based pixel difference analysis scheme. Specifically, the following four steps are required for generating the boundary region segmentation decision map;
first, a Difference Map (DM) between the fractional maps obtained from the different source images is calculated, which can be mathematically expressed as:
DM(x，y)＝|SCM 1 (x，y)-SC 2 (x，y)|
then, the difference map is filtered by using RF to obtain a Differentiated Blue Map (DBM), and an expression thereof is specifically as follows:
DBM(x，y)＝RF(DM(x，y)，σ s ，σ r ，N)
in the above equation, DBM represents the result of the filtered output, and σ s 、σ r And N is set to 3, 0.25 and 3, respectively, in the present invention;
then use B t Acquiring a Blank Difference Map (BDM), wherein the expression of the BDM is shown as follows:
BDM(x，y)＝|B 1 (x，y)-B 2 (x，y)|
finally, comparing the BDM (x, y) with the DBM (x, y) to generate a three-region splitting decision graph, and only when the pixel is located in the all-in-focus or all-out-of-focus region, the BDM (x, y) can approach the DBM (x, y) of the corresponding position infinitely, otherwise, it will be smaller than the DBM (x, y) of the corresponding position, according to this feature, the present invention generates a three-region splitting decision graph (RDM) by the following pixel selection rule, the expression of which is specifically shown below:
in the above formula, β represents a parameter for controlling the accuracy, and is set to 0.5 in the present invention;
for the source image I 1 A pixel is regarded as an in-focus pixel when RDM (x, y) =1, as an out-of-focus pixel when RDM (x, y) =2, and as an uncertain pixel when RDM (x, y) =0.5, for a source image I 2 Except for indeterminate pixels and I 1 The remaining cases are complementary except that, again, the uncertain pixels are generally located on the border of, or in the surrounding area of, the in-focus and out-of-focus regions.
And S4, carrying out fusion processing on the region segmentation decision graph according to a pixel selection rule to obtain a final fusion image.
Specifically, in order to obtain a final decision graph with an accurate boundary and a complete focus region, the invention provides the following pixel selection rules to integrate the advantages of the two-region segmentation focus decision graph OMP and the three-region segmentation decision graph RDM, and the specific algorithm is as follows:
for the acquisition of the final decision diagram FDM, only pixels which are consistent in judgment on the focusing attributes of the pixels in the OMP decision diagram and the RDM decision diagram are taken as determined pixels, and besides, the uncertain pixels are set to be 0.5 in FDM;
using the final decision map FDM and the pre-fused image PF, a fully focused and visually well-behaved and natural fused image F can be generated, whose expression is specifically as follows:
in FDM, the pixels judged to be in a focusing or focusing area are directly copied from the corresponding source image to a fusion result F, and for uncertain pixels, the method obtains the pixels from a pre-fusion image PF, so that smooth conversion from one source image to another image is realized.
Referring to fig. 2, the multi-focus image fusion system based on structural similarity and region segmentation includes:
the pre-fusion module is used for carrying out weighting processing on the multi-set source image based on a fusion strategy of visual saliency detection to obtain a pre-fusion image;
the enhancement module is used for enhancing the pre-fusion image through a multi-scale detail enhancement strategy of Gaussian filtering and logarithmic energy difference to obtain a pre-fusion image after detail enhancement;
the segmentation module is used for carrying out segmentation processing on the pre-fused image after the details are enhanced based on the structural similarity index to obtain a region segmentation decision diagram;
and the fusion module is used for performing fusion processing on the region segmentation decision graph according to the pixel selection rule to obtain a final fusion image.
The simulation experiment of the invention is as follows:
in order to further show the advantages and effectiveness of the present invention, the present invention and 5 most advanced existing image fusion algorithms are subjected to a set of comparative tests, and the performance of each algorithm is analyzed on subjective visual evaluation, as shown in fig. 3 (a) and 3 (b), two multi-focus source images are shown, and fig. 4 (a) - (f) are respectively: the method comprises the steps of carrying out multi-focus image boundary finding algorithm (BF) based on multi-scale morphological focus measurement, multi-focus image Fusion algorithm (GRW) based on multi-scale focus measurement and generalized random walk, multi-set image Fusion algorithm (MFF-GAN) based on self-adaption and gradient joint constraint and unsupervised generation of confrontation network, unified unsupervised image Fusion network (U2 Fusion), multifunctional compressive decomposition network (SDNet) for real-time image Fusion and difference map under the Fusion algorithm (SSRS) of the scheme, observing the difference map of the figure 4, obviously showing that a large amount of residual information appears in background areas from figures 4 (c) to (e), proving that the method can not directly obtain pixel information from a source image and can reduce the definition of a Fusion result, observing figure 4 (a), seeing that the Fusion result of the method loses a large amount of focused pixel information and can not effectively judge the focusing attributes of different pixels, and figure 4 (b) shows that residual artifacts appear at the boundary, thus the condition of the boundary appears in the Fusion result, while figure 4 (f) has no influence on the prior art, and therefore the method is superior to the prior art.
The contents in the above method embodiments are all applicable to the present system embodiment, the functions specifically implemented by the present system embodiment are the same as those in the above method embodiment, and the beneficial effects achieved by the present system embodiment are also the same as those achieved by the above method embodiment.
While the invention has been described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (10)
1. The multi-focus image fusion method based on structural similarity and region segmentation is characterized by comprising the following steps of:
weighting a multi-set source image based on a fusion strategy of visual saliency detection to obtain a pre-fusion image;
enhancing the pre-fusion image through a multi-scale detail enhancement strategy of Gaussian filtering and logarithmic energy difference to obtain a pre-fusion image with enhanced details;
segmenting the pre-fused image after the details are enhanced based on the structural similarity index to obtain a region segmentation decision diagram;
and performing fusion processing on the region segmentation decision graph according to a pixel selection rule to obtain a final fusion image.
2. The multi-focus image fusion method based on structural similarity and region segmentation according to claim 1, wherein the fusion strategy based on visual saliency detection performs weighting processing on the multi-focus source images to obtain a pre-fusion image, and specifically includes:
acquiring a first poly set source image and a second poly set source image;
carrying out visual saliency detection on the first multi-set source image and the second multi-set source image to obtain corresponding saliency values;
carrying out normalization processing on the significance value to obtain a normalized significance value;
and based on the normalized significance value, carrying out weighted average processing on the first multi-set source image and the second multi-set source image to obtain a pre-fusion image.
3. The multi-focus image fusion method based on structural similarity and region segmentation according to claim 2, wherein the step of performing enhancement processing on the pre-fusion image through a multi-scale detail enhancement strategy of gaussian filtering and logarithmic energy difference to obtain a pre-fusion image after detail enhancement specifically comprises:
performing convolution operation on the pre-fusion image through a Gaussian function to obtain detail information of the pre-fusion image;
acquiring a high-frequency component to be fused of the pre-fused image based on the detail information of the pre-fused image;
obtaining the energy of the high-frequency component to be fused by a log-energy high-frequency component fusion algorithm, and carrying out logarithm processing to obtain a log-energy value of the high-frequency component to be fused;
calculating the difference value between log-energy of different high-frequency components to be fused, and introducing a preset threshold value for judgment;
selecting a corresponding high-frequency component fusion rule according to the judgment result to perform fusion processing on the high-frequency component to be fused to obtain a fused high-frequency component;
and adding the fused high-frequency component to the pre-fused image to obtain the pre-fused image with enhanced details.
4. The multi-focus image fusion method based on structural similarity and region segmentation according to claim 3, wherein the expression of the high-frequency component fusion rule is as follows:
in the above formula, FH (x, y) represents the fused high frequency component, HM (x, y) represents the decision diagram of the high frequency component, λ represents the preset threshold, DV represents the difference value between log-energy of different high frequency components to be fused,
representing different high frequency components to be fused,
representing the weighting coefficients.
5. The multi-focus image fusion method based on structural similarity and region segmentation according to claim 4, wherein the step of segmenting the pre-fusion image after detail enhancement based on the structural similarity index to obtain the region segmentation decision map specifically comprises:
acquiring a multi-set source image, and calculating the structural similarity of the multi-set source image and the pre-fusion image after the detail enhancement based on the focusing area of the image to obtain a similarity score map;
storing the similarity score map into a matrix to generate a score map;
based on the multi-channel signal of the image, carrying out iterative filtering processing on the score map through a recursive filter to obtain a score map;
carrying out decision processing on the fractional graph through a preset decision rule to obtain an initial two-region segmentation decision graph;
carrying out consistency detection and judgment on central pixels and peripheral pixels of the initial two-region segmentation decision map to obtain a two-region segmentation decision map, wherein the two-region segmentation decision map comprises a full focus region and a full defocus region;
analyzing the fractional graph based on a pixel difference method of RF to obtain a three-region segmentation decision graph, wherein the three-region segmentation decision graph comprises a full focus region, a full defocus region and an uncertain region;
and integrating the two-region segmentation decision diagram and the three-region segmentation decision diagram to obtain a region segmentation decision diagram.
6. The multi-focus image fusion method based on structural similarity and region segmentation according to claim 5, wherein the expression of the preset decision rule is as follows:
in the above formula, IMP (x, y) represents the initial two-region segmentation decision diagram, B 1 (x,y),B 2 (x,y),...,B T (x, y) represents the corresponding score map.
7. The multi-focus image fusion method based on structural similarity and region segmentation according to claim 6, wherein the expression of the pixel selection rule in the RF-based pixel difference method is as follows:
in the above equation, RDM (x, y) represents a three-region segmentation decision map, and β represents a parameter for controlling accuracy.
8. The multi-focus image fusion method based on structural similarity and region segmentation according to claim 7, wherein the step of performing fusion processing on the region segmentation decision map according to a pixel selection rule to obtain a final fusion image specifically comprises:
comprehensively processing the two-region segmentation decision graph and the three-region segmentation decision graph through a preset pixel selection rule to obtain a final decision graph;
and (4) performing fusion processing on the final decision diagram and the pre-fusion image by considering the effects of generating complete focusing and vision to obtain a final fusion image.
9. The multi-focus image fusion method based on structural similarity and region segmentation according to claim 8, wherein the expression of the preset pixel selection rule is specifically as follows:
in the above equation, FDM (x, y) represents a final decision diagram, OMP (x, y) represents a two-region segmentation decision diagram, and RDM (x, y) represents a three-region segmentation decision diagram.
10. The multi-focus image fusion system based on structural similarity and region segmentation is characterized by comprising the following modules:
the pre-fusion module is used for carrying out weighting processing on the multi-set source image based on a fusion strategy of visual saliency detection to obtain a pre-fusion image;
the enhancement module is used for enhancing the pre-fusion image through a multi-scale detail enhancement strategy of Gaussian filtering and logarithmic energy difference to obtain a pre-fusion image after detail enhancement;
the segmentation module is used for segmenting the pre-fusion image after the detail enhancement based on the structural similarity index to obtain a region segmentation decision diagram;
and the fusion module is used for performing fusion processing on the region segmentation decision graph according to the pixel selection rule to obtain a final fusion image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211579054.9A CN115829895A (en) | 2022-12-08 | 2022-12-08 | Multi-focus image fusion method and system based on structural similarity and region segmentation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211579054.9A CN115829895A (en) | 2022-12-08 | 2022-12-08 | Multi-focus image fusion method and system based on structural similarity and region segmentation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115829895A true CN115829895A (en) | 2023-03-21 |
Family
ID=85545580
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211579054.9A Pending CN115829895A (en) | 2022-12-08 | 2022-12-08 | Multi-focus image fusion method and system based on structural similarity and region segmentation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115829895A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118214958A (en) * | 2024-05-21 | 2024-06-18 | 环球数科集团有限公司 | Video synthesis system based on super-division fusion |
-
2022
- 2022-12-08 CN CN202211579054.9A patent/CN115829895A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118214958A (en) * | 2024-05-21 | 2024-06-18 | 环球数科集团有限公司 | Video synthesis system based on super-division fusion |
| CN118214958B (en) * | 2024-05-21 | 2024-07-16 | 环球数科集团有限公司 | Video synthesis system based on super-division fusion |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bhat et al. | Multi-focus image fusion techniques: a survey | |
| CN109376611B (en) | Video significance detection method based on 3D convolutional neural network | |
| Singh et al. | A review of image fusion: Methods, applications and performance metrics | |
| Kou et al. | A multi-focus image fusion method via region mosaicking on Laplacian pyramids | |
| Chakrabarti et al. | Depth and deblurring from a spectrally-varying depth-of-field | |
| KR102388831B1 (en) | Apparatus and Method for Fusing Intelligent Multi Focus Image | |
| CN110334779A (en) | A kind of multi-focus image fusing method based on PSPNet detail extraction | |
| CN113129236B (en) | Single low-light image enhancement method and system based on Retinex and convolutional neural network | |
| CN106447640B (en) | Multi-focus image fusing method and device based on dictionary learning, rotation guiding filtering | |
| Anwar et al. | Deblur and deep depth from single defocus image | |
| CN111179208B (en) | Infrared-visible light image fusion method based on saliency map and convolutional neural network | |
| Chang | Single underwater image restoration based on adaptive transmission fusion | |
| Gao et al. | Improving the performance of infrared and visible image fusion based on latent low-rank representation nested with rolling guided image filtering | |
| Saleem et al. | A non-reference evaluation of underwater image enhancement methods using a new underwater image dataset | |
| CN110555820A (en) | Image fusion method based on convolutional neural network and dynamic guide filtering | |
| Wang et al. | New insights into multi-focus image fusion: A fusion method based on multi-dictionary linear sparse representation and region fusion model | |
| Xiao et al. | DMDN: Degradation model-based deep network for multi-focus image fusion | |
| CN113763300A (en) | Multi-focus image fusion method combining depth context and convolution condition random field | |
| Choudhary et al. | Mathematical modeling and simulation of multi-focus image fusion techniques using the effect of image enhancement criteria: a systematic review and performance evaluation | |
| CN115829895A (en) | Multi-focus image fusion method and system based on structural similarity and region segmentation | |
| Singh et al. | Construction of fused image with improved depth-of-field based on guided co-occurrence filtering | |
| Debarnot et al. | Deep-blur: Blind identification and deblurring with convolutional neural networks | |
| CN111445465B (en) | Method and equipment for detecting and removing snow or rain belt of light field image based on deep learning | |
| CN109376782A (en) | Support vector machines cataract stage division and device based on eye image feature | |
| CN112508828A (en) | Multi-focus image fusion method based on sparse representation and guided filtering |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |