Nothing Special   »   [go: up one dir, main page]

CN110751608B - Night scene high dynamic range image fusion method and device and electronic equipment - Google Patents

Night scene high dynamic range image fusion method and device and electronic equipment Download PDF

Info

Publication number
CN110751608B
CN110751608B CN201911009224.8A CN201911009224A CN110751608B CN 110751608 B CN110751608 B CN 110751608B CN 201911009224 A CN201911009224 A CN 201911009224A CN 110751608 B CN110751608 B CN 110751608B
Authority
CN
China
Prior art keywords
image
fused
brightness
adjusted
fusion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911009224.8A
Other languages
Chinese (zh)
Other versions
CN110751608A (en
Inventor
王涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Force Map New Chongqing Technology Co ltd
Original Assignee
Beijing Megvii Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Megvii Technology Co Ltd filed Critical Beijing Megvii Technology Co Ltd
Priority to CN201911009224.8A priority Critical patent/CN110751608B/en
Publication of CN110751608A publication Critical patent/CN110751608A/en
Application granted granted Critical
Publication of CN110751608B publication Critical patent/CN110751608B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/70Denoising; Smoothing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/77Retouching; Inpainting; Scratch removal
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/90Dynamic range modification of images or parts thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20212Image combination
    • G06T2207/20221Image fusion; Image merging

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Studio Devices (AREA)

Abstract

本发明提供了一种夜景高动态范围图像融合方法、装置和电子设备,所述方法包括:获取多个待融合图像,所述待融合图像至少包括过曝图像、欠曝图像和正常曝光图像中的任意两种;根据所述待融合图像确定参考图像;根据所述参考图像对所述待融合图像的亮度进行调整,得到调整后待融合图像;将所述调整后待融合图像进行融合,得到HDR融合图像。这样,既不必确定相机参数信息或者预先场景先验信息即可对待融合图像进行HDR融合,时间复杂度低,也可以减少不同曝光图像之间的噪声差异,极大地减少甚至避免融合得到的图像出现突兀的亮区域以及不自然的噪声过度。从而可以在低时间复杂度的情况下,达到更好的融合效果。

Figure 201911009224

The present invention provides a night scene high dynamic range image fusion method, device and electronic device. The method includes: acquiring a plurality of images to be fused, and the images to be fused at least include an over-exposed image, an under-exposed image and a normal exposure image. Determine a reference image according to the image to be fused; adjust the brightness of the image to be fused according to the reference image to obtain the adjusted image to be fused; fuse the adjusted image to be fused to obtain HDR fused image. In this way, it is not necessary to determine the camera parameter information or pre-scenario prior information to perform HDR fusion on the images to be fused, and the time complexity is low, and the noise difference between different exposure images can also be reduced, greatly reducing or even avoiding the appearance of fused images. Obtrusive bright areas and unnatural excessive noise. Thus, a better fusion effect can be achieved with low time complexity.

Figure 201911009224

Description

一种夜景高动态范围图像融合方法、装置和电子设备A night scene high dynamic range image fusion method, device and electronic device

技术领域technical field

本发明涉及图像融合技术领域,具体而言,涉及一种夜景高动态范围图像融合方法、装置和电子设备。The present invention relates to the technical field of image fusion, and in particular, to a night scene high dynamic range image fusion method, device and electronic device.

背景技术Background technique

高动态范围图像(High-Dynamic Range,简称HDR),根据不同的曝光时间的图像,并利用每个曝光时间相对应最佳细节的图像来合成的,相比普通的图像,可以提供更多的动态范围和图像细节,能够更好的反映出真实环境中的视觉效果。High-Dynamic Range (HDR) images are synthesized based on images with different exposure times and images with the best details corresponding to each exposure time. Compared with ordinary images, it can provide more Dynamic range and image detail to better reflect the visual effect in the real environment.

目前hdr图像生成的算法有很多,但是整体上大致一般可以分成2类,一种是传统的HDR算法,但是传统的HDR算法,需要相机参数信息或者预先场景先验信息,要求条件相对苛刻并且准确的先验信息计算较难,而且时间复杂度较高。另一种是多张不同曝光值图像通过某种规则直接融合方式,比如论文<不同曝光值图像的直接融合方法>等方式,这类方式无需相机参数或者场景先验信息,算法速度较快。但是这种方案很难在夜景hdr时候使用,由于夜景环境下,相同场景亮暗差异更大,并且不同曝光图像噪声差异严重,很容易造成结果图像出现突兀的亮区域和不自然的噪声过渡。At present, there are many algorithms for HDR image generation, but they can be roughly divided into two categories. One is the traditional HDR algorithm, but the traditional HDR algorithm requires camera parameter information or pre-scenario prior information, and the requirements are relatively harsh and accurate. It is difficult to calculate the prior information of , and the time complexity is high. The other is to directly fuse multiple images with different exposure values through certain rules, such as the paper <Direct Fusion Method of Images with Different Exposure Values>. This method does not require camera parameters or scene prior information, and the algorithm is faster. However, this scheme is difficult to use in night scene hdr, because in the night scene environment, the difference between the light and dark of the same scene is greater, and the noise difference between different exposure images is serious, it is easy to cause abrupt bright areas and unnatural noise transitions in the resulting image.

因此,迫切需要一种无需先验信息且夜景高动态范围图像融合效果好的方法及装置。Therefore, there is an urgent need for a method and device that does not require prior information and has a good effect of night scene high dynamic range image fusion.

发明内容SUMMARY OF THE INVENTION

本发明解决的问题是如何从样本数据中快速确定困难样本。The problem solved by the present invention is how to quickly determine difficult samples from sample data.

为解决上述问题,本发明首先提供一种夜景高动态范围图像融合方法,其包括:In order to solve the above problems, the present invention first provides a night scene high dynamic range image fusion method, which includes:

获取多个待融合图像,所述待融合图像至少包括过曝图像、欠曝图像和正常曝光图像中的任意两种;acquiring a plurality of images to be fused, the images to be fused at least include any two of overexposed images, underexposed images and normal exposure images;

根据所述欠曝图像或正常曝光图像进行亮度调整后确定参考图像;Determine a reference image after performing brightness adjustment according to the underexposed image or the normally exposed image;

根据所述参考图像对所述待融合图像的亮度进行调整,得到调整后待融合图像;adjusting the brightness of the image to be fused according to the reference image to obtain the adjusted image to be fused;

将所述调整后待融合图像进行融合,得到HDR融合图像。The adjusted images to be fused are fused to obtain an HDR fusion image.

这样,既不必确定相机参数信息或者预先场景先验信息即可对待融合图像进行HDR融合,时间复杂度低,也可以减少不同曝光图像之间的噪声差异,极大地减少甚至避免融合得到的图像出现突兀的亮区域以及不自然的噪声过度。从而可以在低时间复杂度的情况下,达到更好的融合效果,减少了融合图像中出现突兀的亮区域及不自然噪声的可能。In this way, it is not necessary to determine camera parameter information or pre-scenario prior information to perform HDR fusion on the images to be fused, with low time complexity, and can also reduce noise differences between different exposure images, greatly reducing or even avoiding the appearance of fused images. Obtrusive bright areas and unnatural excessive noise. Therefore, a better fusion effect can be achieved with low time complexity, and the possibility of abrupt bright areas and unnatural noises appearing in the fusion image is reduced.

可选的,所述根据所述参考图像对所述待融合图像的亮度进行调整中,所述待融合图像包括用于确定所述参考图像的所述欠曝图像或者所述正常曝光图像。Optionally, in the adjustment of the brightness of the image to be fused according to the reference image, the image to be fused includes the underexposed image or the normal exposure image used to determine the reference image.

可选的,所述根据所述参考图像对所述待融合图像的亮度进行调整中,所述待融合图像为除用于确定所述参考图像的所述欠曝图像或者所述正常曝光图像之外的所有待融合图像;且Optionally, in the adjustment of the brightness of the image to be fused according to the reference image, the image to be fused is the difference between the underexposed image or the normal exposure image used to determine the reference image. All images to be fused except for; and

所述将所述调整后待融合图像进行融合,包括:将所述参考图像和所述调整后待融合图像进行融合。The fusing of the adjusted image to be fused includes: fusing the reference image and the adjusted image to be fused.

可选的,所述根据所述参考图像对所述待融合图像的亮度进行调整,包括:Optionally, the adjusting the brightness of the to-be-fused image according to the reference image includes:

针对每一个所述待融合图像,根据所述参考图像的平均亮度对所述待融合图像的平均亮度进行整体调整,整体调整后所述待融合图像的平均亮度与所述参考图像的平均亮度相同;For each image to be fused, the average brightness of the image to be fused is adjusted as a whole according to the average brightness of the reference image. After the overall adjustment, the average brightness of the image to be fused is the same as the average brightness of the reference image. ;

对整体调整后的所述待融合图像的亮度进行微调,得到调整后待融合图像;Fine-tune the brightness of the overall adjusted image to be fused to obtain an adjusted image to be fused;

遍历所有待融合图像,得到对应的所述调整后待融合图像。Traverse all the images to be fused to obtain the corresponding adjusted images to be fused.

这样,通过整体调整和微调,将待融合图像调整为与参考图像非常接近的调整后待融合图像,消除了(极大地减小了)待融合图像与参考图像的亮度差异,同时通过微调,减小了噪声干扰。In this way, through the overall adjustment and fine-tuning, the image to be fused is adjusted to be an adjusted image to be fused that is very close to the reference image, which eliminates (greatly reduces) the difference in brightness between the image to be fused and the reference image. Less noise interference.

可选的,所述根据所述参考图像的平均亮度对所述待融合图像的平均亮度进行整体调整,包括:Optionally, the overall adjustment of the average brightness of the image to be fused according to the average brightness of the reference image includes:

根据所述待融合图像和所述参考图像中各个像素点的亮度,确定所述待融合图像和所述参考图像的平均亮度;According to the brightness of each pixel in the to-be-fused image and the reference image, determine the average brightness of the to-be-fused image and the reference image;

根据所述待融合图像和所述参考图像的平均亮度确定所述待融合图像和所述参考图像的亮度差异比例;Determine the brightness difference ratio between the to-be-fused image and the reference image according to the average brightness of the to-be-fused image and the reference image;

根据所述亮度差异比例对所述待融合图像中各个像素点的亮度进行整体调整,整体调整后所述待融合图像的平均亮度和所述参考图像的平均亮度相同。The brightness of each pixel in the to-be-fused image is adjusted as a whole according to the brightness difference ratio, and the average brightness of the to-be-fused image after the overall adjustment is the same as the average brightness of the reference image.

这样,通过整体调整可以快速将所述待融合图像的亮度贴近所述参考图像,且等比例调整减少了整体调整造成失真的可能性。In this way, the brightness of the image to be fused can be quickly brought close to the reference image through the overall adjustment, and the proportional adjustment reduces the possibility of distortion caused by the overall adjustment.

可选的,所述根据所述亮度差异比例对所述待融合图像中各个像素点的亮度进行整体调整,包括:通过将所述待融合图像中各个像素点的亮度均与所述亮度差异比例相乘进行所述整体调整。Optionally, the overall adjustment of the brightness of each pixel point in the image to be fused according to the brightness difference ratio includes: by comparing the brightness of each pixel point in the to-be-fused image with the brightness difference ratio. Multiply to make the overall adjustment.

可选的,所述对整体调整后的所述待融合图像的亮度进行微调,得到调整后待融合图像,包括:Optionally, performing fine-tuning on the brightness of the overall adjusted image to be fused to obtain the adjusted image to be fused, including:

将所述参考图像和整体调整后的所述待融合图像缩小;reducing the reference image and the overall adjusted image to be fused;

对缩小后的所述参考图像和缩小后的所述待融合图像进行相同参数的引导滤波;Perform guided filtering with the same parameters on the reduced reference image and the reduced image to be fused;

根据滤波后的所述参考图像和滤波后的所述待融合图像中各个像素点的亮度,确定各个像素点的差异比例,并存入调整掩码中;According to the filtered reference image and the filtered brightness of each pixel in the to-be-fused image, determine the difference ratio of each pixel, and store it in the adjustment mask;

将所述调整掩码放大到与整体调整前的所述待融合图像大小相同;Enlarging the adjustment mask to the same size as the image to be fused before the overall adjustment;

根据放大后的所述调整掩码的差异比例,对所述整体调整后的所述待融合图像中对应位置处的像素点的亮度进行调整,得到调整后待融合图像。According to the difference ratio of the enlarged adjustment mask, the brightness of the pixel at the corresponding position in the overall adjusted image to be fused is adjusted to obtain the adjusted image to be fused.

这样,将平均亮度调整(整体调整)后的待融合图像进行小图放大进行微调,可以保证待融合图像附近相似区域像素强度接近,避免噪声干扰。In this way, the image to be fused after the average brightness adjustment (overall adjustment) is zoomed in and fine-tuned, which can ensure that the pixel intensities of similar areas near the image to be fused are close to avoid noise interference.

可选的,所述对所述整体调整后的所述待融合图像中对应位置处的像素点的亮度进行调整,包括:通过将所述整体调整后的所述待融合图像中对应位置处的像素点的亮度与对应的所述差异比例相乘进行调整。Optionally, the adjusting the brightness of the pixels at the corresponding positions in the image to be fused after the overall adjustment includes: adjusting the brightness of the pixels at the corresponding positions in the image to be fused after the overall adjustment. The brightness of the pixel is adjusted by multiplying the corresponding difference ratio.

可选的,所述将所述参考图像和所述调整后待融合图像进行融合,得到HDR融合图像,包括:Optionally, the fusion of the reference image and the adjusted image to be fused to obtain an HDR fusion image, including:

根据所述调整后待融合图像和所述参考图像,确定所述调整后待融合图像的融合权重和所述参考图像的融合权重;According to the adjusted image to be fused and the reference image, determine the fusion weight of the adjusted image to be fused and the fusion weight of the reference image;

根据所述调整后待融合图像的融合权重和所述参考图像的融合权重,将所述调整后待融合图像和所述参考图像进行融合,得到HDR融合图像。According to the fusion weight of the adjusted image to be fused and the fusion weight of the reference image, the adjusted image to be fused and the reference image are fused to obtain an HDR fusion image.

这样,通过分配融合权重后再进行图像融合,可以使得最终得到的HDR融合图像的整体亮度噪声都很自然。In this way, by assigning fusion weights and then performing image fusion, the overall luminance noise of the final HDR fusion image can be made natural.

可选的,所述根据所述欠曝图像或正常曝光图像进行亮度调整后确定参考图像,包括:Optionally, the determining a reference image after performing brightness adjustment according to the underexposed image or the normal exposure image includes:

从所述待融合图像中选取一个欠曝图像或正常曝光图像作为待参考图像;Select an underexposed image or a normal exposure image from the to-be-fused images as the to-be-referenced image;

根据所述待参考图像确定所述待参考图像中各个像素点的过曝掩码;Determine the overexposure mask of each pixel in the to-be-referenced image according to the to-be-referenced image;

获取预设的调整参数,根据所述调整参数和所述过曝掩码确定所述待参考图像中各个像素点的亮度调整参数;Acquire preset adjustment parameters, and determine brightness adjustment parameters of each pixel in the to-be-referenced image according to the adjustment parameters and the overexposure mask;

根据所述亮度调整参数调整所述待参考图像中各个像素点的亮度,得到所述参考图像。Adjust the brightness of each pixel in the to-be-referenced image according to the brightness adjustment parameter to obtain the reference image.

这样,通过预设调整参数,对像素点的亮度进行调整,从而可以将调整后的亮度作为最终生成的HDR融合图像的整体亮度,从而与客户的喜好风格对应,增加客户的体验感和使用的舒适性。并且通过检测过曝区域,防止在亮度调整时候,导致亮区过曝,并且由于过曝掩码过渡平滑自然,所有也不会出现亮度异常跳变的过渡带。In this way, the brightness of the pixels is adjusted by preset adjustment parameters, so that the adjusted brightness can be used as the overall brightness of the final generated HDR fusion image, so as to correspond to the customer's preferred style and increase the customer's sense of experience and use. comfort. And by detecting the overexposed area, it prevents the bright area from being overexposed when the brightness is adjusted, and because the transition of the overexposure mask is smooth and natural, there will be no transition band with abnormal brightness jumps.

可选的,所述过曝掩码的获取方式为:Optionally, the acquisition method of the overexposure mask is:

E=1-min(n,1)E=1-min(n,1)

Figure BDA0002243687920000051
Figure BDA0002243687920000051

式中,E为像素点的过曝掩码,p为待参考图像中像素点的亮度,pt为亮度阈值,s为平滑度,n为中间变量。In the formula, E is the overexposure mask of the pixel point, p is the brightness of the pixel point in the image to be referenced, pt is the brightness threshold, s is the smoothness, and n is the intermediate variable.

可选的,所述亮度调整参数的获取方式为:Optionally, the acquisition method of the brightness adjustment parameter is:

a=(A-1)×E+1a=(A-1)×E+1

式中,a为像素点的亮度调整参数,A为调整参数,E为像素点的过曝掩码。In the formula, a is the brightness adjustment parameter of the pixel, A is the adjustment parameter, and E is the overexposure mask of the pixel.

可选的,所述待融合图像为所述过曝图像时,对应的所述调整后待融合图像的融合权重的获取方式为:Optionally, when the image to be fused is the overexposed image, the corresponding method for obtaining the fusion weight of the image to be fused after adjustment is:

Figure BDA0002243687920000052
Figure BDA0002243687920000052

式中,w+为过曝图像对应的调整后待融合图像的融合权重,p为调整后待融合图像中像素点的亮度,pt为亮度阈值,s为平滑度。In the formula, w + is the fusion weight of the adjusted image to be fused corresponding to the overexposed image, p is the brightness of the pixels in the adjusted image to be fused, p t is the brightness threshold, and s is the smoothness.

可选的,所述待融合图像为所述欠曝图像时,对应的所述调整后待融合图像的融合权重的获取方式为:Optionally, when the image to be fused is the underexposed image, the corresponding acquisition method of the fusion weight of the image to be fused after adjustment is:

Figure BDA0002243687920000053
Figure BDA0002243687920000053

式中,w-为欠曝图像对应的调整后待融合图像的融合权重,p为调整后待融合图像中像素点的亮度,pt为亮度阈值,s为平滑度。In the formula, w - is the fusion weight of the adjusted image to be fused corresponding to the underexposed image, p is the brightness of the pixels in the adjusted image to be fused, p t is the brightness threshold, and s is the smoothness.

再次提供一种夜景高动态范围图像融合装置,其包括:Once again, a night scene high dynamic range image fusion device is provided, which includes:

获取单元,用于获取多个待融合图像,所述待融合图像至少包括过曝图像、欠曝图像和正常曝光图像中的任意两种;an acquisition unit, configured to acquire a plurality of images to be fused, the images to be fused at least include any two of an overexposed image, an underexposed image and a normal exposure image;

确定单元,用于根据所述欠曝图像或正常曝光图像进行亮度调整后确定参考图像;a determining unit, configured to determine a reference image after performing brightness adjustment according to the underexposed image or the normally exposed image;

调整单元,用于根据所述参考图像对所述待融合图像的亮度进行调整,得到调整后待融合图像;an adjustment unit, configured to adjust the brightness of the image to be fused according to the reference image to obtain the adjusted image to be fused;

融合单元,用于将所述调整后待融合图像进行融合,得到HDR融合图像。A fusion unit, configured to fuse the adjusted images to be fused to obtain an HDR fusion image.

这样,既不必确定相机参数信息或者预先场景先验信息即可对待融合图像进行HDR融合,时间复杂度低,也可以减少不同曝光图像之间的噪声差异,极大地减少甚至避免融合得到的图像出现突兀的亮区域以及不自然的噪声过度。从而可以在低时间复杂度的情况下,达到更好的融合效果,减少了融合图像中出现突兀的亮区域及不自然噪声的可能。In this way, it is not necessary to determine camera parameter information or pre-scenario prior information to perform HDR fusion on the images to be fused, with low time complexity, and can also reduce noise differences between different exposure images, greatly reducing or even avoiding the appearance of fused images. Obtrusive bright areas and unnatural excessive noise. Therefore, a better fusion effect can be achieved with low time complexity, and the possibility of abrupt bright areas and unnatural noises appearing in the fusion image is reduced.

从次再提供一种电子设备,包括处理器以及存储器,所述存储器存储有控制程序,所述控制程序被处理器执行时实现上述所述的夜景高动态范围图像融合方法。Next, an electronic device is provided, including a processor and a memory, wherein the memory stores a control program, and when the control program is executed by the processor, the above-mentioned night scene high dynamic range image fusion method is implemented.

最后提供一种计算机可读存储介质,存储有指令,其特征在于,所述指令被处理器加载并执行时实现上述所述的夜景高动态范围图像融合方法。Finally, a computer-readable storage medium is provided, which stores instructions, wherein, when the instructions are loaded and executed by a processor, the above-mentioned night scene high dynamic range image fusion method is implemented.

附图说明Description of drawings

图1将多张不同曝光值的夜景图像通过直接融合得到的融合图像;Figure 1 is a fusion image obtained by directly fusing multiple night scene images with different exposure values;

图2为根据本发明实施例的夜景高动态范围图像融合方法的流程图;2 is a flowchart of a night scene high dynamic range image fusion method according to an embodiment of the present invention;

图3为根据本发明实施例的夜景高动态范围图像融合方法步骤30的流程图;3 is a flowchart of step 30 of a night scene high dynamic range image fusion method according to an embodiment of the present invention;

图4为根据本发明实施例的夜景高动态范围图像融合方法步骤31的流程图;4 is a flowchart of step 31 of a night scene high dynamic range image fusion method according to an embodiment of the present invention;

图5为根据本发明实施例的夜景高动态范围图像融合方法步骤32的流程图;5 is a flowchart of step 32 of a night scene high dynamic range image fusion method according to an embodiment of the present invention;

图6A为根据本发明实施例直接小图放大得到的调整后待融合图像;6A is an adjusted image to be fused obtained by directly zooming in a small image according to an embodiment of the present invention;

图6B为根据本发明实施例先平均亮度调整再小图放大得到的调整后待融合图像;6B is an image to be fused after adjustment obtained by first adjusting the average brightness and then enlarging the small image according to an embodiment of the present invention;

图7为根据本发明实施例的夜景高动态范围图像融合方法步骤40的流程图;7 is a flowchart of step 40 of a night scene high dynamic range image fusion method according to an embodiment of the present invention;

图8为根据本发明实施例的夜景高动态范围图像融合方法步骤20的流程图;8 is a flowchart of step 20 of a night scene high dynamic range image fusion method according to an embodiment of the present invention;

图9为根据本发明实施例的夜景高动态范围图像融合装置的结构框图;9 is a structural block diagram of a night scene high dynamic range image fusion device according to an embodiment of the present invention;

图10为根据本发明实施例的一种电子设备的结构框图;10 is a structural block diagram of an electronic device according to an embodiment of the present invention;

图11为根据本发明实施例的另一种电子设备的框图。FIG. 11 is a block diagram of another electronic device according to an embodiment of the present invention.

附图标记说明:Description of reference numbers:

1-获取单元,2-确定单元,3-调整单元,4-融合单元,12-电子设备,14-外部设备,16-处理单元,18-总线,20-网络适配器,22-输入/输出(I/O)接口,24-显示器,28-系统存储器,30-随机存取存储器,32-高速缓存存储器,34-存储系统,40-实用工具,42-程序模块。1-acquisition unit, 2-determination unit, 3-adjustment unit, 4-fusion unit, 12-electronic device, 14-external device, 16-processing unit, 18-bus, 20-network adapter, 22-input/output ( I/O) interface, 24-display, 28-system memory, 30-random access memory, 32-cache memory, 34-storage system, 40-utility, 42-program module.

具体实施方式Detailed ways

为使本发明的上述目的、特征和优点能够更为明显易懂,下面结合附图对本发明的具体实施例做详细的说明。In order to make the above objects, features and advantages of the present invention more clearly understood, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

显然,所说明的实施例是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域技术人员在没有做出创造性劳动的前提下获得的所有其他实施例,都属于本发明保护的范围。Obviously, the illustrated embodiments are some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present invention.

为了便于理解,在本发明中,需要对其中的技术问题进行详细阐述。In order to facilitate understanding, in the present invention, the technical problems therein need to be explained in detail.

高动态范围图像(High-Dynamic Range,简称HDR),相比普通的图像,可以提供更多的动态范围和图像细节,根据不同的曝光时间的LDR(Low-Dynamic Range,低动态范围图像),并利用每个曝光时间相对应最佳细节的LDR图像来合成最终HDR图像。它能够更好的反映出真实环境中的视觉效果。High-dynamic range images (High-Dynamic Range, referred to as HDR), compared with ordinary images, can provide more dynamic range and image details, according to different exposure time LDR (Low-Dynamic Range, low dynamic range image), And use the LDR images with the best detail corresponding to each exposure time to synthesize the final HDR image. It can better reflect the visual effect in the real environment.

根据曝光时间的不同,LDR图像的名称也不同。若曝光时间不足,则得到的LDR图像为欠曝图像;若曝光时间处于正常范围,则得到的LDR图像为正常曝光图像;若曝光时间过长,则得到的LDR图像为过曝图像。Depending on the exposure time, LDR images have different names. If the exposure time is insufficient, the obtained LDR image is an underexposed image; if the exposure time is in the normal range, the obtained LDR image is a normal exposure image; if the exposure time is too long, the obtained LDR image is an overexposed image.

现有的HDR算法,一种是先确定相机参数信息或者预先场景先验信息,然后根据相机参数信息或者预先场景先验信息对图像进行融合,这种HDR算法需要先确定相机参数信息或者预先场景先验信息,计算较难,而且时间复杂度较高;一种是将多张不同曝光值图像通过某种规则直接融合,但是这种方法只能适用于图像噪声差异小的情况下,一旦处于夜景环境下,相同场景的亮暗差异很大,不同曝光图像噪声差异严重,就会使得融合得到的图像出现突兀的亮区域以及不自然的噪声过度。In the existing HDR algorithm, one is to first determine the camera parameter information or pre-scenario prior information, and then fuse the image according to the camera parameter information or pre-scenario prior information. This HDR algorithm needs to determine the camera parameter information or pre-scenario first. Prior information, the calculation is difficult, and the time complexity is high; one is to directly fuse multiple images with different exposure values through a certain rule, but this method can only be applied to the case where the image noise difference is small. In the night scene environment, the brightness and darkness of the same scene are very different, and the noise of different exposure images is very different, which will make the fused image appear abrupt bright areas and unnatural excessive noise.

如图1所示,其为将多张不同曝光值的夜景图像通过直接融合得到的融合图像(即通过第二种方法对夜景图像融合得到的融合图像);从该图我们可以明显看出,其上面出现了很多突兀的亮区域,且噪声过度很不自然。As shown in Figure 1, it is a fusion image obtained by directly fusing multiple night scene images with different exposure values (that is, a fusion image obtained by fusing night scene images through the second method); from this figure, we can clearly see that, There are many abrupt bright areas on it, and the excessive noise is unnatural.

本公开实施例提供了一种夜景高动态范围图像融合方法,该方法可以由夜景高动态范围图像融合装置来执行,该夜景高动态范围图像融合装置可以集成在手机、笔记本、服务器、摄像机、照相机、PAD等电子设备中。如图2所示,其为根据本发明实施例的夜景高动态范围图像融合方法的流程图一;其中,所述夜景高动态范围图像融合方法,包括:The embodiment of the present disclosure provides a night scene high dynamic range image fusion method, the method can be performed by a night scene high dynamic range image fusion device, and the night scene high dynamic range image fusion device can be integrated in a mobile phone, a notebook, a server, a video camera, a camera , PAD and other electronic equipment. As shown in FIG. 2 , it is a flow chart 1 of a night scene high dynamic range image fusion method according to an embodiment of the present invention; wherein, the night scene high dynamic range image fusion method includes:

步骤10,获取多个待融合图像,所述待融合图像至少包括过曝图像、欠曝图像和正常曝光图像中的任意两种;Step 10, acquiring a plurality of images to be fused, the images to be fused at least include any two of overexposed images, underexposed images and normal exposure images;

其中,所述待融合图像的获取方式可以为直接通过电子设备拍摄,也可以是通过其他设备输入该电子设备中,也可以为其他可行的获取方式。Wherein, the acquisition method of the image to be fused may be directly photographed by an electronic device, or input into the electronic device by other equipment, or may be other feasible acquisition methods.

需要说明的是,所述待融合图像均是对同一对象的不同曝光时长的图像,也即是说,所述待融合图像中各个像素点是一一对应的,仅仅是亮度(像素点的像素值或者像素强度值)不同而已。It should be noted that the images to be fused are images of the same object with different exposure durations, that is to say, each pixel in the image to be fused is in a one-to-one correspondence, only brightness (pixels of a pixel) value or pixel intensity value) are different.

可选的,所述待融合图像的数量为至少2个。Optionally, the number of the to-be-fused images is at least two.

其中,所述待融合图像至少包括过曝图像、欠曝图像和正常曝光图像中的任意两种。Wherein, the to-be-fused image includes at least any two of an overexposed image, an underexposed image and a normal exposure image.

其中,对待融合图像是过曝图像还是欠曝图像还是正常曝光图像的判断,可以根据拍摄待融合图像的电子设备内的预设判断条件来判断,也可以是根据待融合图像的曝光时长或者根据实际情况确定来确定。Among them, the judgment of whether the image to be fused is an overexposed image, an underexposed image or a normal exposure image can be determined according to a preset judgment condition in the electronic device that shoots the image to be fused, or it can be determined according to the exposure time of the image to be fused or according to The actual situation is determined to be determined.

对于过曝图像来说,其优点在于图像噪声低,缺点在于由于像素点的亮度(像素点的像素值或者像素强度值)最高为255.0,超出该亮度的部分也显示为255.0,也即是说,部分信息由于过曝的原因,统一变为了255.0的亮度,从而损失了对应的信息,也即是说,过曝图像的信息量较少。For overexposed images, the advantage is that the image noise is low, and the disadvantage is that since the brightness of the pixel (the pixel value of the pixel or the pixel intensity value) is up to 255.0, the part beyond the brightness is also displayed as 255.0, that is to say , some of the information is uniformly changed to 255.0 brightness due to overexposure, thus losing the corresponding information, that is to say, the amount of information of the overexposed image is less.

对于欠曝图像来说,恰好相反,其优点为过曝区域的信息量较多,缺点在于图像的噪声较高。For underexposed images, the opposite is true. The advantage is that the overexposed area contains more information, while the disadvantage is that the image has high noise.

因此,所述待融合图像不能仅为过曝图像或者仅为欠曝图像,这样会导致缺点无法通过图像融合的方式消除。Therefore, the to-be-fused image cannot be only an overexposed image or only an underexposed image, which will lead to the disadvantage that cannot be eliminated by means of image fusion.

步骤20,根据所述欠曝图像或正常曝光图像进行亮度调整后确定参考图像;Step 20, determining a reference image after performing brightness adjustment according to the underexposed image or the normally exposed image;

其中,所述参考图像是通过对所述欠曝图像或正常曝光图像中的其中一个进行亮度调整后得到的。这样,通过待融合图像生成参考图像,从而不必确定相机参数信息或者预先场景先验信息即可对待融合图像进行HDR融合,时间复杂度低。Wherein, the reference image is obtained by adjusting the brightness of one of the underexposure image or the normal exposure image. In this way, the reference image is generated from the image to be fused, so that the HDR fusion of the image to be fused can be performed without determining the camera parameter information or the prior scene information in advance, and the time complexity is low.

其中,选取正常曝光图像或欠曝图像;这是由于过曝图像的过曝区域中信息量少,若以过曝图像生成参考图像,则会导致参考图像中的过曝区域信息量过少,其他待融合图像根据参考图像进行调整时,对应过曝区域的部分容易失真,进而导致融合的HDR图像失真严重;不选择过曝图像,则可以避免过曝区域造成的融合HDR图像失真。Among them, the normal exposure image or the underexposure image is selected; this is because the amount of information in the overexposed area of the overexposed image is small. When other images to be fused are adjusted according to the reference image, the part corresponding to the overexposed area is easily distorted, which will lead to serious distortion of the fused HDR image; if the overexposed image is not selected, the distortion of the fused HDR image caused by the overexposed area can be avoided.

步骤30,根据所述参考图像对所述待融合图像的亮度进行调整,得到调整后待融合图像;Step 30: Adjust the brightness of the image to be fused according to the reference image to obtain the adjusted image to be fused;

这样,通过对待融合图像进行处理,使得调整后待融合图像与参考图像更接近,从而减少不同曝光图像之间的噪声差异,极大地减少甚至避免融合得到的图像出现突兀的亮区域以及不自然的噪声过度。In this way, by processing the image to be fused, the adjusted image to be fused is closer to the reference image, thereby reducing the noise difference between different exposure images, greatly reducing or even avoiding the appearance of abrupt bright areas and unnatural images in the fused image. Excessive noise.

步骤40,将所述调整后待融合图像进行融合,得到HDR融合图像。Step 40 , fuse the adjusted images to be fused to obtain an HDR fusion image.

这样,既不必确定相机参数信息或者预先场景先验信息即可对待融合图像进行HDR融合,时间复杂度低,也可以减少不同曝光图像之间的噪声差异,极大地减少甚至避免融合得到的图像出现突兀的亮区域以及不自然的噪声过度。从而可以在低时间复杂度的情况下,达到更好的融合效果,减少了融合图像中出现突兀的亮区域及不自然噪声的可能。In this way, it is not necessary to determine camera parameter information or pre-scenario prior information to perform HDR fusion on the images to be fused, with low time complexity, and can also reduce noise differences between different exposure images, greatly reducing or even avoiding the appearance of fused images. Obtrusive bright areas and unnatural excessive noise. Therefore, a better fusion effect can be achieved with low time complexity, and the possibility of abrupt bright areas and unnatural noises appearing in the fusion image is reduced.

可选的,所述根据所述参考图像对所述待融合图像的亮度进行调整中,所述待融合图像包括用于确定所述参考图像的所述欠曝图像或者所述正常曝光图像。Optionally, in the adjustment of the brightness of the image to be fused according to the reference image, the image to be fused includes the underexposed image or the normal exposure image used to determine the reference image.

在此需要说明的是,所述待融合图像在处理时,由于其中一个待融合图像通过亮度调整后确定了参考图像(也可以说是生成了参考图像),因此在后续步骤中对该待融合图像(确定了参考图像的待融合图像)的处理,可以有两种方式来处理,此为第一种处理方式。It should be noted here that, during the processing of the images to be fused, since one of the images to be fused determines a reference image (or a reference image is generated) after brightness adjustment, the image to be fused is processed in the subsequent steps. There are two ways to process the image (the image to be fused for which the reference image is determined), and this is the first processing way.

也即是说,在步骤30,根据所述参考图像对所述待融合图像的亮度进行调整,得到调整后待融合图像中,仍然对该待融合图像(确定了参考图像的待融合图像)进行亮度调整,得到调整后待融合图像;然后在步骤40,将所述调整后待融合图像进行融合,得到HDR融合图像中,直接将所有的调整后待融合图像(也即是包括对确定了参考图像的待融合图像进行亮度调整后得到的那个调整后待融合图像)进行融合,即可得到HDR融合图像。That is to say, in step 30, the brightness of the image to be fused is adjusted according to the reference image, and in the adjusted image to be fused, the image to be fused (the image to be fused for which the reference image is determined) is still processed. Adjust the brightness to obtain the adjusted image to be fused; then in step 40, fuse the adjusted image to be fused to obtain the HDR fusion image, directly merging all the adjusted images to be fused (that is, including the reference to the determined reference) The HDR fusion image can be obtained by merging the adjusted image to be fused obtained after brightness adjustment of the image to be fused.

这样,无需特别区分待融合图像中确定了参考图像的待融合图像和其余的待融合图像的区别,直接对所有待融合图像进行亮度调整和融合,简单方便。In this way, it is simple and convenient to directly perform brightness adjustment and fusion on all the to-be-fused images without special distinction between the to-be-fused image for which the reference image is determined and the rest of the to-be-fused images in the to-be-fused images.

可选的,所述根据所述参考图像对所述待融合图像的亮度进行调整中,所述待融合图像为除用于确定所述参考图像的所述欠曝图像或者所述正常曝光图像之外的所有待融合图像;且Optionally, in the adjustment of the brightness of the image to be fused according to the reference image, the image to be fused is the difference between the underexposed image or the normal exposure image used to determine the reference image. All images to be fused except for; and

所述将所述调整后待融合图像进行融合,包括:将所述参考图像和所述调整后待融合图像进行融合。The fusing of the adjusted image to be fused includes: fusing the reference image and the adjusted image to be fused.

此为前述处理方式中的第二种处理方式。This is the second processing method in the aforementioned processing methods.

也即是说,在步骤30,根据所述参考图像对所述待融合图像的亮度进行调整,得到调整后待融合图像中,不再对该待融合图像(确定了参考图像的待融合图像)进行亮度调整;然后在步骤40,将所述调整后待融合图像进行融合,得到HDR融合图像中,直接将参考图像和所有的调整后待融合图像(也即是不包括对确定了参考图像的待融合图像进行亮度调整后得到的那个调整后待融合图像,实际上是用参考图像代替了该调整后待融合图像)进行融合,即可得到HDR融合图像。That is to say, in step 30, the brightness of the image to be fused is adjusted according to the reference image, and in the adjusted image to be fused, the image to be fused (the image to be fused of the reference image is determined) is no longer there. Perform brightness adjustment; then in step 40, fuse the adjusted images to be fused to obtain the HDR fusion image, directly combine the reference image and all the adjusted images to be fused (that is, not including the reference image that has been determined). The adjusted image to be fused, which is obtained after the brightness of the image to be fused is adjusted, actually replaces the adjusted image to be fused with the reference image) for fusion, and the HDR fusion image can be obtained.

这样,无需再对确定了参考图像的待融合图像进行亮度调整,而是以参考图像代替该待融合图像调整后的图像,直接与其它的所有待融合图像进行亮度调整和融合,简单方便。可选的,如图3所示,所述步骤30,根据所述参考图像对所述待融合图像的亮度进行调整,包括:In this way, there is no need to adjust the brightness of the image to be fused for which the reference image is determined, but the adjusted image of the image to be fused is replaced by the reference image, and the brightness adjustment and fusion are directly performed with all other images to be fused, which is simple and convenient. Optionally, as shown in FIG. 3 , in step 30, adjusting the brightness of the image to be fused according to the reference image includes:

步骤31,针对每一个所述待融合图像,根据所述参考图像的平均亮度对所述待融合图像的平均亮度进行整体调整,整体调整后所述待融合图像的平均亮度与所述参考图像的平均亮度相同;Step 31: For each image to be fused, the average brightness of the image to be fused is adjusted as a whole according to the average brightness of the reference image, and the average brightness of the image to be fused after the overall adjustment is the same as the average brightness of the reference image. The average brightness is the same;

其中,所述待融合图像有多个,为了对所有所述待融合图像均进行调整,先对其中一个进行调整后,再对其余的所述待融合图像分别进行调整,直至全部调整完毕为止;在具体施行该方案时,也可以采用并行的方式,同时对多个待融合图像进行处理。Wherein, there are multiple images to be fused, in order to adjust all the images to be fused, first adjust one of the images to be fused, and then adjust the remaining images to be fused separately until all the adjustments are completed; When implementing the solution, a parallel manner can also be used to process multiple images to be fused at the same time.

其中,参考图像和所述待融合图像具有多个像素点,每个像素点上均具有对应的亮度(也可以称为像素点的像素值或者像素强度值)或亮度值;参考图像和所述待融合图像的平均亮度,即为该图像所有像素点的亮度的平均值。Wherein, the reference image and the image to be fused have a plurality of pixel points, and each pixel point has a corresponding brightness (also referred to as a pixel value or a pixel intensity value of a pixel point) or a brightness value; the reference image and the image to be fused The average brightness of the image to be fused is the average value of the brightness of all pixels in the image.

本步骤中的整体调整,是指同时对所有像素点进行等比例的调整,从而避免在调整过程中造成失真。The overall adjustment in this step refers to adjusting all pixel points in equal proportions at the same time, so as to avoid distortion during the adjustment process.

步骤32,对整体调整后的所述待融合图像的亮度进行微调,得到调整后待融合图像;Step 32: Fine-tune the brightness of the overall adjusted image to be fused to obtain an adjusted image to be fused;

整体调整简单方便,但是同时也会把噪声部分进行等比例的放大或缩小,无法单独对其中的某些像素点进行单独调整。The overall adjustment is simple and convenient, but at the same time, the noise part will be enlarged or reduced in equal proportions, and some of the pixels cannot be adjusted individually.

通过本步骤的微调,可以对待融合图像的噪声部分进行微调,从而避免噪声干扰,最终使得微调后的待融合图像(调整后待融合图像)与参考图像更为接近。Through the fine-tuning in this step, the noise part of the image to be fused can be fine-tuned to avoid noise interference, and finally the fine-tuned image to be fused (the adjusted image to be fused) is closer to the reference image.

步骤33,遍历所有待融合图像,得到对应的所述调整后待融合图像。Step 33: Traverse all the images to be fused to obtain the corresponding adjusted images to be fused.

通过重复执行步骤31-32,从而对其他未调整的待融合图像进行调整,得到对应的所述调整后待融合图像。By repeating steps 31-32, other unadjusted images to be fused are adjusted to obtain the corresponding adjusted images to be fused.

这样,通过整体调整和微调,将待融合图像调整为与参考图像非常接近的调整后待融合图像,消除了(极大地减小了)待融合图像与参考图像的亮度差异,同时通过微调,减小了噪声干扰。In this way, through the overall adjustment and fine-tuning, the image to be fused is adjusted to be an adjusted image to be fused that is very close to the reference image, which eliminates (greatly reduces) the difference in brightness between the image to be fused and the reference image. Less noise interference.

可选的,如图4所示,所述步骤31,根据所述参考图像的平均亮度对所述待融合图像的平均亮度进行整体调整,包括:Optionally, as shown in FIG. 4 , in step 31, the average brightness of the to-be-fused image is adjusted as a whole according to the average brightness of the reference image, including:

步骤311,根据所述待融合图像和所述参考图像中各个像素点的亮度,确定所述待融合图像和所述参考图像的平均亮度;Step 311, according to the brightness of each pixel in the to-be-fused image and the reference image, determine the average brightness of the to-be-fused image and the reference image;

其中,参考图像和所述待融合图像的平均亮度,即为该图像所有像素点的亮度的平均值。The average brightness of the reference image and the to-be-fused image is the average value of the brightness of all pixels in the image.

步骤312,根据所述待融合图像和所述参考图像的平均亮度确定所述待融合图像和所述参考图像的亮度差异比例;Step 312: Determine the brightness difference ratio between the to-be-fused image and the reference image according to the average brightness of the to-be-fused image and the reference image;

其中,所述亮度差异比例为所述参考图像的平均亮度与所述待融合图像的平均亮度的比值。The luminance difference ratio is a ratio of the average luminance of the reference image to the average luminance of the to-be-fused image.

比如,待融合图像的平均亮度为127.0,参考图像的平均亮度为65.0;则所述待融合图像和所述参考图像的亮度差异比例为65/127。(该亮度差异比例是作为一个中间变量来整体调整所述待融合图像,因此分子与分母的位置可以颠倒,只需要在后续整体调整中选择对应的除以操作即可)For example, the average brightness of the image to be fused is 127.0, and the average brightness of the reference image is 65.0; then the ratio of the brightness difference between the image to be fused and the reference image is 65/127. (The brightness difference ratio is used as an intermediate variable to adjust the image to be fused as a whole, so the positions of the numerator and denominator can be reversed, just select the corresponding division operation in the subsequent overall adjustment)

步骤313,根据所述亮度差异比例对所述待融合图像中各个像素点的亮度进行整体调整,整体调整后所述待融合图像的平均亮度和所述参考图像的平均亮度相同。Step 313: Adjust the brightness of each pixel in the to-be-fused image as a whole according to the brightness difference ratio, and after the overall adjustment, the average brightness of the to-be-fused image is the same as the average brightness of the reference image.

本步骤中的整体调整,即为将所述待融合图像中各个像素点的亮度均乘以所述亮度差异比例,从而使得整体调整后的所述待融合图像的平均亮度和所述参考图像的平均亮度相同。The overall adjustment in this step is to multiply the brightness of each pixel in the to-be-fused image by the brightness difference ratio, so that the overall adjusted average brightness of the to-be-fused image and the reference image Average brightness is the same.

可选的,所述对所述整体调整后的所述待融合图像中对应位置处的像素点的亮度进行调整,包括:通过将所述整体调整后的所述待融合图像中对应位置处的像素点的亮度与对应的所述差异比例相乘进行调整。Optionally, the adjusting the brightness of the pixels at the corresponding positions in the image to be fused after the overall adjustment includes: adjusting the brightness of the pixels at the corresponding positions in the image to be fused after the overall adjustment. The brightness of the pixel is adjusted by multiplying the corresponding difference ratio.

其中,所述亮度差异比例为所述参考图像的平均亮度与所述待融合图像的平均亮度的比值。该亮度差异比例是作为一个中间变量来整体调整所述待融合图像,因此分子与分母的位置可以颠倒,只需要在后续整体调整中选择对应的除以操作即可。The luminance difference ratio is a ratio of the average luminance of the reference image to the average luminance of the to-be-fused image. The luminance difference ratio is used as an intermediate variable to adjust the image to be fused as a whole, so the positions of the numerator and the denominator can be reversed, and it is only necessary to select the corresponding division operation in the subsequent overall adjustment.

比如,若待融合图像的平均亮度为127.0,参考图像的平均亮度为65.0;如果将所述待融合图像和所述参考图像的亮度差异比例记为127/65,则整体调整即是将所述待融合图像中各个像素点的亮度均除以亮度差异比例;如果将所述待融合图像和所述参考图像的亮度差异比例记为65/127,则整体调整即是将所述待融合图像中各个像素点的亮度均乘以亮度差异比例;整体调整后待融合图像的平均亮度为65.0。For example, if the average brightness of the image to be fused is 127.0, and the average brightness of the reference image is 65.0; if the ratio of the brightness difference between the image to be fused and the reference image is recorded as 127/65, the overall adjustment is to adjust the The brightness of each pixel in the image to be fused is divided by the ratio of the brightness difference; if the ratio of the brightness difference between the image to be fused and the reference image is recorded as 65/127, the overall adjustment is to divide the image to be fused. The brightness of each pixel is multiplied by the brightness difference ratio; the average brightness of the image to be fused after the overall adjustment is 65.0.

这样,通过整体调整可以快速将所述待融合图像的亮度贴近所述参考图像,且等比例调整减少了整体调整造成失真的可能性。In this way, the brightness of the image to be fused can be quickly brought close to the reference image through the overall adjustment, and the proportional adjustment reduces the possibility of distortion caused by the overall adjustment.

可选的,如图5所示,所述步骤32,对整体调整后的所述待融合图像的亮度进行微调,得到调整后待融合图像,包括:Optionally, as shown in FIG. 5 , in step 32, the brightness of the image to be fused after the overall adjustment is fine-tuned to obtain the adjusted image to be fused, including:

步骤321,将所述参考图像和整体调整后的所述待融合图像缩小;Step 321, reducing the reference image and the overall adjusted image to be fused;

其中,将所述参考图像和所述待融合图像缩小,是指将所述参考图像和所述待融合图像由大图缩小到小图,比如由4000像素×3000像素的大图,按照1/16的比例,缩小为1000像素×750像素的小图。Wherein, reducing the reference image and the image to be fused refers to reducing the reference image and the image to be fused from a large image to a small image, such as a large image of 4000 pixels×3000 pixels, according to 1/ A scale of 16, reduced to a small image of 1000 pixels x 750 pixels.

其中,将大图缩小到小图的具体缩小方式,可以根据实际情况确定,但是需要明确的是,所述参考图像和整体调整后的所述待融合图像需要通过同样的方式进行缩小。The specific reduction method for reducing the large image to the small image can be determined according to the actual situation, but it should be clarified that the reference image and the image to be fused after the overall adjustment need to be reduced in the same way.

这其中需要说明的是,如果所述待融合图像的数量较多,需要重复执行步骤32,以及步骤321-325,那么在首次执行本步骤中,可以对所述参考图像和整体调整后的所述待融合图像进行相同比例的缩小,在非首次执行本步骤时,可以不再对参考图像进行缩小而是直接读取首次执行时获得的缩小后的参考图像,仅缩小整体调整后的所述待融合图像即可。It should be noted that if the number of the images to be fused is large, step 32 and steps 321-325 need to be repeatedly performed, then when this step is performed for the first time, the reference image and the overall adjusted all The image to be fused is reduced in the same proportion. When this step is not performed for the first time, the reference image can no longer be reduced, but the reduced reference image obtained during the first execution can be directly read, and only the overall adjusted The images can be merged.

步骤322,对缩小后的所述参考图像和缩小后的所述待融合图像进行相同参数的引导滤波;Step 322, performing guided filtering with the same parameters on the reduced reference image and the reduced image to be fused;

本步骤中,相同参数的引导滤波可以为除了输入图像和引导图像(可能将输入图像作为引导图像)不同外,其余部分均相同。In this step, the guided filtering of the same parameters may be the same except that the input image and the guiding image (possibly using the input image as the guiding image) are different.

这其中需要说明的是,如果所述待融合图像的数量较多,需要重复执行步骤32,以及步骤321-325,那么在首次执行本步骤中,可以对所述参考图像和所述待融合图像进行相同参数的引导滤波;在非首次执行本步骤时,可以不再对参考图像进行引导滤波而是直接读取首次执行时获得的滤波后的参考图像,仅对所述待融合图像进行相同参数的引导滤波即可。It should be noted that if the number of the images to be fused is large, step 32 and steps 321 to 325 need to be performed repeatedly, then when this step is performed for the first time, the reference image and the image to be fused can be Perform guided filtering with the same parameters; when this step is not performed for the first time, the guided filtering may not be performed on the reference image but directly read the filtered reference image obtained during the first execution, and only perform the same parameters on the image to be fused The guided filter can be used.

通过引导滤波,可以使得所述待融合图像和所述参考图像中附近相似区域像素强度接近,避免噪声干扰。Through guided filtering, the pixel intensities of nearby similar regions in the image to be fused and the reference image can be made close to avoid noise interference.

步骤323,根据滤波后的所述参考图像和滤波后的所述待融合图像中各个像素点的亮度,确定各个像素点的差异比例,并存入调整掩码中;Step 323, according to the filtered reference image and the filtered brightness of each pixel in the to-be-fused image, determine the difference ratio of each pixel, and store it in the adjustment mask;

其中,所述待融合图像和所述参考图像中各个像素点的差异比例,即为所述参考图像的对应像素点的亮度和所述待融合图像的对应像素点的亮度的比值。比如,待融合图像中某个像素点的亮度为139.0,参考图像中对应位置的像素点的亮度为61.0;则所述待融合图像和所述参考图像中该像素点的差异比例为61/139。(该差异比例是作为一个中间变量来微调所述待融合图像,因此分子与分母的位置可以颠倒,只需要在后续微调中选择对应的除以操作即可)The difference ratio of each pixel in the image to be fused and the reference image is the ratio of the brightness of the corresponding pixel of the reference image to the brightness of the corresponding pixel of the image to be fused. For example, the brightness of a certain pixel in the image to be fused is 139.0, and the brightness of the pixel at the corresponding position in the reference image is 61.0; then the difference ratio of the pixel in the image to be fused and the reference image is 61/139 . (The difference ratio is used as an intermediate variable to fine-tune the image to be fused, so the positions of the numerator and denominator can be reversed, and it is only necessary to select the corresponding division operation in the subsequent fine-tuning)

需要说明的是,上述差异比例是针对各个像素点分别进行计算的,也即是所述参考图像有n个像素点,就需要计算n次,得到n个差异比例。若缩小后的参考图像(待融合图像)为1000像素×750像素的小图,则具有75,0000个像素点,需要计算75,0000个差异比例。It should be noted that the above difference ratios are calculated for each pixel respectively, that is, if the reference image has n pixels, it needs to be calculated n times to obtain n difference ratios. If the reduced reference image (the image to be fused) is a small image of 1000 pixels×750 pixels, there are 750,000 pixels, and 750,000 difference ratios need to be calculated.

其中,所述调整掩码是与待融合图像对应的矩阵或图像,其中记载着所有计算得到的差异比例(n个,75,0000个),且存储到对应位置。The adjustment mask is a matrix or image corresponding to the image to be fused, in which all calculated difference ratios (n, 75,0000) are recorded and stored in the corresponding position.

步骤324,将所述调整掩码放大到与整体调整前的所述待融合图像大小相同;Step 324, enlarging the adjustment mask to the same size as the image to be fused before the overall adjustment;

本步骤中,即是将调整掩码由小图放大到大图,该大图的尺寸大小与整体调整前的所述待融合图像相同。In this step, the adjustment mask is enlarged from a small image to a large image, and the size of the large image is the same as the image to be fused before the overall adjustment.

其中,将小图放大到大图的具体放大方式,可以根据实际情况确定。The specific enlargement method for enlarging the small image to the large image can be determined according to the actual situation.

比如,调整掩码为1000像素×750像素的小图(如果将调整掩码视为一张图),本步骤即为按照16/1的比例,放大为4000像素×3000像素的大图。这样放大后,调整掩码与各个像素点的对应位置处均会标有对应的差异比例。For example, if the adjustment mask is a small image of 1000 pixels × 750 pixels (if the adjustment mask is regarded as a single image), this step is to enlarge the image to a large image of 4000 pixels × 3000 pixels according to the ratio of 16/1. After zooming in in this way, the corresponding position of the adjustment mask and each pixel point will be marked with the corresponding difference ratio.

步骤325,根据放大后的所述调整掩码的差异比例,对所述整体调整后的所述待融合图像中对应位置处的像素点的亮度进行调整,得到调整后待融合图像。Step 325: According to the difference ratio of the enlarged adjustment mask, adjust the brightness of the pixels at the corresponding positions in the overall adjusted image to be fused to obtain the adjusted image to be fused.

可选的,所述对所述整体调整后的所述待融合图像中对应位置处的像素点的亮度进行调整,包括:通过将所述整体调整后的所述待融合图像中对应位置处的像素点的亮度与对应的所述差异比例相乘进行调整。Optionally, the adjusting the brightness of the pixels at the corresponding positions in the image to be fused after the overall adjustment includes: adjusting the brightness of the pixels at the corresponding positions in the image to be fused after the overall adjustment. The brightness of the pixel is adjusted by multiplying the corresponding difference ratio.

其中,所述待融合图像和所述参考图像中各个像素点的差异比例,即为所述参考图像的对应像素点的亮度和所述待融合图像的对应像素点的亮度的比值。The difference ratio of each pixel in the image to be fused and the reference image is the ratio of the brightness of the corresponding pixel of the reference image to the brightness of the corresponding pixel of the image to be fused.

该差异比例是作为一个中间变量来微调所述待融合图像,因此分子与分母的位置可以颠倒,只需要在后续微调中选择对应的除以操作即可。The difference ratio is used as an intermediate variable to fine-tune the image to be fused, so the positions of the numerator and the denominator can be reversed, and it is only necessary to select the corresponding division operation in the subsequent fine-tuning.

比如,若待融合图像中某个像素点的亮度为139.0,参考图像中对应位置的像素点的亮度为61.0;如果将所述待融合图像和所述参考图像中该像素点的差异比例记为139/61,则微调即是将所述待融合图像中该个像素点的亮度除以差异比例;如果将所述待融合图像和所述参考图像中该像素点的差异比例记为61/139,则微调即是将所述待融合图像中该个像素点的亮度乘以差异比例。For example, if the brightness of a certain pixel in the image to be fused is 139.0, the brightness of the pixel at the corresponding position in the reference image is 61.0; if the difference ratio of the pixel in the image to be fused and the reference image is recorded as 139/61, then fine-tuning is to divide the brightness of the pixel in the image to be fused by the difference ratio; if the difference ratio of the pixel in the image to be fused and the reference image is recorded as 61/139 , then the fine-tuning is to multiply the brightness of the pixel in the to-be-fused image by the difference ratio.

这样,将平均亮度调整(整体调整)后的待融合图像进行小图放大进行微调,可以保证待融合图像附近相似区域像素强度接近,避免噪声干扰。In this way, the image to be fused after the average brightness adjustment (overall adjustment) is zoomed in and fine-tuned, which can ensure that the pixel intensities of similar areas near the image to be fused are close to avoid noise interference.

需要说明的是,小图放大过程中会出现像素扩散导致像素比例外泄,如果没有事先将待融合图像亮度拉到和参考图像一个水平,那么亮度差异会很大的区域上,这些比例权重外泄会导致亮度调整时候,有像素使用到错误的区域亮度比例,如果相邻区域亮度比例变化较大时候,容易导待融合图像调整亮度后图像出现黑边或者白边的情况,如图6A所示,即为直接小图放大得到的调整后待融合图像,可以看到出现了很多黑边和白边的情况,失真严重。It should be noted that pixel diffusion will occur during the zoom-in process of the small image, resulting in the leakage of the pixel ratio. If the brightness of the image to be fused is not pulled to the same level as the reference image in advance, then the brightness difference will be large in the area. When the brightness is adjusted, some pixels use the wrong area brightness ratio. If the brightness ratio of adjacent areas changes greatly, it is easy to cause black or white edges in the image to be fused after the brightness is adjusted, as shown in Figure 6A. It is the image to be fused after adjustment obtained by directly zooming in on the small image. It can be seen that there are many black and white borders, and the distortion is serious.

因此,先平均亮度调整再小图放大,可以防止在小图放大过程中,由于像素扩散导致的调整后待融合图像出现了很多黑边和白边的情况,如图6B所示,即为先平均亮度调整再小图放大得到的调整后待融合图像,可以看出基本无黑边和白边的情况。Therefore, adjusting the average brightness first and then zooming in on the small image can prevent a lot of black and white borders in the image to be fused after adjustment due to pixel diffusion during the process of zooming in on the small image, as shown in Figure 6B. After adjusting the average brightness and zooming in on the small image to obtain the adjusted image to be fused, it can be seen that there are basically no black borders and white borders.

可选的,如图7所示,所述步骤40,将所述参考图像和所述调整后待融合图像进行融合,得到HDR融合图像,包括:Optionally, as shown in FIG. 7 , in step 40, the reference image and the adjusted image to be fused are fused to obtain an HDR fusion image, including:

步骤41,根据所述调整后待融合图像和所述参考图像,确定所述调整后待融合图像的融合权重和所述参考图像的融合权重;Step 41, according to the adjusted image to be fused and the reference image, determine the fusion weight of the adjusted image to be fused and the fusion weight of the reference image;

其中,根据所述调整后待融合图像确定所述调整后待融合图像的融合权重,根据所述参考图像,确定所述参考图像的融合权重。在此需要说明的是,融合权重的确定是以单个像素点为单位进行逐个确定的。The fusion weight of the adjusted image to be fused is determined according to the adjusted image to be fused, and the fusion weight of the reference image is determined according to the reference image. It should be noted here that the determination of the fusion weight is performed one by one in a unit of a single pixel.

其中,我们可以确定所述参考图像为或者近似是所述调整后待融合图像的其中之一,因此,所述参考图像的融合权重的确定方式和调整后待融合图像的融合权重的确定方式也很相似或相同;在此为了便于计算,我们将其均通过同样的方式来确定权重。Among them, we can determine that the reference image is or is approximately one of the adjusted images to be fused. Therefore, the method of determining the fusion weight of the reference image and the method of determining the fusion weight of the adjusted image to be fused are also are very similar or the same; here, for the sake of calculation, we will use the same method to determine the weight.

根据图像形成过程中的曝光时间不同,所述待融合图像分为过曝图像、欠曝图像和正常曝光图像三种,不同种类的待融合图像的优缺点不同,因此融合权重的确定方式也不同。According to the different exposure times in the image forming process, the images to be fused are divided into three types: over-exposed images, under-exposed images and normal exposure images. Different types of images to be fused have different advantages and disadvantages, so the determination methods of fusion weights are also different .

对于过曝图像而言,其像素点的亮度越低,分配的融合权重越高;对于欠曝图像而言,其像素点的亮度越高,分配的融合权重越高;对于正常曝光图像而言,当像素点的亮度越靠近中间位置(中间位置的亮度由亮度阈值来决定),其分配的融合权重越高,越靠近两端位置,其分配的融合权重越低。For an overexposed image, the lower the brightness of the pixel, the higher the assigned fusion weight; for an underexposed image, the higher the brightness of the pixel, the higher the assigned fusion weight; for a normal exposure image , when the brightness of the pixel is closer to the middle position (the brightness of the middle position is determined by the brightness threshold), the assigned fusion weight is higher, and the closer to the two ends, the lower the assigned fusion weight.

可选的,所述待融合图像为所述过曝图像时,对应的所述调整后待融合图像的融合权重的获取方式为:Optionally, when the image to be fused is the overexposed image, the corresponding method for obtaining the fusion weight of the image to be fused after adjustment is:

Figure BDA0002243687920000161
Figure BDA0002243687920000161

式中,w+为过曝图像对应的调整后待融合图像的融合权重,p为调整后待融合图像中像素点的亮度,pt为亮度阈值,s为平滑度。In the formula, w + is the fusion weight of the adjusted image to be fused corresponding to the overexposed image, p is the brightness of the pixels in the adjusted image to be fused, p t is the brightness threshold, and s is the smoothness.

可选的,所述待融合图像为所述欠曝图像时,对应的所述调整后待融合图像(或者参考图像)的融合权重的获取方式为:Optionally, when the image to be fused is the underexposed image, the corresponding method for obtaining the fusion weight of the adjusted image to be fused (or the reference image) is:

Figure BDA0002243687920000171
Figure BDA0002243687920000171

式中,w-为欠曝图像对应的调整后待融合图像的融合权重,p为调整后待融合图像中像素点的亮度,pt为亮度阈值,s为平滑度。In the formula, w - is the fusion weight of the adjusted image to be fused corresponding to the underexposed image, p is the brightness of the pixels in the adjusted image to be fused, p t is the brightness threshold, and s is the smoothness.

其中,平滑度s的取值范围为【1,10】;这样,通过设置平滑度,使得融合权重更加过渡自然平滑,不存在截断分成,更方便进行融合。Among them, the value range of the smoothness s is [1, 10]; in this way, by setting the smoothness, the fusion weight is made more natural and smooth, and there is no truncation and division, which is more convenient for fusion.

其中,所述亮度阈值pt根据实际情况或工程调试确定。Wherein, the brightness threshold pt is determined according to the actual situation or engineering debugging.

可选的,所述待融合图像为所述正常曝光图像时,对应的所述调整后待融合图像(或者参考图像)的融合权重的获取方式为:Optionally, when the image to be fused is the normal exposure image, the corresponding method for obtaining the fusion weight of the adjusted image to be fused (or the reference image) is:

Figure BDA0002243687920000172
Figure BDA0002243687920000172

式中,w为正常曝光图像对应的调整后待融合图像的融合权重,p为调整后待融合图像中像素点的亮度,pt为亮度阈值,s为平滑度。In the formula, w is the fusion weight of the adjusted image to be fused corresponding to the normal exposure image, p is the brightness of the pixel in the adjusted image to be fused, p t is the brightness threshold, and s is the smoothness.

步骤42,根据所述调整后待融合图像的融合权重和所述参考图像的融合权重,将所述调整后待融合图像和所述参考图像进行融合,得到HDR融合图像。Step 42: According to the adjusted fusion weight of the image to be fused and the fusion weight of the reference image, fuse the adjusted image to be fused and the reference image to obtain an HDR fusion image.

在此需要说明的是,图像融合是以单个像素点为单位进行逐个确定的;也即是将所述调整后待融合图像和所述参考图像对应位置处的所有像素点的亮度进行计算,得到的新的亮度即为HDR融合图像对应位置处的像素点的亮度。这样,对图像中各个位置处的像素点逐个进行计算,最终得到HDR融合图像的所有像素点的亮度。It should be noted here that the image fusion is determined one by one with a single pixel as the unit; that is, the brightness of all pixels at the corresponding positions of the adjusted image to be fused and the reference image is calculated to obtain The new brightness is the brightness of the pixel at the corresponding position of the HDR fusion image. In this way, the pixel points at each position in the image are calculated one by one, and finally the brightness of all the pixel points of the HDR fusion image is obtained.

其中,所述调整后待融合图像和所述参考图像对应位置处的所有像素点的亮度进行计算,得到HDR融合图像对应位置处的像素点的亮度,即为将所述调整后待融合图像和所述参考图像对应位置处的像素点的亮度与该图像的融合权重相乘后求和,然后除以所有该位置处所有融合权重之和(所述调整后待融合图像和所述参考图像在位置处的融合权重的和),得到HDR融合图像对应位置处的像素点的亮度。Wherein, the brightness of all pixels at the corresponding positions of the adjusted image to be fused and the reference image is calculated to obtain the brightness of the pixels at the corresponding positions of the HDR fusion image, that is, the adjusted image to be fused and the corresponding position of the reference image. The brightness of the pixel at the corresponding position of the reference image is multiplied by the fusion weight of the image and summed, and then divided by the sum of all fusion weights at all the positions (the adjusted image to be fused and the reference image are The sum of the fusion weights at the positions) to obtain the brightness of the pixels at the corresponding positions of the HDR fusion image.

这样,通过分配融合权重后再进行图像融合,可以使得最终得到的HDR融合图像的整体亮度噪声都很自然。In this way, by assigning fusion weights and then performing image fusion, the overall luminance noise of the final HDR fusion image can be made natural.

可选的,如图8所示,所述步骤20,根据所述欠曝图像或正常曝光图像进行亮度调整后确定参考图像,包括:Optionally, as shown in FIG. 8 , in step 20, a reference image is determined after brightness adjustment is performed according to the underexposed image or the normally exposed image, including:

步骤21,从所述待融合图像中选取一个欠曝图像或正常曝光图像作为待参考图像;Step 21, select an underexposed image or a normal exposure image from the to-be-fused image as the to-be-referenced image;

其中,选取的正常曝光图像或欠曝图像;这是由于过曝图像的过曝区域中信息量少,若以过曝图像生成参考图像,则会导致参考图像中的过曝区域信息量过少,其他待融合图像根据参考图像进行调整时,对应过曝区域的部分容易失真,进而导致融合的HDR图像失真严重;不选择过曝图像,则可以避免过曝区域造成的融合HDR图像失真。Among them, the selected normal exposure image or underexposed image; this is due to the small amount of information in the overexposed area of the overexposed image. If the overexposed image is used to generate the reference image, the information content of the overexposed area in the reference image will be too small. , when other images to be fused are adjusted according to the reference image, the part corresponding to the overexposed area is easily distorted, which will lead to serious distortion of the fused HDR image; if the overexposed image is not selected, the distortion of the fused HDR image caused by the overexposed area can be avoided.

因此所述待参考图像为正常曝光图像或欠曝图像,从而可以避免生成的参考图像中的由于信息量不足导致过曝区域失真。Therefore, the to-be-referenced image is a normal-exposed image or an under-exposed image, so that distortion of the over-exposed area in the generated reference image due to insufficient information can be avoided.

可选的,若所述待融合图像包括正常曝光图像和欠曝图像,则选取所述正常曝光图像中的一个作为所述待参考图像;若所述待融合图像包括欠曝图像但不包括正常曝光图像,则选取所述欠曝图像中的一个作为所述待参考图像。Optionally, if the to-be-fused image includes a normal exposure image and an underexposed image, select one of the normal exposure images as the to-be-referenced image; if the to-be-fused image includes an underexposed image but does not include a normal exposure image. exposure image, select one of the underexposed images as the to-be-referenced image.

步骤22,根据所述待参考图像确定所述待参考图像中各个像素点的过曝掩码;Step 22, determining the overexposure mask of each pixel in the to-be-referenced image according to the to-be-referenced image;

其中,所述过曝掩码是用以反映对应的像素点是否位于过曝区域,也即是说,0值的过曝掩码代表对应的像素点没有过曝,不位于过曝区域;正值的过曝掩码代表对应的像素点过曝,位于过曝区域。Wherein, the overexposure mask is used to reflect whether the corresponding pixel is located in the overexposure area, that is to say, the overexposure mask with a value of 0 represents that the corresponding pixel point is not overexposed and is not located in the overexposed area; The value of the overexposure mask represents the overexposure of the corresponding pixel, which is located in the overexposed area.

可选的,所述过曝掩码的获取方式为:Optionally, the acquisition method of the overexposure mask is:

E=min(n,1)E=min(n,1)

Figure BDA0002243687920000191
Figure BDA0002243687920000191

式中,E为像素点的过曝掩码,p为待参考图像中像素点的亮度,pt为亮度阈值,s为平滑度,n为中间变量。In the formula, E is the overexposure mask of the pixel point, p is the brightness of the pixel point in the image to be referenced, pt is the brightness threshold, s is the smoothness, and n is the intermediate variable.

通过该获取公式,可以看出,若亮度小于等于亮度阈值时,过曝掩码为0;若亮度大于亮度阈值,过曝掩码为大于0小于等于1的正值。From this acquisition formula, it can be seen that if the brightness is less than or equal to the brightness threshold, the overexposure mask is 0; if the brightness is greater than the brightness threshold, the overexposure mask is a positive value greater than 0 and less than or equal to 1.

这样,通过设置亮度阈值,可以判断像素点是否过曝,通过设置平滑度,使得对过曝区域(过曝掩码为正值的像素点构成的区域)的检测结果更加过渡自然平滑,不存在截断分成,更方便融合时候。In this way, by setting the brightness threshold, it is possible to determine whether the pixel is overexposed, and by setting the smoothness, the detection result of the overexposed area (the area formed by the pixels whose overexposure mask is a positive value) is more natural and smooth, and there is no Truncated and divided, it is more convenient to merge.

步骤23,获取预设的调整参数,根据所述调整参数和所述过曝掩码确定所述待参考图像中各个像素点的亮度调整参数;Step 23: Obtain preset adjustment parameters, and determine brightness adjustment parameters of each pixel in the to-be-referenced image according to the adjustment parameters and the overexposure mask;

其中,所述预设的调整参数可以预设在电子设备内,以便于需要时进行读取;该调整参数可以是相机等电子设备默认的数值,也可以是由客户根据感官确定的数值。The preset adjustment parameters may be preset in the electronic device to facilitate reading when needed; the adjustment parameters may be default values of electronic devices such as cameras, or values determined by customers according to their senses.

需要说明的是,亮度调整参数即为像素点的亮度放大倍数;因此对于完全过曝的像素点(亮度为255.0),其放大倍数为1倍(也即是保持不变,不再放大)。It should be noted that the brightness adjustment parameter is the brightness magnification of the pixel; therefore, for a pixel that is completely overexposed (brightness is 255.0), its magnification is 1 times (that is, it remains unchanged and no longer magnifies).

可选的,所述亮度调整参数的获取方式为:Optionally, the acquisition method of the brightness adjustment parameter is:

a=(A-1)×E+1a=(A-1)×E+1

式中,a为像素点的亮度调整参数,A为调整参数,E为像素点的过曝掩码。In the formula, a is the brightness adjustment parameter of the pixel, A is the adjustment parameter, and E is the overexposure mask of the pixel.

这样,通过计算方式确定亮度调整参数,简单、快速、方便。In this way, the brightness adjustment parameter is determined by calculation, which is simple, fast and convenient.

步骤24,根据所述亮度调整参数调整所述待参考图像中各个像素点的亮度,得到所述参考图像。Step 24: Adjust the brightness of each pixel in the to-be-referenced image according to the brightness adjustment parameter to obtain the reference image.

本步骤中,根据所述亮度调整参数调整所述待参考图像中各个像素点的亮度,即为将述亮度调整参数与各个像素点的亮度相乘,从而得到调整后的所述参考图像。In this step, adjusting the brightness of each pixel in the to-be-referenced image according to the brightness adjustment parameter is to multiply the brightness adjustment parameter by the brightness of each pixel to obtain the adjusted reference image.

这样,通过预设调整参数,对像素点的亮度进行调整,从而可以将调整后的亮度作为最终生成的HDR融合图像的整体亮度,从而与客户的喜好风格对应,增加客户的体验感和使用的舒适性。并且通过检测过曝区域,防止在亮度调整时候,导致亮区过曝,并且由于过曝掩码过渡平滑自然,所有也不会出现亮度异常跳变的过渡带。In this way, the brightness of the pixels is adjusted by preset adjustment parameters, so that the adjusted brightness can be used as the overall brightness of the final generated HDR fusion image, so as to correspond to the customer's preferred style and increase the customer's sense of experience and use. comfort. And by detecting the overexposed area, it prevents the bright area from being overexposed when the brightness is adjusted, and because the transition of the overexposure mask is smooth and natural, there will be no transition band with abnormal brightness jumps.

本公开实施例提供了一种夜景高动态范围图像融合装置,用于执行本发明上述内容所述的夜景高动态范围图像融合方法,以下对所述夜景高动态范围图像融合装置进行详细描述。Embodiments of the present disclosure provide a night scene high dynamic range image fusion device for implementing the night scene high dynamic range image fusion method described in the foregoing content of the present invention. The night scene high dynamic range image fusion device is described in detail below.

如图9所示,一种夜景高动态范围图像融合装置,其包括:As shown in Figure 9, a night scene high dynamic range image fusion device, which includes:

获取单元1,用于获取多个待融合图像,所述待融合图像至少包括过曝图像、欠曝图像和正常曝光图像中的任意两种;an acquisition unit 1, configured to acquire a plurality of images to be fused, the images to be fused at least include any two of an overexposed image, an underexposed image and a normal exposure image;

确定单元2,用于根据所述欠曝图像或正常曝光图像进行亮度调整后确定参考图像;A determination unit 2, configured to determine a reference image after performing brightness adjustment according to the underexposed image or the normally exposed image;

调整单元3,用于根据所述参考图像对所述待融合图像的亮度进行调整,得到调整后待融合图像;Adjustment unit 3, configured to adjust the brightness of the to-be-fused image according to the reference image, to obtain the adjusted to-be-fused image;

融合单元4,用于将所述调整后待融合图像进行融合,得到HDR融合图像。The fusion unit 4 is configured to fuse the adjusted images to be fused to obtain an HDR fusion image.

这样,既不必确定相机参数信息或者预先场景先验信息即可对待融合图像进行HDR融合,时间复杂度低,也可以减少不同曝光图像之间的噪声差异,极大地减少甚至避免融合得到的图像出现突兀的亮区域以及不自然的噪声过度。从而可以在低时间复杂度的情况下,达到更好的融合效果,减少了融合图像中出现突兀的亮区域及不自然噪声的可能。In this way, it is not necessary to determine camera parameter information or pre-scenario prior information to perform HDR fusion on the images to be fused, with low time complexity, and can also reduce noise differences between different exposure images, greatly reducing or even avoiding the appearance of fused images. Obtrusive bright areas and unnatural excessive noise. Therefore, a better fusion effect can be achieved with low time complexity, and the possibility of abrupt bright areas and unnatural noises appearing in the fusion image is reduced.

可选的,所述调整单元3,还用于:针对每一个所述待融合图像,根据所述参考图像的平均亮度对所述待融合图像的平均亮度进行整体调整,整体调整后所述待融合图像的平均亮度与所述参考图像的平均亮度相同;对整体调整后的所述待融合图像的亮度进行微调,得到调整后待融合图像;遍历所有待融合图像,得到对应的所述调整后待融合图像。Optionally, the adjustment unit 3 is further configured to: for each image to be fused, perform an overall adjustment on the average brightness of the image to be fused according to the average brightness of the reference image, and after the overall adjustment, the image to be fused is adjusted. The average brightness of the fused image is the same as the average brightness of the reference image; fine-tune the brightness of the overall adjusted image to be fused to obtain the adjusted image to be fused; traverse all the images to be fused to obtain the corresponding adjusted image image to be fused.

可选的,所述调整单元3,还用于:根据所述待融合图像和所述参考图像中各个像素点的亮度,确定所述待融合图像和所述参考图像的平均亮度;根据所述待融合图像和所述参考图像的平均亮度确定所述待融合图像和所述参考图像的亮度差异比例;根据所述亮度差异比例对所述待融合图像中各个像素点的亮度进行整体调整,整体调整后所述待融合图像的平均亮度和所述参考图像的平均亮度相同。Optionally, the adjustment unit 3 is further configured to: determine the average brightness of the to-be-fused image and the reference image according to the brightness of each pixel in the to-be-fused image and the reference image; The average brightness of the image to be fused and the reference image determines the ratio of the brightness difference between the image to be fused and the reference image; After adjustment, the average brightness of the image to be fused is the same as the average brightness of the reference image.

可选的,所述调整单元3,还用于:将所述参考图像和整体调整后的所述待融合图像缩小;对缩小后的所述参考图像和缩小后的所述待融合图像进行相同参数的引导滤波;根据滤波后的所述参考图像和滤波后的所述待融合图像中各个像素点的亮度,确定各个像素点的差异比例,并存入调整掩码中;将所述调整掩码放大到与整体调整前的所述待融合图像大小相同;根据放大后的所述调整掩码的差异比例,对所述整体调整后的所述待融合图像中对应位置处的像素点的亮度进行调整,得到调整后待融合图像。Optionally, the adjustment unit 3 is further configured to: reduce the reference image and the overall adjusted image to be fused; perform the same operation on the reduced reference image and the reduced image to be fused. Guided filtering of parameters; according to the filtered reference image and the filtered brightness of each pixel in the to-be-fused image, determine the difference ratio of each pixel, and store it in an adjustment mask; put the adjustment mask The code is enlarged to the same size as the image to be fused before the overall adjustment; according to the difference ratio of the enlarged adjustment mask, the brightness of the pixel at the corresponding position in the image to be fused after the overall adjustment is adjusted. Adjust to obtain the adjusted image to be fused.

可选的,所述待融合图像至少包括过曝图像、欠曝图像和正常曝光图像中的任意两种。Optionally, the to-be-fused image includes at least any two of an overexposed image, an underexposed image, and a normal exposure image.

可选的,所述融合单元4,还用于:根据所述调整后待融合图像和所述参考图像,确定所述调整后待融合图像的融合权重和所述参考图像的融合权重;根据所述调整后待融合图像的融合权重和所述参考图像的融合权重,将所述调整后待融合图像和所述参考图像进行融合,得到HDR融合图像。Optionally, the fusion unit 4 is further configured to: determine the fusion weight of the adjusted image to be fused and the fusion weight of the reference image according to the adjusted image to be fused and the reference image; The adjusted fusion weight of the image to be fused and the fusion weight of the reference image are fused, and the adjusted image to be fused and the reference image are fused to obtain an HDR fusion image.

可选的,所述确定单元2,还用于:从所述待融合图像中选取一个欠曝图像或正常曝光图像作为待参考图像;根据所述待参考图像确定所述待参考图像中各个像素点的过曝掩码;获取预设的调整参数,根据所述调整参数和所述过曝掩码确定所述待参考图像中各个像素点的亮度调整参数;根据所述亮度调整参数调整所述待参考图像中各个像素点的亮度,得到所述参考图像。Optionally, the determining unit 2 is further configured to: select an underexposed image or a normal exposure image from the to-be-fused image as the to-be-referenced image; determine each pixel in the to-be-referenced image according to the to-be-referenced image point overexposure mask; obtain preset adjustment parameters, determine brightness adjustment parameters of each pixel in the to-be-referenced image according to the adjustment parameters and the overexposure mask; adjust the brightness adjustment parameters according to the brightness adjustment parameters The reference image is obtained from the brightness of each pixel in the to-be-referenced image.

可选的,所述过曝掩码的获取方式为:Optionally, the acquisition method of the overexposure mask is:

E=1-min(n,1)E=1-min(n,1)

Figure BDA0002243687920000211
Figure BDA0002243687920000211

式中,E为像素点的过曝掩码,p为待参考图像中像素点的亮度,pt为亮度阈值,s为平滑度,n为中间变量。In the formula, E is the overexposure mask of the pixel point, p is the brightness of the pixel point in the image to be referenced, pt is the brightness threshold, s is the smoothness, and n is the intermediate variable.

可选的,所述亮度调整参数的获取方式为:Optionally, the acquisition method of the brightness adjustment parameter is:

a=(A-1)×E+1a=(A-1)×E+1

式中,a为像素点的亮度调整参数,A为调整参数,E为像素点的过曝掩码。In the formula, a is the brightness adjustment parameter of the pixel, A is the adjustment parameter, and E is the overexposure mask of the pixel.

可选的,所述待融合图像为所述过曝图像时,对应的所述调整后待融合图像的融合权重的获取方式为:Optionally, when the image to be fused is the overexposed image, the corresponding method for obtaining the fusion weight of the image to be fused after adjustment is:

Figure BDA0002243687920000221
Figure BDA0002243687920000221

式中,w+为过曝图像对应的调整后待融合图像的融合权重,p为调整后待融合图像中像素点的亮度,pt为亮度阈值,s为平滑度。In the formula, w + is the fusion weight of the adjusted image to be fused corresponding to the overexposed image, p is the brightness of the pixels in the adjusted image to be fused, p t is the brightness threshold, and s is the smoothness.

可选的,所述待融合图像为所述欠曝图像时,对应的所述调整后待融合图像的融合权重的获取方式为:Optionally, when the image to be fused is the underexposed image, the corresponding acquisition method of the fusion weight of the image to be fused after adjustment is:

Figure BDA0002243687920000222
Figure BDA0002243687920000222

式中,w_为欠曝图像对应的调整后待融合图像的融合权重,p为调整后待融合图像中像素点的亮度,pt为亮度阈值,s为平滑度。In the formula, w _ is the fusion weight of the adjusted image to be fused corresponding to the underexposed image, p is the brightness of the pixel in the adjusted image to be fused, p t is the brightness threshold, and s is the smoothness.

需要说明的是,以上所描述的装置实施例仅仅是示意性的,例如,所述单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,又例如,多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。It should be noted that the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, for example, multiple units. Or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

以上描述了夜景高动态范围图像融合装置的内部功能和结构,如图10所示,实际中,该夜景高动态范围图像融合装置可实现为电子设备,包括:处理器以及存储器,所述存储器存储有控制程序,所述控制程序被处理器执行时实现上述所述的夜景高动态范围图像融合方法。The above describes the internal function and structure of the night scene high dynamic range image fusion device. As shown in FIG. 10, in practice, the night scene high dynamic range image fusion device can be implemented as an electronic device, including: a processor and a memory, and the memory stores There is a control program, which implements the above-mentioned night scene high dynamic range image fusion method when the control program is executed by the processor.

图11是根据本发明实施例示出的另一种电子设备的框图。图11显示的电子设备12仅仅是一个示例,不应对本申请实施例的功能和使用范围带来任何限制。Fig. 11 is a block diagram of another electronic device according to an embodiment of the present invention. The electronic device 12 shown in FIG. 11 is only an example, and should not impose any limitations on the functions and scope of use of the embodiments of the present application.

如图11所示,电子设备12可以通用电子设备的形式实现。电子设备12的组件可以包括但不限于:一个或者多个处理器或者处理单元16,系统存储器28,连接不同系统组件(包括系统存储器28和处理单元16)的总线18。As shown in FIG. 11 , the electronic device 12 may be implemented in the form of a general-purpose electronic device. Components of electronic device 12 may include, but are not limited to, one or more processors or processing units 16 , system memory 28 , and a bus 18 connecting various system components including system memory 28 and processing unit 16 .

总线18表示几类总线结构中的一种或多种,包括存储器总线或者存储器控制器,外围总线,图形加速端口,处理器或者使用多种总线结构中的任意总线结构的局域总线。举例来说,这些体系结构包括但不限于工业标准体系结构(Industry StandardArchitecture;以下简称:ISA)总线,微通道体系结构(Micro Channel Architecture;以下简称:MAC)总线,增强型ISA总线、视频电子标准协会(Video Electronics StandardsAssociation;以下简称:VESA)局域总线以及外围组件互连(Peripheral ComponentInterconnection;以下简称:PCI)总线。Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures. For example, these architectures include, but are not limited to, Industry Standard Architecture (hereinafter referred to as: ISA) bus, Micro Channel Architecture (hereinafter referred to as: MAC) bus, enhanced ISA bus, video electronic standard Association (Video Electronics Standards Association; hereinafter referred to as: VESA) local bus and Peripheral Component Interconnection (Peripheral Component Interconnection; hereinafter referred to as: PCI) bus.

电子设备12典型地包括多种计算机系统可读介质。这些介质可以是任何能够被电子设备12访问的可用介质,包括易失性和非易失性介质,可移动的和不可移动的介质。Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including both volatile and non-volatile media, removable and non-removable media.

存储器28可以包括易失性存储器形式的计算机系统可读介质,例如随机存取存储器(Random Access Memory;以下简称:RAM)30和/或高速缓存存储器32。电子设备12可以进一步包括其它可移动/不可移动的、易失性/非易失性的计算机可读存储介质。仅作为举例,存储系统34可以用于读写不可移动的、非易失性磁介质(图中未显示,通常称为“硬盘驱动器”)。尽管图11中未示出,可以提供用于对可移动非易失性磁盘(例如“软盘”)读写的磁盘驱动器,以及对可移动非易失性光盘(例如:光盘只读存储器(Compact Disc Read OnlyMemory;以下简称:CD-ROM)、数字多功能只读光盘(Digital Video Disc Read OnlyMemory;以下简称:DVD-ROM)或者其它光介质)读写的光盘驱动器。在这些情况下,每个驱动器可以通过一个或者多个数据介质接口与总线18相连。存储器28可以包括至少一个程序产品,该程序产品具有一组(例如至少一个)程序模块,这些程序模块被配置以执行本申请各实施例的功能。The memory 28 may include a computer system readable medium in the form of a volatile memory, such as a random access memory (Random Access Memory; hereinafter referred to as: RAM) 30 and/or a cache memory 32 . Electronic device 12 may further include other removable/non-removable, volatile/non-volatile computer-readable storage media. For example only, storage system 34 may be used to read and write to non-removable, non-volatile magnetic media (not shown, commonly referred to as "hard disk drives"). Although not shown in FIG. 11, disk drives for reading and writing to removable non-volatile magnetic disks (eg, "floppy disks") and removable non-volatile optical disks (eg, compact disk read only memory) may be provided. Disc Read OnlyMemory; hereinafter referred to as: CD-ROM), Digital Video Disc Read Only Memory (hereinafter referred to as: DVD-ROM) or other optical media) read and write optical disc drives. In these cases, each drive may be connected to bus 18 through one or more data media interfaces. Memory 28 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of various embodiments of the present application.

具有一组(至少一个)程序模块42的程序/实用工具40,可以存储在例如存储器28中,这样的程序模块42包括但不限于操作系统、一个或者多个应用程序、其它程序模块以及程序数据,这些示例中的每一个或某种组合中可能包括网络环境的实现。程序模块42通常执行本申请所描述的实施例中的功能和/或方法。A program/utility 40 having a set (at least one) of program modules 42, which may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data , each or some combination of these examples may include an implementation of a network environment. Program modules 42 generally perform the functions and/or methods of the embodiments described herein.

电子设备12也可以与一个或多个外部设备14(例如键盘、指向设备、显示器24等)通信,还可与一个或者多个使得用户能与该计算机系统/服务器12交互的设备通信,和/或与使得该计算机系统/服务器12能与一个或多个其它电子设备进行通信的任何设备(例如网卡,调制解调器等等)通信。这种通信可以通过输入/输出(I/O)接口22进行。并且,电子设备12还可以通过网络适配器20与一个或者多个网络(例如局域网(Local Area Network;以下简称:LAN),广域网(Wide Area Network;以下简称:WAN)和/或公共网络,例如因特网)通信。如图所示,网络适配器20通过总线18与电子设备12的其它模块通信。要说明的是,尽管图中未示出,可以结合电子设备12使用其它硬件和/或软件模块,包括但不限于:微代码、设备驱动器、冗余处理单元、外部磁盘驱动阵列、RAID系统、磁带驱动器以及数据备份存储系统等。The electronic device 12 may also communicate with one or more external devices 14 (eg, keyboard, pointing device, display 24, etc.), and may also communicate with one or more devices that enable a user to interact with the computer system/server 12, and/or Or with any device (eg, network card, modem, etc.) that enables the computer system/server 12 to communicate with one or more other electronic devices. Such communication may take place through input/output (I/O) interface 22 . In addition, the electronic device 12 can also communicate with one or more networks (eg, Local Area Network (hereinafter referred to as: LAN), Wide Area Network (hereinafter referred to as: WAN) and/or public network, such as the Internet through the network adapter 20 ) ) communication. As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18 . It is noted that, although not shown, other hardware and/or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, Tape drives and data backup storage systems, etc.

处理单元16通过运行存储在系统存储器28中的程序,从而执行各种功能应用以及数据处理,例如实现前述实施例中提及的方法。The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28 , for example, implements the methods mentioned in the foregoing embodiments.

本发明的电子设备可以是服务器,也可以有限算力的终端设备,本发明的轻量级网络结构尤其适用于后者。所述终端设备的基体实现包括但不限于:智能移动通信终端、无人机、机器人、便携式图像处理设备、安防设备等等。本公开实施例提供了一种计算机可读存储介质,存储有指令,所述指令被处理器加载并执行时实现上述所述的夜景高动态范围图像融合方法。The electronic device of the present invention may be a server or a terminal device with limited computing power, and the lightweight network structure of the present invention is especially suitable for the latter. The base implementation of the terminal device includes but is not limited to: intelligent mobile communication terminal, unmanned aerial vehicle, robot, portable image processing device, security device and so on. An embodiment of the present disclosure provides a computer-readable storage medium storing instructions, and when the instructions are loaded and executed by a processor, the above-mentioned night scene high dynamic range image fusion method is implemented.

本发明实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)或processor(处理器)执行本发明实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。The technical solutions of the embodiments of the present invention are essentially or make contributions to the prior art, or all or part of the technical solutions may be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several The instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the embodiments of the present invention. The aforementioned storage medium includes: a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk and other mediums that can store program codes.

虽然本公开披露如上,但本公开的保护范围并非仅限于此。本领域技术人员在不脱离本公开的精神和范围的前提下,可进行各种变更与修改,这些变更与修改均将落入本发明的保护范围。Although the present disclosure is disclosed above, the scope of protection of the present disclosure is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present disclosure, and these changes and modifications will fall within the protection scope of the present invention.

Claims (17)

1.一种夜景高动态范围图像融合方法,其特征在于,包括:1. a night scene high dynamic range image fusion method, is characterized in that, comprises: 获取多个待融合图像,所述待融合图像至少包括过曝图像、欠曝图像和正常曝光图像中的任意两种;acquiring a plurality of images to be fused, the images to be fused at least include any two of overexposed images, underexposed images and normal exposure images; 根据所述欠曝图像或正常曝光图像进行亮度调整后确定参考图像,包括:确定一个所述欠曝图像或正常曝光图像中各个像素点的过曝掩码,根据预设的调整参数和所述过曝掩码确定各个所述像素点的亮度调整参数,根据所述亮度调整参数调整各个所述像素点的亮度,得到所述参考图像;Determining a reference image after performing brightness adjustment according to the underexposure image or the normal exposure image includes: determining an overexposure mask for each pixel in the underexposure image or the normal exposure image, according to preset adjustment parameters and the The overexposure mask determines the brightness adjustment parameter of each of the pixel points, and adjusts the brightness of each of the pixel points according to the brightness adjustment parameter to obtain the reference image; 根据所述参考图像对所述待融合图像的亮度进行调整,得到调整后待融合图像;adjusting the brightness of the image to be fused according to the reference image to obtain the adjusted image to be fused; 将所述调整后待融合图像进行融合,得到HDR融合图像。The adjusted images to be fused are fused to obtain an HDR fusion image. 2.根据权利要求1所述的夜景高动态范围图像融合方法,其特征在于,所述根据所述参考图像对所述待融合图像的亮度进行调整中,所述待融合图像包括用于确定所述参考图像的所述欠曝图像或者所述正常曝光图像。2 . The night scene high dynamic range image fusion method according to claim 1 , wherein in adjusting the brightness of the image to be fused according to the reference image, the image to be fused includes a method for determining the image to be fused. 3 . the underexposed image or the normally exposed image of the reference image. 3.根据权利要求1所述的夜景高动态范围图像融合方法,其特征在于,所述根据所述参考图像对所述待融合图像的亮度进行调整中,所述待融合图像为除了用于确定所述参考图像的所述欠曝图像或者所述正常曝光图像之外的所有待融合图像;且3 . The night scene high dynamic range image fusion method according to claim 1 , wherein, in the adjustment of the brightness of the to-be-fused image according to the reference image, the to-be-fused image is not used for determining 3 . all images to be fused other than the underexposed image or the normally exposed image of the reference image; and 所述将所述调整后待融合图像进行融合,包括:将所述参考图像和所述调整后待融合图像进行融合。The fusing of the adjusted image to be fused includes: fusing the reference image and the adjusted image to be fused. 4.根据权利要求1所述的夜景高动态范围图像融合方法,其特征在于,所述根据所述参考图像对所述待融合图像的亮度进行调整,包括:4. The night scene high dynamic range image fusion method according to claim 1, wherein the adjusting the brightness of the to-be-fused image according to the reference image comprises: 针对每一个所述待融合图像,根据所述参考图像的平均亮度对所述待融合图像的平均亮度进行整体调整,整体调整后所述待融合图像的平均亮度与所述参考图像的平均亮度相同;For each image to be fused, the average brightness of the image to be fused is adjusted as a whole according to the average brightness of the reference image. After the overall adjustment, the average brightness of the image to be fused is the same as the average brightness of the reference image. ; 对整体调整后的所述待融合图像的亮度进行微调,得到调整后待融合图像;Fine-tune the brightness of the overall adjusted image to be fused to obtain an adjusted image to be fused; 遍历所有待融合图像,得到对应的所述调整后待融合图像。Traverse all the images to be fused to obtain the corresponding adjusted images to be fused. 5.根据权利要求4所述的夜景高动态范围图像融合方法,其特征在于,所述根据所述参考图像的平均亮度对所述待融合图像的平均亮度进行整体调整,包括:5. The night scene high dynamic range image fusion method according to claim 4, wherein the overall adjustment of the average brightness of the image to be fused according to the average brightness of the reference image, comprising: 根据所述待融合图像和所述参考图像中各个像素点的亮度,确定所述待融合图像和所述参考图像的平均亮度;According to the brightness of each pixel in the to-be-fused image and the reference image, determine the average brightness of the to-be-fused image and the reference image; 根据所述待融合图像和所述参考图像的平均亮度确定所述待融合图像和所述参考图像的亮度差异比例;Determine the brightness difference ratio between the to-be-fused image and the reference image according to the average brightness of the to-be-fused image and the reference image; 根据所述亮度差异比例对所述待融合图像中各个像素点的亮度进行整体调整,整体调整后所述待融合图像的平均亮度和所述参考图像的平均亮度相同。The brightness of each pixel in the to-be-fused image is adjusted as a whole according to the brightness difference ratio, and the average brightness of the to-be-fused image after the overall adjustment is the same as the average brightness of the reference image. 6.根据权利要求5所述的夜景高动态范围图像融合方法,其特征在于,所述根据所述亮度差异比例对所述待融合图像中各个像素点的亮度进行整体调整,包括:通过将所述待融合图像中各个像素点的亮度均与所述亮度差异比例相乘进行所述整体调整。6. The night scene high dynamic range image fusion method according to claim 5, wherein the overall adjustment of the brightness of each pixel in the to-be-fused image according to the brightness difference ratio comprises: The overall adjustment is performed by multiplying the brightness of each pixel in the image to be fused by the brightness difference ratio. 7.根据权利要求4所述的夜景高动态范围图像融合方法,其特征在于,所述对整体调整后的所述待融合图像的亮度进行微调,得到调整后待融合图像,包括:7. The night scene high dynamic range image fusion method according to claim 4, characterized in that, performing fine-tuning on the brightness of the overall adjusted image to be fused to obtain the adjusted image to be fused, comprising: 将所述参考图像和整体调整后的所述待融合图像缩小;reducing the reference image and the overall adjusted image to be fused; 对缩小后的所述参考图像和缩小后的所述待融合图像进行相同参数的引导滤波;Perform guided filtering with the same parameters on the reduced reference image and the reduced image to be fused; 根据滤波后的所述参考图像和滤波后的所述待融合图像中各个像素点的亮度,确定各个像素点的差异比例,并存入调整掩码中;According to the filtered reference image and the filtered brightness of each pixel in the to-be-fused image, determine the difference ratio of each pixel, and store it in the adjustment mask; 将所述调整掩码放大到与整体调整前的所述待融合图像大小相同;Enlarging the adjustment mask to the same size as the image to be fused before the overall adjustment; 根据放大后的所述调整掩码的差异比例,对所述整体调整后的所述待融合图像中对应位置处的像素点的亮度进行调整,得到调整后待融合图像。According to the difference ratio of the enlarged adjustment mask, the brightness of the pixel at the corresponding position in the overall adjusted image to be fused is adjusted to obtain the adjusted image to be fused. 8.根据权利要求7所述的夜景高动态范围图像融合方法,其特征在于,所述对所述整体调整后的所述待融合图像中对应位置处的像素点的亮度进行调整,包括:通过将所述整体调整后的所述待融合图像中对应位置处的像素点的亮度与对应的所述差异比例相乘进行调整。8 . The night scene high dynamic range image fusion method according to claim 7 , wherein the adjusting the brightness of the pixel points at the corresponding positions in the image to be fused after the overall adjustment comprises: by: 8 . The adjustment is performed by multiplying the brightness of the pixel at the corresponding position in the image to be fused after the overall adjustment by the corresponding difference ratio. 9.根据权利要求1-8中任一所述的夜景高动态范围图像融合方法,其特征在于,所述将所述参考图像和所述调整后待融合图像进行融合,得到HDR融合图像,包括:9 . The night scene high dynamic range image fusion method according to claim 1 , wherein the fusion of the reference image and the adjusted image to be fused to obtain an HDR fusion image, comprising: 10 . : 根据所述调整后待融合图像和所述参考图像,确定所述调整后待融合图像的融合权重和所述参考图像的融合权重;According to the adjusted image to be fused and the reference image, determine the fusion weight of the adjusted image to be fused and the fusion weight of the reference image; 根据所述调整后待融合图像的融合权重和所述参考图像的融合权重,将所述调整后待融合图像和所述参考图像进行融合,得到HDR融合图像。According to the fusion weight of the adjusted image to be fused and the fusion weight of the reference image, the adjusted image to be fused and the reference image are fused to obtain an HDR fusion image. 10.根据权利要求9所述的夜景高动态范围图像融合方法,其特征在于,所述待融合图像为所述过曝图像时,对应的所述调整后待融合图像的融合权重的获取方式为:10 . The night scene high dynamic range image fusion method according to claim 9 , wherein when the image to be fused is the overexposed image, the corresponding acquisition method of the fusion weight of the adjusted image to be fused is: 10 . :
Figure FDA0003586408810000031
Figure FDA0003586408810000031
式中,w+为过曝图像对应的调整后待融合图像的融合权重,p为调整后待融合图像中像素点的亮度,pt为亮度阈值,s为平滑度。In the formula, w + is the fusion weight of the adjusted image to be fused corresponding to the overexposed image, p is the brightness of the pixels in the adjusted image to be fused, p t is the brightness threshold, and s is the smoothness.
11.根据权利要求9所述的夜景高动态范围图像融合方法,其特征在于,所述待融合图像为所述欠曝图像时,对应的所述调整后待融合图像的融合权重的获取方式为:11 . The night scene high dynamic range image fusion method according to claim 9 , wherein when the image to be fused is the underexposed image, the corresponding acquisition method of the fusion weight of the adjusted image to be fused is: 11 . :
Figure FDA0003586408810000032
Figure FDA0003586408810000032
式中,w-为欠曝图像对应的调整后待融合图像的融合权重,p为调整后待融合图像中像素点的亮度,pt为亮度阈值,s为平滑度。In the formula, w - is the fusion weight of the adjusted image to be fused corresponding to the underexposed image, p is the brightness of the pixels in the adjusted image to be fused, p t is the brightness threshold, and s is the smoothness.
12.根据权利要求1-8中任一所述的夜景高动态范围图像融合方法,其特征在于,所述根据所述欠曝图像或正常曝光图像进行亮度调整后确定参考图像,包括:12. The night scene high dynamic range image fusion method according to any one of claims 1-8, wherein the determining a reference image after performing brightness adjustment according to the underexposed image or the normal exposure image, comprising: 从所述待融合图像中选取一个欠曝图像或正常曝光图像作为待参考图像;Select an underexposed image or a normal exposure image from the to-be-fused images as the to-be-referenced image; 根据所述待参考图像确定所述待参考图像中各个像素点的过曝掩码;Determine the overexposure mask of each pixel in the to-be-referenced image according to the to-be-referenced image; 获取预设的调整参数,根据所述调整参数和所述过曝掩码确定所述待参考图像中各个像素点的亮度调整参数;Acquire preset adjustment parameters, and determine brightness adjustment parameters of each pixel in the to-be-referenced image according to the adjustment parameters and the overexposure mask; 根据所述亮度调整参数调整所述待参考图像中各个像素点的亮度,得到所述参考图像。Adjust the brightness of each pixel in the to-be-referenced image according to the brightness adjustment parameter to obtain the reference image. 13.根据权利要求12所述的夜景高动态范围图像融合方法,其特征在于,所述过曝掩码的获取方式为:13. The night scene high dynamic range image fusion method according to claim 12, wherein the acquisition mode of the overexposure mask is: E=1-min(n,1)E=1-min(n,1)
Figure FDA0003586408810000041
Figure FDA0003586408810000041
式中,E为像素点的过曝掩码,p为待参考图像中像素点的亮度,pt为亮度阈值,s为平滑度,n为中间变量。In the formula, E is the overexposure mask of the pixel point, p is the brightness of the pixel point in the image to be referenced, pt is the brightness threshold, s is the smoothness, and n is the intermediate variable.
14.根据权利要求12所述的夜景高动态范围图像融合方法,其特征在于,所述亮度调整参数的获取方式为:14. The night scene high dynamic range image fusion method according to claim 12, wherein the acquisition method of the brightness adjustment parameter is: a=(A-1)×E+1a=(A-1)×E+1 式中,a为像素点的亮度调整参数,A为调整参数,E为像素点的过曝掩码。In the formula, a is the brightness adjustment parameter of the pixel, A is the adjustment parameter, and E is the overexposure mask of the pixel. 15.一种夜景高动态范围图像融合装置,其特征在于,包括:15. A night scene high dynamic range image fusion device, characterized in that, comprising: 获取单元(1),用于获取多个待融合图像,所述待融合图像至少包括过曝图像、欠曝图像和正常曝光图像中的任意两种;an acquisition unit (1), configured to acquire a plurality of images to be fused, the images to be fused at least include any two of an overexposed image, an underexposed image and a normal exposure image; 确定单元(2),用于根据所述欠曝图像或正常曝光图像进行亮度调整后确定参考图像,包括:确定一个所述欠曝图像或正常曝光图像中各个像素点的过曝掩码,根据预设的调整参数和所述过曝掩码确定各个所述像素点的亮度调整参数,根据所述亮度调整参数调整各个所述像素点的亮度,得到所述参考图像;A determining unit (2), configured to determine a reference image after performing brightness adjustment according to the underexposed image or the normally exposed image, comprising: determining an overexposure mask of each pixel in the underexposed image or the normally exposed image, according to The preset adjustment parameter and the overexposure mask determine the brightness adjustment parameter of each of the pixel points, and adjust the brightness of each of the pixel points according to the brightness adjustment parameter to obtain the reference image; 调整单元(3),用于根据所述参考图像对所述待融合图像的亮度进行调整,得到调整后待融合图像;an adjustment unit (3), configured to adjust the brightness of the to-be-fused image according to the reference image, to obtain the adjusted to-be-fused image; 融合单元(4),用于将所述调整后待融合图像进行融合,得到HDR融合图像。A fusion unit (4), configured to fuse the adjusted images to be fused to obtain an HDR fusion image. 16.一种电子设备,包括处理器以及存储器,其特征在于,所述存储器存储有控制程序,所述控制程序被处理器执行时实现如权利要求1-14中任一所述的夜景高动态范围图像融合方法。16. An electronic device, comprising a processor and a memory, wherein the memory stores a control program, and when the control program is executed by the processor, the high dynamic night scene according to any one of claims 1-14 is realized Range image fusion method. 17.一种计算机可读存储介质,存储有指令,其特征在于,所述指令被处理器加载并执行时实现如权利要求1-14中任一所述的夜景高动态范围图像融合方法。17. A computer-readable storage medium storing instructions, wherein when the instructions are loaded and executed by a processor, the night scene high dynamic range image fusion method according to any one of claims 1-14 is implemented.
CN201911009224.8A 2019-10-23 2019-10-23 Night scene high dynamic range image fusion method and device and electronic equipment Active CN110751608B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911009224.8A CN110751608B (en) 2019-10-23 2019-10-23 Night scene high dynamic range image fusion method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911009224.8A CN110751608B (en) 2019-10-23 2019-10-23 Night scene high dynamic range image fusion method and device and electronic equipment

Publications (2)

Publication Number Publication Date
CN110751608A CN110751608A (en) 2020-02-04
CN110751608B true CN110751608B (en) 2022-08-16

Family

ID=69279411

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911009224.8A Active CN110751608B (en) 2019-10-23 2019-10-23 Night scene high dynamic range image fusion method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN110751608B (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110611750B (en) * 2019-10-31 2022-03-22 北京迈格威科技有限公司 A method, device and electronic device for generating a high dynamic range image of night scene
CN111372001B (en) * 2020-03-17 2021-09-03 捷开通讯(深圳)有限公司 Image fusion method and device, storage medium and mobile terminal
CN111311532B (en) * 2020-03-26 2022-11-11 深圳市商汤科技有限公司 Image processing method and device, electronic device and storage medium
CN115552457A (en) * 2020-05-06 2022-12-30 Oppo广东移动通信有限公司 Method and apparatus for noise reduction, and computer usable medium storing software for implementing the method
CN113962844A (en) * 2020-07-20 2022-01-21 武汉Tcl集团工业研究院有限公司 Image fusion method, storage medium and terminal device
CN112614083A (en) * 2020-12-18 2021-04-06 北京迈格威科技有限公司 Image fusion method and device and electronic system
CN112651918B (en) * 2021-01-15 2024-11-22 北京小米松果电子有限公司 Image processing method and device, electronic device, and storage medium
CN113781370B (en) * 2021-08-19 2024-09-13 北京旷视科技有限公司 Image enhancement method and device and electronic equipment
CN117061841B (en) * 2023-06-12 2024-06-25 深圳市博盛医疗科技有限公司 Dual-wafer endoscope imaging method and imaging device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105894449A (en) * 2015-11-11 2016-08-24 乐卡汽车智能科技(北京)有限公司 Method and system for overcoming abrupt color change in image fusion processes
CN105976325A (en) * 2016-06-29 2016-09-28 上海小蚁科技有限公司 Method for adjusting brightness of multiple images
CN108205796A (en) * 2016-12-16 2018-06-26 大唐电信科技股份有限公司 A kind of fusion method and device of more exposure images
WO2018136373A1 (en) * 2017-01-20 2018-07-26 Microsoft Technology Licensing, Llc Image fusion and hdr imaging
CN109767413A (en) * 2019-01-11 2019-05-17 深圳岚锋创视网络科技有限公司 A kind of the HDR method, apparatus and portable terminal of anti-motion artifacts
CN110166709A (en) * 2019-06-13 2019-08-23 Oppo广东移动通信有限公司 Night scene image processing method, device, electronic equipment and storage medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10706512B2 (en) * 2017-03-07 2020-07-07 Adobe Inc. Preserving color in image brightness adjustment for exposure fusion

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105894449A (en) * 2015-11-11 2016-08-24 乐卡汽车智能科技(北京)有限公司 Method and system for overcoming abrupt color change in image fusion processes
CN105976325A (en) * 2016-06-29 2016-09-28 上海小蚁科技有限公司 Method for adjusting brightness of multiple images
CN108205796A (en) * 2016-12-16 2018-06-26 大唐电信科技股份有限公司 A kind of fusion method and device of more exposure images
WO2018136373A1 (en) * 2017-01-20 2018-07-26 Microsoft Technology Licensing, Llc Image fusion and hdr imaging
CN109767413A (en) * 2019-01-11 2019-05-17 深圳岚锋创视网络科技有限公司 A kind of the HDR method, apparatus and portable terminal of anti-motion artifacts
CN110166709A (en) * 2019-06-13 2019-08-23 Oppo广东移动通信有限公司 Night scene image processing method, device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN110751608A (en) 2020-02-04

Similar Documents

Publication Publication Date Title
CN110751608B (en) Night scene high dynamic range image fusion method and device and electronic equipment
WO2021082580A1 (en) Night scene high dynamic range image generation method, device, and electronic apparatus
US11765466B2 (en) Imaging control method and apparatus, electronic device, and computer readable storage medium
US8077218B2 (en) Methods and apparatuses for image processing
CN109218628B (en) Image processing method, device, electronic device and storage medium
WO2020034735A1 (en) Imaging control method and electronic device
WO2020034737A1 (en) Imaging control method, apparatus, electronic device, and computer-readable storage medium
US10410327B2 (en) Shallow depth of field rendering
US11715184B2 (en) Backwards-compatible high dynamic range (HDR) images
JP2021530911A (en) Night view photography methods, devices, electronic devices and storage media
CN109348089A (en) Night scene image processing method, device, electronic equipment and storage medium
CN111028190A (en) Image processing method, image processing device, storage medium and electronic equipment
CN112565636B (en) Image processing method, device, equipment and storage medium
AU2013206601A1 (en) Variable blend width compositing
EP3836532A1 (en) Control method and apparatus, electronic device, and computer readable storage medium
CN112598609A (en) Dynamic image processing method and device
CN114359021A (en) Processing method and device for rendered picture, electronic equipment and medium
WO2021258300A1 (en) In-screen photography control method, terminal device and storage medium
CN111915529A (en) A video dark light enhancement method, device, mobile terminal and storage medium
CN112446848B (en) Image processing method and device and electronic equipment
CN110874816B (en) Image processing method, device, mobile terminal and storage medium
CN110166710A (en) Image composition method, device, equipment and medium
CN117710264B (en) Dynamic range calibration method of image and electronic equipment
CN119211739A (en) Image processing method, device, medium, and electronic device
CN117876237A (en) Image processing method, apparatus, electronic device, and computer-readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20241212

Address after: No. 257, 2nd Floor, Building 9, No. 2 Huizhu Road, Liangjiang New District, Yubei District, Chongqing 401120

Patentee after: Force Map New (Chongqing) Technology Co.,Ltd.

Country or region after: China

Address before: 316-318, block a, Rongke Information Center, No.2, South Road, Academy of Sciences, Haidian District, Beijing, 100190

Patentee before: MEGVII (BEIJING) TECHNOLOGY Co.,Ltd.

Country or region before: China

TR01 Transfer of patent right