DICNet: achieve low-light image enhancement with image decomposition, illumination enhancement, and color restoration
Abstract
Low-light image enhancement (LLIE) is mainly used to restore image degradation caused by environmental noise, lighting effects, and other factors. Despite many relevant works combating environmental interference, LLIE currently still faces multiple limitations, such as noise, unnatural color recovery, and severe loss of details, etc. To effectively overcome these limitations, we propose a DICNet based on the Retinex theory. DICNet consists of three components: image decomposition, illumination enhancement, and color restoration. To avoid the influence of noise during the enhancement process, we use feature maps after the image high-frequency component denoising process to guide image decomposition and suppress noise interference. For illumination enhancement, we propose a feature separation method that considering the influence of different lighting intensities and preserves details. In addition, to address the insufficient high-low-level feature fusion of the U-Net used in color restoration, we design a Feature Cross-Fusion Module and propose a feature fusion connection plug-in to ensure natural and realistic color restoration. Based on a large number of experiments on publicly available datasets, our method outperforms existing state-of-the-art methods in both performance and visual quality.
References
[1]
Pisano ED, Zong S, Hemminger BM, DeLuca M, Johnston RE, Muller K, Braeuning MP, and Pizer SM Contrast limited adaptive histogram equalization image processing to improve the detection of simulated spiculations in dense mammograms J. Digit. Imaging 1998 11 193-200
[2]
Farid H Blind inverse gamma correction IEEE Trans. Image Process. 2001 10 10 1428-1433
[3]
Jobson DJ, Rahman Z-U, and Woodell GA Properties and performance of a center/surround Retinex IEEE Trans. Image Process. 1997 6 3 451-462
[4]
Rahman, Z., Jobson, D.J., Woodell, G.A.: Multi-scale Retinex for color image enhancement. In: Proceedings of 3rd IEEE International Conference on Image Processing, vol. 3, pp. 1003–1006. IEEE (1996).
[5]
Jobson DJ, Rahman Z-U, and Woodell GA A multiscale Retinex for bridging the gap between color images and the human observation of scenes IEEE Trans. Image Process. 1997 6 7 965-976
[6]
Wei, C., Wang, W., Yang, W., Liu, J.: Deep Retinex decomposition for low-light enhancement (2018). arXiv:1808.04560.
[7]
Guo, C., Li, C., Guo, J., Loy, C.C., Hou, J., Kwong, S., Cong, R.: Zero-reference deep curve estimation for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1780–1789 (2020).
[8]
Zhang, Y., Zhang, J., Guo, X.: Kindling the darkness: a practical low-light image enhancer. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 1632–1640 (2019).
[9]
Land EH The Retinex theory of color vision Sci. Am. 1977 237 6 108-129 arXiv:jstor:2495.3876
[10]
Guo X, Li Y, and Ling H Lime: low-light image enhancement via illumination map estimation IEEE Trans. Image Process. 2016 26 2 982-993
[11]
Fu, X., Zeng, D., Huang, Y., Zhang, X.-P., Ding, X.: A weighted variational model for simultaneous reflectance and illumination estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2782–2790 (2016).
[12]
Cai J, Gu S, and Zhang L Learning a deep single image contrast enhancer from multi-exposure images IEEE Trans. Image Process. 2018 27 4 2049-2062
[13]
Lore KG, Akintayo A, and Sarkar S Llnet: a deep autoencoder approach to natural low-light image enhancement Pattern Recogn. 2017 61 650-662
[14]
Chen, C., Chen, Q., Xu, J., Koltun, V.: Learning to see in the dark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3291–3300 (2018).
[15]
Jiang Y, Gong X, Liu D, Cheng Y, Fang C, Shen X, Yang J, Zhou P, and Wang Z Enlightengan: deep light enhancement without paired supervision IEEE Trans. Image Process. 2021 30 2340-2349
[16]
Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: Cbam: Convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018).
[17]
Ibrahim H and Kong NSP Brightness preserving dynamic histogram equalization for image contrast enhancement IEEE Trans. Consum. Electron. 2007 53 4 1752-1758
[18]
Wang C and Ye Z Brightness preserving histogram equalization with maximum entropy: a variational perspective IEEE Trans. Consum. Electron. 2005 51 4 1326-1334
[19]
Chen S-D and Ramli AR Minimum mean brightness error bi-histogram equalization in contrast enhancement IEEE Trans. Consum. Electron. 2003 49 4 1310-1319
[20]
Reza AM Realization of the contrast limited adaptive histogram equalization (clahe) for real-time image enhancement J. VLSI Signal Process. Syst. Signal Image Video Technol. 2004 38 35-44
[21]
Beghdadi A and Le Negrate A Contrast enhancement technique based on local detection of edges Comput. Vis. Graph. Image Process. 1989 46 2 162-174
[22]
Peli E Contrast in complex images JOSA A 1990 7 10 2032-2040
[23]
Wang C, Zhang H, and Liu L Total generalized variation-based Retinex image decomposition Vis. Comput. 2021 37 1 77-93
[24]
Shen, L., Yue, Z., Feng, F., Chen, Q., Liu, S., Ma, J.: Msr-net: low-light image enhancement using deep convolutional network (2017). arXiv:1711.02488.
[25]
Dabov K, Foi A, Katkovnik V, and Egiazarian K Image denoising by sparse 3-d transform-domain collaborative filtering IEEE Trans. Image Process. 2007 16 8 2080-2095
[26]
Wang, R., Zhang, Q., Fu, C.-W., Shen, X., Zheng, W.-S., Jia, J.: Underexposed photo enhancement using deep illumination estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6849–6857 (2019).
[27]
Wu, W., Weng, J., Zhang, P., Wang, X., Yang, W., Jiang, J.: Uretinex-net: Retinex-based deep unfolding network for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5901–5910 (2022).
[28]
Yu X, Li H, and Yang H Two-stage image decomposition and color regulator for low-light image enhancement Vis. Comput. 2023 39 9 4165-4175
[29]
Guo S, Wang W, Wang X, and Xu X Low-light image enhancement with joint illumination and noise data distribution transformation Vis. Comput. 2023 39 4 1363-1374
[30]
Zhang, Z., Zheng, H., Hong, R., Xu, M., Yan, S., Wang, M.: Deep color consistent network for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1899–1908 (2022).
[31]
Kim, H., Choi, S.-M., Kim, C.-S., Koh, Y.J.: Representative color transform for image enhancement. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4459–4468 (2021).
[32]
Hai J, Xuan Z, Yang R, Hao Y, Zou F, Lin F, and Han S R2rnet: low-light image enhancement via real-low to real-normal network J. Vis. Commun. Image Represent. 2023 90 103712
[33]
Zhou F, Sun X, Dong J, and Zhu XX Surroundnet: towards effective low-light image enhancement Pattern Recogn. 2023
[34]
Hinton GE and Salakhutdinov RR Reducing the dimensionality of data with neural networks Science 2006 313 5786 504-507
[35]
Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol P-A, and Bottou L Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion J. Mach. Learn. Res. 2010
[36]
Hek M and Sunj TXO Single image haze removal using dark channel prior IEEE Trans. Pattern Anal. Mach. Intell. 2011 33 12 2341
[37]
Li C, Guo C, Han L, Jiang J, Cheng M-M, Gu J, and Loy CC Low-light image and video enhancement using deep learning: a survey IEEE Trans. Pattern Anal. Mach. Intell. 2021 44 12 9396-9416
[38]
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016).
[39]
Maas, A.L., Hannun, A.Y., Ng, A.Y., et al.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of Icml, vol. 30, p. 3. Atlanta, Georgia, USA (2013)
[40]
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017).
[41]
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Computer Vision—ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part II 14, pp. 694–711. Springer (2016).
[42]
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5–9, 2015, Proceedings, Part III 18, pp. 234–241. Springer (2015).
[43]
Zhang, R., Isola, P., Efros, A.A.: Colorful image colorization. In: Computer Vision—ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part III 14, pp. 649–666. Springer (2016).
[44]
Baldassarre, F., Morín, D.G., Rodés-Guirao, L.: Deep koalarization: image colorization using cnns and inception-resnet-v2 (2017). arXiv:1712.03400.
[45]
Dudhane, A., Zamir, S.W., Khan, S., Khan, F.S., Yang, M.-H.: Burst image restoration and enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5759–5768 (2022).
[46]
Wang Z, Bovik AC, Sheikh HR, and Simoncelli EP Image quality assessment: from error visibility to structural similarity IEEE Trans. Image Process. 2004 13 4 600-612
[47]
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization (2014). arXiv:1412.6980
[48]
Liu, R., Ma, L., Zhang, J., Fan, X., Luo, Z.: Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10561–10570 (2021).
[49]
Jin, Y., Yang, W., Tan, R.T.: Unsupervised night image enhancement: when layer decomposition meets light-effects suppression. In: Computer Vision—ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXVII, pp. 404–421. Springer (2022).
[50]
Ma, L., Ma, T., Liu, R., Fan, X., Luo, Z.: Toward fast, flexible, and robust low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5637–5646 (2022).
[51]
Wang, H., Xu, K., Lau, R.W.: Local color distributions prior for image enhancement. In: Computer Vision—ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XVIII, pp. 343–359. Springer (2022).
[52]
Wen, J., Wu, C., Zhang, T., Yu, Y., Swierczynski, P.: Self-reference deep adaptive curve estimation for low-light image enhancement (2023). arXiv:2308.08197
[53]
Wang, C., Wu, H., Jin, Z.: Fourllie: Boosting low-light image enhancement by Fourier frequency information. In: Proceedings of the 31st ACM International Conference on Multimedia, pp. 7459–7469 (2023)
[54]
Fu, Z., Yang, Y., Tu, X., Huang, Y., Ding, X., Ma, K.-K.: Learning a simple low-light image enhancer from paired low-light instances. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22252–22261 (2023)
[55]
Nguyen, H., Tran, D., Nguyen, K., Nguyen, R.: Psenet: progressive self-enhancement network for unsupervised extreme-light image enhancement. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1756–1765 (2023).
[56]
Mittal A, Soundararajan R, and Bovik AC Making a completely blind image quality analyzer IEEE Signal Process. Lett. 2012 20 3 209-212
[57]
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–595 (2018).
[58]
Yang W, Wang W, Huang H, Wang S, and Liu J Sparse gradient regularized deep Retinex network for robust low-light image enhancement IEEE Trans. Image Process. 2021 30 2072-2086
Index Terms
- DICNet: achieve low-light image enhancement with image decomposition, illumination enhancement, and color restoration
Index terms have been assigned to the content through auto-classification.
Recommendations
A deep Retinex network for underwater low-light image enhancement
AbstractUnderwater images suffer from color cast and low contrast due to the light absorption and scattering. Especially when natural light is not sufficient, large dark areas appear in the captured image, making it impossible to understand the image ...
Low-light image enhancement based on sharpening-smoothing image filter
AbstractLow-light images suffer from poor visibility, severe noise, low contrast, and low brightness. To overcome these issues, many image enhancement methods have been proposed. Few techniques solve these problems simultaneously. This paper presents a ...
Low-light image enhancement based on multi-illumination estimation
AbstractImages captured by cameras in low-light conditions have low quality and appear dark due to insufficient light exposure, which critically affects the view. Most of the traditional enhancement methods are based on the entire image for exposure ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2024. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 22 February 2024
Accepted: 02 January 2024
Author Tags
Qualifiers
- Research-article
Funding Sources
- the Natural Science Foundation of Heilongjiang Province
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 22 Dec 2024