research-article

Underwater Image Enhancement via Minimal Color Loss and Locally Adaptive Contrast Enhancement

Authors:

Peixian Zhuang,

Chongyi LiAuthors Info & Claims

IEEE Transactions on Image Processing, Volume 31

Pages 3997 - 4010

https://doi.org/10.1109/TIP.2022.3177129

Published: 01 January 2022 Publication History

Abstract

Underwater images typically suffer from color deviations and low visibility due to the wavelength-dependent light absorption and scattering. To deal with these degradation issues, we propose an efficient and robust underwater image enhancement method, called MLLE. Specifically, we first locally adjust the color and details of an input image according to a minimum color loss principle and a maximum attenuation map-guided fusion strategy. Afterward, we employ the integral and squared integral maps to compute the mean and variance of local image blocks, which are used to adaptively adjust the contrast of the input image. Meanwhile, a color balance strategy is introduced to balance the color differences between channel a and channel b in the CIELAB color space. Our enhanced results are characterized by vivid color, improved contrast, and enhanced details. Extensive experiments on three underwater image enhancement datasets demonstrate that our method outperforms the state-of-the-art methods. Our method is also appealing in its fast processing speed within 1s for processing an image of size <inline-formula> <tex-math notation="LaTeX">$1024\times 1024 \times 3$ </tex-math></inline-formula> on a single CPU. Experiments further suggest that our method can effectively improve the performance of underwater image segmentation, keypoint detection, and saliency detection. The project page is available at <monospace><uri>https://li-chongyi.github.io/proj</uri></monospace>

References

[1]

X. Chen, J. Yu, S. Kong, Z. Wu, X. Fang, and L. Wen, “Towards real-time advancement of underwater visual quality with GAN,” IEEE Trans. Ind. Electron., vol. 66, no. 12, pp. 9350–9359, Dec. 2019.

[2]

Y. Wanget al., “Real-time underwater onboard vision sensing system for robotic gripping,” IEEE Trans. Instrum. Meas., vol. 70, pp. 1–11, 2021.

[3]

L. Jianget al., “Underwater species detection using channel sharpening attention,” in Proc. 29th ACM Int. Conf. Multimedia, Oct. 2021, pp. 4259–4267.

[4]

C. Li, J. Guo, and C. Guo, “Emerging from water: Underwater image color correction based on weakly supervised color transfer,” IEEE Signal Process. Lett., vol. 25, no. 3, pp. 323–327, Mar. 2018.

[5]

W. Zhang, Y. Wang, and C. Li, “Underwater image enhancement by attenuated color channel correction and detail preserved contrast enhancement,” IEEE J. Ocean. Eng., early access, Mar. 29, 2022. 10.1109/JOE.2022.3140563.

[6]

R. Liu, X. Fan, M. Zhu, M. Hou, and Z. Luo, “Real-world underwater enhancement: Challenges, benchmarks, and solutions under natural light,” IEEE Trans. Circuits Syst. Video Technol., vol. 30, no. 12, pp. 4861–4875, Dec. 2020.

Digital Library

[7]

C. Liet al., “An underwater image enhancement benchmark dataset and beyond,” IEEE Trans. Image Process., vol. 29, pp. 4376–4389, 2020.

[8]

X. Fu and X. Cao, “Underwater image enhancement with global–local networks and compressed-histogram equalization,” Signal Process., Image Commun., vol. 86, Aug. 2020, Art. no.

[9]

Y. Guo, H. Li, and P. Zhuang, “Underwater image enhancement using a multiscale dense generative adversarial network,” IEEE J. Ocean. Eng., vol. 45, no. 3, pp. 862–870, Jul. 2020.

[10]

T. Treibitz and Y. Y. Schechner, “Active polarization descattering,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 3, pp. 385–399, Mar. 2009.

Digital Library

[11]

Z. Murez, T. Treibitz, R. Ramamoorthi, and D. J. Kriegman, “Photometric stereo in a scattering medium,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 9, pp. 1880–1891, Sep. 2017.

Digital Library

[12]

Y. Zhou, Q. Wu, K. Yan, L. Feng, and W. Xiang, “Underwater image restoration using color-line model,” IEEE Trans. Circuits Syst. Video Technol., vol. 29, no. 3, pp. 907–911, Mar. 2019.

Digital Library

[13]

Z. Liang, X. Ding, Y. Wang, X. Yan, and X. Fu, “GUDCP: Generalization of underwater dark channel prior for underwater image restoration,” IEEE Trans. Circuits Syst. Video Technol., early access, Sep. 20, 2021. 10.1109/TCSVT.2021.3114230.

[14]

W. Zhang, L. Dong, T. Zhang, and W. Xu, “Enhancing underwater image via color correction and bi-interval contrast enhancement,” Signal Process., Image Commun., vol. 90, Jan. 2021, Art. no.

[15]

W. Zhang, L. Dong, and W. Xu, “Retinex-inspired color correction and detail preserved fusion for underwater image enhancement,” Comput. Electron. Agricult., vol. 192, Jan. 2022, Art. no.

[16]

X. Yeet al., “Deep joint depth estimation and color correction from monocular underwater images based on unsupervised adaptation networks,” IEEE Trans. Circuits Syst. Video Technol., vol. 30, no. 11, pp. 3995–4008, Nov. 2020.

[17]

Y. Wang, J. Guo, H. Gao, and H. Yue, “UIEC²-Net: CNN-based underwater image enhancement using two color space,” Signal Process., Image Commun., vol. 96, Aug. 2021, Art. no.

[18]

P. Liu, C. Zhang, H. Qi, G. Wang, and H. Zheng, “Multi-attention DenseNet: A scattering medium imaging optimization framework for visual data pre-processing of autonomous driving systems,” IEEE Trans. Intell. Transp. Syst., early access, Feb. 1, 2022. 10.1109/TITS.2022.3145815.

[19]

J. S. Jaffe, “Computer modeling and the design of optimal underwater imaging systems,” IEEE J. Ocean. Eng., vol. 15, no. 2, pp. 101–111, Apr. 1990.

[20]

C. Li, J. Guo, C. Guo, R. Cong, and J. Gong, “A hybrid method for underwater image correction,” Pattern Recognit. Lett., vol. 94, pp. 62–67, Jul. 2017.

Digital Library

[21]

X. Xue, Z. Hao, L. Ma, Y. Wang, and R. Liu, “Joint luminance and chrominance learning for underwater image enhancement,” IEEE Signal Process. Lett., vol. 28, pp. 818–822, 2021.

[22]

C. Tan, G. Seet, A. Sluzek, and D. He, “A novel application of range-gated underwater laser imaging system (ULIS) in near-target turbid medium,” Opt. Lasers Eng., vol. 43, no. 9, pp. 995–1009, 2005.

[23]

Y. Zhao, W. He, H. Ren, Y. Li, and Y. Fu, “Polarization descattering imaging through turbid water without prior knowledge,” Opt. Lasers Eng., vol. 148, Jan. 2022, Art. no.

[24]

J. Li and Y. Li, “Underwater image restoration algorithm for free-ascending deep-sea tripods,” Opt. Laser Technol., vol. 110, pp. 129–134, Feb. 2019.

[25]

J. Y. Chiang and Y.-C. Chen, “Underwater image enhancement by wavelength compensation and dehazing,” IEEE Trans. Image Process., vol. 21, no. 4, pp. 1756–1769, Apr. 2012.

Digital Library

[26]

P. Drews-Jr, E. R. Nascimento, S. S. C. Botelho, and M. F. M. Campos, “Underwater depth estimation and image restoration based on single images,” IEEE Comput. Graph. Appl., vol. 36, no. 2, pp. 24–35, Mar./Apr. 2016.

Digital Library

[27]

C.-Y. Li, J.-C. Guo, R.-M. Cong, Y.-W. Pang, and B. Wang, “Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior,” IEEE Trans. Image Process., vol. 25, no. 12, pp. 5664–5677, Dec. 2016.

Digital Library

[28]

Y.-T. Peng and P. C. Cosman, “Underwater image restoration based on image blurriness and light absorption,” IEEE Trans. Image Process., vol. 26, no. 4, pp. 1579–1594, Apr. 2017.

Digital Library

[29]

Y. Wang, H. Liu, and L.-P. Chau, “Single underwater image restoration using adaptive attenuation-curve prior,” IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 65, no. 3, pp. 992–1002, Mar. 2018.

[30]

Y.-T. Peng, K. Cao, and P. C. Cosman, “Generalization of the dark channel prior for single image restoration,” IEEE Trans. Image Process., vol. 27, no. 6, pp. 2856–2868, Jun. 2018.

[31]

D. Berman, D. Levy, S. Avidan, and T. Treibitz, “Underwater single image color restoration using haze-lines and a new quantitative dataset,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 8, pp. 2822–2837, Aug. 2021.

[32]

D. Akkaynak and T. Treibitz, “Sea-thru: A method for removing water from underwater images,” in Proc. IEEE/CVF CVPR, Jun. 2019, pp. 1682–1691.

[33]

R. Liu, L. Ma, J. Zhang, X. Fan, and Z. Luo, “Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 10556–10565.

[34]

P. Zhuang, C. Li, and J. Wu, “Bayesian retinex underwater image enhancement,” Eng. Appl. Artif. Intell., vol. 101, May 2021, Art. no.

[35]

L. Dong, W. Zhang, and W. Xu, “Underwater image enhancement via integrated RGB and LAB color models,” Signal Process., Image Commun., vol. 104, May 2022, Art. no.

[36]

A. S. A. Ghani and N. A. M. Isa, “Automatic system for improving underwater image contrast and color through recursive adaptive histogram modification,” Comput. Electron. Agric., vol. 141, pp. 181–195, Sep. 2017.

Digital Library

[37]

K. Z. M. Azmi, A. S. A. Ghani, Z. M. Yusof, and Z. Ibrahim, “Natural-based underwater image color enhancement through fusion of swarm-intelligence algorithm,” Appl. Soft Comput., vol. 85, Dec. 2019, Art. no.

[38]

W. Zhang, X. Pan, X. Xie, L. Li, Z. Wang, and C. Han, “Color correction and adaptive contrast enhancement for underwater image enhancement,” Comput. Electr. Eng., vol. 91, May 2021, Art. no.

[39]

C. Ancuti, C. O. Ancuti, T. Haber, and P. Bekaert, “Enhancing underwater images and videos by fusion,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2012, pp. 81–88.

[40]

C. O. Ancuti, C. Ancuti, C. D. Vleeschouwer, and P. Bekaert, “Color balance and fusion for underwater image enhancement,” IEEE Trans. Image Process., vol. 27, no. 1, pp. 379–393, Jan. 2017.

[41]

P. Guo, D. Zeng, Y. Tian, S. Liu, H. Liu, and D. Li, “Multi-scale enhancement fusion for underwater sea cucumber images based on human visual system modelling,” Comput. Electron. Agricult., vol. 175, Aug. 2020, Art. no.

[42]

S.-B. Gao, M. Zhang, Q. Zhao, X.-S. Zhang, and Y.-J. Li, “Underwater image enhancement using adaptive retinal mechanisms,” IEEE Trans. Image Process., vol. 28, no. 11, pp. 5580–5595, Nov. 2019.

Digital Library

[43]

J. Yuan, W. Cao, Z. Cai, and B. Su, “An underwater image vision enhancement algorithm based on contour bougie morphology,” IEEE Trans. Geosci. Remote Sens., vol. 59, no. 10, pp. 8117–8128, Oct. 2021.

[44]

X. Fu, B. Liang, Y. Huang, X. Ding, and J. Paisley, “Lightweight pyramid networks for image deraining,” IEEE Trans. Neural Netw. Learn. Syst., vol. 31, no. 6, pp. 1794–1807, Jun. 2020.

[45]

C. Li, C. Guo, and C. L. Chen, “Learning to enhance low-light image via zero-reference deep curve estimation,” IEEE Trans. Pattern Anal. Mach. Intell., early access, Mar. 3, 2021. 10.1109/TPAMI.2021.3063604.

Digital Library

[46]

C. Li, C. Guo, J. Guo, P. Han, H. Fu, and R. Cong, “PDR-Net: Perception-inspired single image dehazing network with refinement,” IEEE Trans. Multimedia, vol. 22, no. 3, pp. 704–716, Mar. 2020.

[47]

X. Fu, W. Wang, Y. Huang, X. Ding, and J. Paisley, “Deep multiscale detail networks for multiband spectral image sharpening,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 5, pp. 2090–2104, May 2021.

[48]

C. Guo, C. Li, J. Guo, R. Cong, H. Fu, and P. Han, “Hierarchical features driven residual learning for depth map super-resolution,” IEEE Trans. Image Process., vol. 28, no. 5, pp. 2545–2557, May 2019.

Digital Library

[49]

J. Li, K. A. Skinner, R. M. Eustice, and M. Johnson-Roberson, “WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images,” IEEE Robot. Autom. Lett., vol. 3, no. 1, pp. 387–394, Jan. 2018.

[50]

M. J. Islam, Y. Xia, and J. Sattar, “Fast underwater image enhancement for improved visual perception,” IEEE Robot. Autom. Lett., vol. 5, no. 2, pp. 3227–3234, Apr. 2020.

[51]

C. Li, S. Anwar, and F. Porikli, “Underwater scene prior inspired deep underwater image and video enhancement,” Pattern Recognit., vol. 98, Feb. 2020, Art. no.

Digital Library

[52]

C. Li, S. Anwar, J. Hou, R. Cong, C. Guo, and W. Ren, “Underwater image enhancement via medium transmission-guided multi-color space embedding,” IEEE Trans. Image Process., vol. 30, pp. 4985–5000, 2021.

[53]

S. Anwar and C. Li, “Diving deeper into underwater image enhancement: A survey,” Signal Process., Image Commun., vol. 89, Nov. 2020, Art. no.

[54]

C. O. Ancuti, C. Ancuti, C. D. Vleeschouwer, and R. Garcia, “Locally adaptive color correction for underwater image dehazing and matching,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Jul. 2017, pp. 997–1005.

[55]

C. O. Ancuti, C. Ancuti, C. D. Vleeschouwer, and M. Sbetr, “Color channel transfer for image dehazing,” IEEE Signal Process. Lett., vol. 26, no. 9, pp. 1413–1417, Sep. 2019.

[56]

C. O. Ancuti, C. Ancuti, C. D. Vleeschouwer, and M. Sbert, “Color channel compensation (3C): A fundamental pre-processing step for image enhancement,” IEEE Trans. Image Process., vol. 29, pp. 2653–2665, 2019.

[57]

Z. Liang, Y. Wang, X. Ding, Z. Mi, and X. Fu, “Single underwater image enhancement by attenuation map guided color correction and detail preserved dehazing,” Neurocomputing, vol. 425, pp. 160–172, Feb. 2021.

[58]

E. H. Land, “The Retinex theory of color vision,” Sci. Amer., vol. 237, no. 6, pp. 108–128, Dec. 1977.

[59]

K. He, J. Sun, and X. Tang, “Guided image filtering,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 6, pp. 1397–1409, Jun. 2013.

Digital Library

[60]

S. Wang, K. Ma, H. Yeganeh, Z. Wang, and W. Lin, “A patch-structure representation method for quality assessment of contrast changed images,” IEEE Signal Process. Lett., vol. 22, no. 12, pp. 2387–2390, Dec. 2015.

[61]

K. Panetta, C. Gao, and S. Agaian, “Human-visual-system-inspired underwater image quality measures,” IEEE J. Ocean. Eng., vol. 41, no. 3, pp. 541–551, Jul. 2016.

[62]

Y. Wanget al., “An imaging-inspired no-reference underwater color image quality assessment metric,” Comput. Elect. Eng., vol. 70, pp. 904–913, Aug. 2017.

[63]

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis., vol. 60, pp. 91–110, Dec. 2004.

Digital Library

[64]

C. Yang, L. Zhang, H. Lu, X. Ruan, and M.-H. Yang, “Saliency detection via graph-based manifold ranking,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 3166–3173.

[65]

R. Liu, L. Ma, J. Zhang, X. Fan, and Z. Luo, “Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 10556–10565.

[66]

X. Fu, Z. Lin, Y. Huang, and X. Ding, “A variational pan-sharpening with local gradient constraints,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 10257–10266.

[67]

C. Ancuti, C. O. Ancuti, C. D. Vleeschouwer, and A. C. Bovik, “Day and night-time dehazing by local airlight estimation,” IEEE Trans. Image Process., vol. 29, pp. 6264–6275, 2020.

Cited By

An SXu LDeng Z(2025)FastUNetPattern Recognition10.1016/j.patcog.2024.110928157:COnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.patcog.2024.110928
Ajith SM G SSamanta RBenoj SAgarwal P(2024)Revitalising Ocean Ecosystems: A Comprehensive Approach to Sustainable Underwater Trash Analysis Using AUVs,Image Enhancement,YOLO Detection,and Material Circularity IndexProceedings of the 2024 9th International Conference on Multimedia and Image Processing10.1145/3665026.3665034(52-58)Online publication date: 20-Apr-2024
https://dl.acm.org/doi/10.1145/3665026.3665034
Quan YTan XHuang YXu YJi HCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Enhancing Underwater Images via Asymmetric Multi-Scale Invertible NetworksProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681098(6182-6191)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681098
Show More Cited By

Recommendations

Underwater image enhancement by color correction and color constancy via Retinex for detail preserving
Abstract
In underwater, light attenuation causes non-uniform illumination that degrades underwater image. To enhance the degraded image, we propose an underwater image enhancement method that includes color correction, color constancy, multi-...
Graphical abstract

Display Omitted
Highlights
- The formulation of color correction compensates the red and blue channels by masking.
Underwater image enhancement via variable contrast and saturation enhancement model
Abstract
Since underwater optical imaging is affected by underwater light absorption and attenuation, the pixels of images distribute unevenly, and image structures are damaged to different degrees, leading to problems such as color distortion, low ...
Contrast Enhancement of RGB Color Images by Histogram Equalization of Color Vectors’ Intensities
Intelligent Computing Methodologies
Abstract
The histogram equalization (HE) is a technique developed for image contrast enhancement of grayscale images. For RGB (Red, Green, Blue) color images, the HE is usually applied in the color channels separately; due to correlation between the color ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Image Processing

IEEE Transactions on Image Processing Volume 31, Issue

2022

3518 pages

ISSN:1057-7149

Issue’s Table of Contents

1941-0042 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Press

Publication History

Published: 01 January 2022

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

73
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

An SXu LDeng Z(2025)FastUNetPattern Recognition10.1016/j.patcog.2024.110928157:COnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.patcog.2024.110928
Ajith SM G SSamanta RBenoj SAgarwal P(2024)Revitalising Ocean Ecosystems: A Comprehensive Approach to Sustainable Underwater Trash Analysis Using AUVs,Image Enhancement,YOLO Detection,and Material Circularity IndexProceedings of the 2024 9th International Conference on Multimedia and Image Processing10.1145/3665026.3665034(52-58)Online publication date: 20-Apr-2024
https://dl.acm.org/doi/10.1145/3665026.3665034
Quan YTan XHuang YXu YJi HCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Enhancing Underwater Images via Asymmetric Multi-Scale Invertible NetworksProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681098(6182-6191)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681098
Bogomasov KGeuer TConrad S(2024)Let's Talk About Image Quality Improvement in Underwater Images for Object DetectionProceedings of the 2024 7th International Conference on Machine Vision and Applications10.1145/3653946.3653947(1-6)Online publication date: 12-Mar-2024
https://dl.acm.org/doi/10.1145/3653946.3653947
Lin PWang YLi YFan ZFu X(2024)Underwater Color Correction Network With Knowledge TransferIEEE Transactions on Multimedia10.1109/TMM.2024.337459826(8088-8103)Online publication date: 12-Mar-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3374598
Jiang QKang YWang ZRen WLi C(2024)Perception-Driven Deep Underwater Image Enhancement Without Paired SupervisionIEEE Transactions on Multimedia10.1109/TMM.2023.332761326(4884-4897)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3327613
Chen LXie YLi YXu QDong J(2024)CWSCNet: Channel-Weighted Skip Connection Network for Underwater Object DetectionIEEE Transactions on Image Processing10.1109/TIP.2024.345724633(5206-5218)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2024.3457246
Qiao NSun CDong LGe Q(2024)Semi-Supervised Feature Distillation and Unsupervised Domain Adversarial Distillation for Underwater Image EnhancementIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.337825234:8(7671-7682)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3378252
Liang ZZhang WRuan RZhuang PXie XLi C(2024)Underwater Image Quality Improvement via Color, Detail, and Contrast RestorationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.329752434:3(1726-1742)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1109/TCSVT.2023.3297524
Hou GLi NZhuang PLi KSun HLi C(2024)Non-Uniform Illumination Underwater Image Restoration via Illumination Channel Sparsity PriorIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.329036334:2(799-814)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1109/TCSVT.2023.3290363
Show More Cited By

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents