Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

MFAF-Net: image dehazing with multi-level features and adaptive fusion

Published: 13 July 2023 Publication History

Abstract

Hazy images often cause blurring, detail loss and color distortion, which makes it difficult to address the other visual tasks such as tracking, classification and object detection. In recent years, significant advances have been made in image dehazing task, dominated by convolutional neural networks (CNNs). Most existing CNNs methods tend to estimate the transmission map and atmospheric light and then recover the haze-free image based on atmospheric scattering model. However, its dehazing performance is limited due to inaccurate estimation. To this end, we present a new architecture called multi-level features and adaptive fusion network (MFAF-Net) for single image dehazing, which can obtain the haze-free image in an end-to-end manner. For one thing, we utilize a novel context enhanced module as the core of feature extraction, and it combines multi-scale dilation convolution layers with feature attention module, which enables to acquire much informative contextual information. For another, we present a new fusion approach called adaptive fusion module for both low- and high-level feature fusion, and it can provide more flexibility when handling features of inconsistent semantic and level; thus, our network restores images with more detailed information. Experimental results on both synthetic and real-world datasets demonstrate that MFAF-Net outperforms existing state-of-the-art methods in terms of quantitative and qualitative evaluation metrics. The code will be made publicly available on Github.

References

[1]
Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. arXiv:1804.02767 (2018)
[2]
Chen C, Wang G, Peng C, Fang Y, and Qin H Exploring rich and efficient spatial temporal interactions for real-time video salient object detection IEEE Trans. Image Process. 2021 PP 99 1-1
[3]
Das DK, Shit S, Ray DN, and Majumder S CGAN: closure-guided attention network for salient object detection Vis. Comput. 2021 38 1-15
[4]
Li J, Zhang S, and Huang T Multi-scale temporal cues learning for video person re-identification IEEE Trans. Image Process. 2020 PP 99 1-1
[5]
Xie J, Ge Y, Zhang J, Huang S, and Wang H Low-resolution assisted three-stream network for person re-identification Vis. Comput. 2021 10 1-11
[6]
Jia Z, Li Y, Tan Z, Wang W, Wang Z, and Yin G Domain-invariant feature extraction and fusion for cross-domain person re-identification Vis. Comput. 2022 39 1-12
[7]
McCartney, E.J.: Optics of the atmosphere: scattering by molecules and particles. nyjw (1976)
[8]
Narasimhan, S.G., Nayar, S.K.: Chromatic framework for vision in bad weather. In: IEEE Computer Society Conference on Computer Vision & Pattern Recognition (2000)
[9]
Narasimhan SG and Nayar SK Vision and the atmosphere Int. J. Comput. Vis. 2002 48 3 233-254
[10]
He K, Sun J, Fellow IEEE, and Tang X Single image haze removal using dark channel prior IEEE Trans. Pattern Anal. Mach. Intell. 2011 33 12 2341-2353
[11]
Zhu Q, Mai J, and Shao L A fast single image haze removal algorithm using color attenuation prior IEEE Trans. Image Process. 2015 24 11 3522-3533
[12]
Berman, D., Treibitz, T., Avidan, S.: Non-local image dehazing. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
[13]
Cai B, Xu X, Jia K, Qing C, and Tao D Dehazenet: an end-to-end system for single image haze removal IEEE Trans. Image Process. 2016 25 11 5187-5198
[14]
Zhang, H., Patel, V.M.: Densely connected pyramid dehazing network (2018)
[15]
Ren, W., Ma, L., Zhang, J., Pan, J., Cao, X., Liu, W., Yang, M.H.: Gated fusion network for single image dehazing (2018)
[16]
Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., Yang, M.H.: Single image dehazing via multi-scale convolutional neural networks (2016)
[17]
Zhang S and He F DRCDN: learning deep residual convolutional dehazing networks Vis. Comput. 2019 36 3 1797-1808
[18]
Yang F and Zhang Q Depth aware image dehazing Vis. Comput. 2021 11 1-9
[19]
Li X, Hua Z, and Li J Attention-based adaptive feature selection for multi-stage image dehazing Vis. Comput. 2022 39 663-678
[20]
Zhang S, Zhang J, He F, and Hou N DRDDN: dense residual and dilated dehazing network Vis. Comput. 2022 39 1-17
[21]
Xie, B., Guo, F., Cai, Z.: Improved single image dehazing using dark channel prior and multi-scale retinex. In: International Conference on Intelligent System Design & Engineering Application (2010)
[22]
Singh D, Kumar V, and Kaur M Single image dehazing using gradient channel prior Appl. Intell. 2019 49 8 4276-4293
[23]
Hu Q, Zhang Y, Zhu Y, Jiang Y, and Song M Single image dehazing algorithm based on sky segmentation and optimal transmission maps Vis. Comput. 2022 39 1-17
[24]
Huang, T., Li, S., Jia, X., Lu, H., Liu, J.: Neighbor2neighbor: self-supervised denoising from single noisy images (2021)
[25]
Khongkraphan K, Phonon A, and Nuiphom S An efficient blind image deblurring using a smoothing function Appl. Comput. Intell. Soft Comput. 2021 2021 9 1-10
[26]
Fan W, Wu Y, and Wang C Single image rain streak removal via layer similarity prior Appl Intell 2021 51 5822-5835
[27]
Yang W, Tan RT, Wang S, Fang Y, and Liu J Single image deraining: from model-based to data-driven and beyond IEEE Trans. Pattern Anal. Mach. Intell. 2020 PP 99 1-1
[28]
Li, B., Peng, X., Wang, Z., Xu, J., Dan, F.: Aod-net: all-in-one dehazing network. In: 2017 IEEE International Conference on Computer Vision (ICCV) (2017)
[29]
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241. Springer, Berlin (2015)
[30]
Chen, D., He, M., Fan, Q., Liao, J., Zhang, L., Hou, D., Yuan, L., Hua, G.: Gated context aggregation network for image dehazing and deraining. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1375–1383. IEEE (2019)
[31]
Ancuti, C.O., Ancuti, C., Vasluianu, F.A., Timofte, R.: Ntire 2020 challenge on nonhomogeneous dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 490–491 (2020)
[32]
Ancuti, C.O., Ancuti, C., Vasluianu, F.A., Timofte, R.: Ntire 2021 nonhomogeneous dehazing challenge report. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 627–646 (2021)
[33]
Das, S.D., Dutta, S.: Fast deep multi-patch hierarchical network for nonhomogeneous image dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 482–483 (2020)
[34]
Engin, D., Gen, A., Ekenel, H.K.: Cycle-dehaze: enhanced cyclegan for single image dehazing. IEEE (2018)
[35]
Li L, Dong Y, Ren W, Pan J, Gao C, Sang N, and Yang MH Semi-supervised image dehazing IEEE Trans. Image Process. 2020 29 2766-2779
[36]
Qin, X., Wang, Z., Bai, Y., Xie, X., Jia, H.: Ffa-net: feature fusion attention network for single image dehazing (2019)
[37]
Wang C, Wu Y, Cai Y, Yao G, Su Z, and Wang H Single image deraining via deep pyramid network with spatial contextual information aggregation Appl. Intell. 2020 50 5 1437-1447
[38]
Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. In: ICLR (2016)
[39]
Lin, T.Y., Dollar, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
[40]
Wang Z Image quality assessment: from error visibility to structural similarity IEEE Trans. Image Process. 2004 13 600-612
[41]
Qu, Y., Chen, Y., Huang, J., Xie, Y.: Enhanced pix2pix dehazing network. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
[42]
Zhang X, Wang T, Wang J, Tang G, and Zhao L Pyramid channel-based feature attention network for image dehazing Comput. Vis. Image Underst. 2020 197–198
[43]
Dong, Y., Liu, Y., Zhang, H., Chen, S., Qiao, Y.: FD-GAN: generative adversarial networks with fusion-discriminator for single image dehazing. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 7, pp. 10729–10736 (2020)
[44]
Shao, Y., Li, L., Ren, W., Gao, C., Sang, N.: Domain adaptation for image dehazing. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
[45]
Ullah H, Muhammad K, Irfan M, Anwar S, Sajjad M, Imran AS, and de Albuquerque VHC Light-DehazeNet: a novel lightweight CNN architecture for single image dehazing IEEE Trans. Image Process. 2021 30 8968-8982
[46]
Liu, X., Ma, Y., Shi, Z., Chen, J.: GridDehazeNet: attention-based multi-scale network for image dehazing. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 7314–7323 (2019)
[47]
Li B, Ren W, Fu D, Tao D, Feng D, Zeng W, and Wang Z Benchmarking single-image dehazing and beyond IEEE Trans. Image Process. 2018 28 492-505
[48]
Ancuti, C., Ancuti, C.O., Timofte, R., Vleeschouwer, C.D.: I-haze: a dehazing benchmark with real hazy and haze-free indoor images. In: International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 620–631. Springer (2018)
[49]
Ancuti, C.O., Ancuti, C., Timofte, R., De Vleeschouwer, C.: O-haze: a dehazing benchmark with real hazy and haze-free outdoor images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 754–762 (2018)
[50]
Huynh-Thu Q and Ghanbari M Scope of validity of PSNR in image/video quality assessment Electron. Lett. 2008 44 13 800-801
[51]
Blau, Y., Mechrez, R., Timofte, R., Michaeli, T., Zelnik-Manor, L.: The 2018 PIRM challenge on perceptual image super-resolution. In: Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pp. 0–0 (2018)
[52]
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
[53]
Fu, J., Xu, J., Tasaka, K., Chen, Z.: Residual squeeze-and-excitation network for fast image deraining. arXiv:2006.00757 (2020)
[54]
Wang C, Fan W, Zhu H, and Su Z Single image deraining via nonlocal squeeze-and-excitation enhancing network Appl. Intell. 2020 50 3 2932-2944
[55]
Xia L, Wu J, Lin Z, Hong L, and Zha H Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining 2018 Cham Springer

Cited By

View all
  • (2024)Knowledge-guided multi-perception attention network for image dehazingThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-03177-240:9(6479-6492)Online publication date: 1-Sep-2024
  • (2024)ZRDNet: zero-reference image defogging by physics-based decomposition–reconstruction mechanism and perception fusionThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-03109-040:8(5357-5374)Online publication date: 1-Aug-2024

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image The Visual Computer: International Journal of Computer Graphics
The Visual Computer: International Journal of Computer Graphics  Volume 40, Issue 4
Apr 2024
748 pages

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 13 July 2023
Accepted: 28 May 2023

Author Tags

  1. Image dehazing
  2. Multi-level feature
  3. Adaptive fusion
  4. Deep learning

Qualifiers

  • Research-article

Funding Sources

  • Winter Olympics Key Project Technology Fund

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 25 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Knowledge-guided multi-perception attention network for image dehazingThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-03177-240:9(6479-6492)Online publication date: 1-Sep-2024
  • (2024)ZRDNet: zero-reference image defogging by physics-based decomposition–reconstruction mechanism and perception fusionThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-03109-040:8(5357-5374)Online publication date: 1-Aug-2024

View Options

View options

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media