Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Drfnet: dual stream recurrent feature sharing network for video dehazing

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

The primary effects of haze on captured images/frames are visibility degradation and color disturbance. Even though extensive research has been done on the tasks of video dehazing, they fail to perform better on varicolored hazy videos. The varicolored haze is still a challenging problem in video de-hazing. To tackle the problem of varicolored haze, the contextual information alone is not sufficient. In addition to adequate contextual information, color balancing is required to restore varicolored hazy images/videos. Therefore, this paper proposes a novel lightweight dual stream recurrent feature sharing network (with only 1.77 M parameters) for video de-hazing. The proposed framework involves: (1) A color balancing module to balance the color of input hazy frame in YCbCr space, (2) A multi-receptive multi-resolution module (MMM), which interlinks the RGB and YCbCr based features to learn global and rich contextual data, (3) Further, we have proposed a feature aggregation residual module (FARM) to strengthen the representative capability during reconstruction, (4) A channel attention module is proposed to resist redundant features by recalibrating weights of input features. Experimental results and ablation study show that the proposed model is superior to existing state-of-the-art approaches for video de-hazing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Availability of data and materials

The datasets used during result analysis of the current study are available publicly as DAVIS-2016, REVIDE and NYU depth.

Code Availability

Not applicable.

References

  1. Li R, Huang Y, Huang F, Yang G (2023) Image dehazing using multi-scale recursive networks. International Journal of Machine Learning and Cybernetics 1–12

  2. Yang Y, Hu W, Huang S, Wan W, Guan J (2022) Progressive image dehazing network based on dual feature extraction modules. International Journal of Machine Learning and Cybernetics 1–12

  3. Hu G, Tan A, He L, Shen H, Chen H, Wang C, Du H (2023) Pyramid feature boosted network for single image dehazing. International Journal of Machine Learning and Cybernetics 1–12

  4. Xu H, Long X, Wang M (2023) Uugan: a gan-based approach towards underwater image enhancement using non-pairwise supervision. Int J Mach Learn Cybern 14(3):725–738

    Article  Google Scholar 

  5. Galshetwar VM, Patil PW, Chaudhary S (2021) Single frame-based video dehazing with adversarial learning. In: International Conference on Computer Vision and Image Processing, pp. 36–47. Springer

  6. Khodja A, Zheng Z, Mo J, Zhang D, Chen L (2021) Rain to rain: learning real rain removal without ground truth. IEEE Access 9:57325–57337

    Article  Google Scholar 

  7. Hambarde P, Dudhane A, Murala S (2019) Single image depth estimation using deep adversarial training. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 989–993. IEEE

  8. Chen L, Liu H, Mo J, Zhang D, Yang J, Lin F, Zheng Z, Jia R (2022) Cross channel aggregation similarity network for salient object detection. Int J Mach Learn Cybern 13(8):2153–2169

    Article  Google Scholar 

  9. Chaudhary S, Murala S (2019) Deep network for human action recognition using weber motion. Neurocomputing 367:207–216

    Article  Google Scholar 

  10. Chaudhary S, Murala S (2018) Depth based end-to-end deep network for human action recognition. IET Computer Vision 13

  11. Chaudhary S, Murala S (2018) Tsnet: Deep network for human action recognition in hazy videos. In: IEEE Conference on SMC, pp. 3981–3986

  12. Dudhane A, Biradar KM, Patil PW, Hambarde P, Murala S (2020) Varicolored image de-hazing. In: Proceedings of the IEEE/CVF Conference on CVPR, pp. 4564–4573

  13. Bian P, Zheng Z, Zhang D (2021) Light-weight multi-channel aggregation network for image super-resolution. In: Pattern Recognition and Computer Vision: 4th Chinese Conference, PRCV 2021, Beijing, China, October 29–November 1, 2021, Proceedings, Part III 4, pp. 287–297. Springer

  14. Yin S, Yang X, Wang Y, Yang Y-H (2021) Visual attention dehazing network with multi-level features refinement and fusion. Pattern Recogn 118:108021

    Article  Google Scholar 

  15. Qin X, Wang Z, Bai Y, Xie X, Jia H (2020) Ffa-net: Feature fusion attention network for single image dehazing. In: Proceedings of the AAAI Conference on Artificial Intelligence, 34:11908–11915

  16. Ren W, Zhang J, Xu X, Ma L, Cao X, Meng G, Liu W (2018) Deep video dehazing with semantic segmentation. IEEE Trans Image Process 28(4):1895–1908

    Article  MathSciNet  Google Scholar 

  17. Zhao S, Zhang L, Shen Y, Zhou Y (2021) Refinednet: A weakly supervised refinement framework for single image dehazing. IEEE TIP 30:3391–3404

    Google Scholar 

  18. Zhang D, Zheng Z, Li M, Liu R (2021) Csart: Channel and spatial attention-guided residual learning for real-time object tracking. Neurocomputing 436:260–272

    Article  Google Scholar 

  19. Bian P, Zheng Z, Zhang D, Chen L, Li M (2021) Single image super-resolution via global-context attention networks. In: 2021 IEEE International Conference on Image Processing (ICIP), pp. 1794–1798. IEEE

  20. Ren D, Zuo W, Hu Q, Zhu P, Meng D (2019) Progressive image deraining networks: A better and simpler baseline, pp. 3932–3941

  21. Kulkarni A, Patil PW, Murala S (2022) Progressive subtractive recurrent lightweight network for video deraining. IEEE SPL 29:229–233

    Google Scholar 

  22. Li R, Pan J, Li Z, Tang J (2018) Single image dehazing via conditional generative adversarial network. In: 2018 IEEE/CVF Conference on CVPR, pp. 8202–8211

  23. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: Proceedings of the IEEE Conference on CVPR, pp. 2472–2481

  24. Patil PW, Gupta S, Rana S, Venkatesh S (2022) Dual-frame spatio-temporal feature modulation for video enhancement. Pattern Recognition, 108822

  25. Zhang X, Dong H, Pan J, Zhu C, Tai Y, Wang C, Li J, Huang F, Wang F (2021) Learning to restore hazy video: A new real-world dataset and a new method. In: CVPR, pp. 9239–9248

  26. Perazzi F, Pont-Tuset J, McWilliams B, Van Gool L, Gross M, Sorkine-Hornung A (2016) A benchmark dataset and evaluation methodology for video object segmentation. In: 2016 IEEE Conference on CVPR, pp. 724–732

  27. Silberman N, Fergus R (2011) Indoor scene segmentation using a structured light sensor. In: 2011 IEEE ICCV Workshops, pp. 601–608

  28. He K, Sun J, Tang X (2011) Single image haze removal using dark channel prior. IEEE Transactions on PAMI 33(12):2341–2353

    Article  Google Scholar 

  29. Dhara S, Roy M, Sen D, Kumar Biswas P (2021) Color cast dependent image dehazing via adaptive airlight refinement and non-linear color balancing. IEEE Transactions on CSVT 31(5):2076–2081

    Google Scholar 

  30. Liu X, Li H, Zhu C (2022) Joint contrast enhancement and exposure fusion for real-world image dehazing. IEEE Trans Multimedia 24:3934–3946

    Article  Google Scholar 

  31. Hodges C, Bennamoun M, Rahmani H (2019) Single image dehazing using deep neural networks. Pattern Recogn Lett 128:70–77

    Article  Google Scholar 

  32. Li B, Peng X, Wang Z, Xu J, Feng D (2017) End-to-end united video dehazing and detection. arXiv preprint arXiv:1709.03919

  33. Kumar R, Balasubramanian R, Kaushik BK (2020) Efficient method and architecture for real-time video defogging. IEEE Trans Intell Transp Syst 22(10):6536–6546

    Article  Google Scholar 

  34. Patil PW, Biradar KM, Dudhane A, Murala S (2020) An end-to-end edge aggregation network for moving object segmentation. In: 2020 IEEE/CVF Conference on CVPR, pp. 8146–8155

  35. Patil PW, Dudhane A, Kulkarni A, Murala S, Gonde AB, Gupta S (2021) An unified recurrent video object segmentation framework for various surveillance environments. IEEE TIP 30:7889–7902

    Google Scholar 

  36. Liu X, Ma Y, Shi Z, Chen J (2019) Griddehazenet: Attention-based multi-scale network for image dehazing. In: 2019 IEEE/CVF ICCV, pp. 7313–7322

  37. Galshetwar VM, Kulkarni A, Chaudhary S (2022) Consolidated adversarial network for video de-raining and de-hazing. In: 2022 18th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–8

  38. Reinhard E, Adhikhmin M, Gooch B, Shirley P (2001) Color transfer between images. IEEE Comput Graphics Appl 21(5):34–41

    Article  Google Scholar 

  39. Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: IEEE Conference on CVPR, pp. 1125–1134

  40. Dudhane A, Aulakh HS, Murala S (2019) Ri-gan: An end-to-end network for single image haze removal. In: 2019 IEEE/CVF Conference on CVPR Workshops, pp. 2014–2023

  41. Que Y, Li S, Lee HJ (2020) Attentive composite residual network for robust rain removal from single images. IEEE Transactions on Multimedia

  42. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  43. Cai B, Xu X, Tao D (2016) Real-time video dehazing based on spatio-temporal mrf. In: PCM

  44. Tassano M, Delon J, Veit T (2020) Fastdvdnet: Towards real-time deep video denoising without flow estimation, pp. 1351–1360

  45. Dong H, Pan J, Xiang L, Hu Z, Zhang X, Wang F, Yang M-H (2020) Multi-scale boosted dehazing network with dense feature fusion. In: 2020 IEEE/CVF Conference on CVPR, pp. 2154–2164

  46. Wang X, Chan KK, Yu K, Dong C, Loy C (2019) Edvr: Video restoration with enhanced deformable convolutional networks. In: 2019 IEEE/CVF Conference on CVPR Workshops, pp. 1954–1963. IEEE Computer Society, ???

  47. Li R (2021) Progressive deep video dehazing without explicit alignment estimation. CoRR arXiv:2107.07837

  48. Huang C, Li J, Li B, Liu D, Lu Y (2022) Neural compression-based feature learning for video restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5872–5881

  49. Xu J, Hu X, Zhu L, Dou Q, Dai J, Qiao Y, Heng P-A (2023) Video dehazing via a multi-range temporal alignment network with physical prior. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18053–18062

  50. Qu Y, Chen Y, Huang J, Xie Y (2019) Enhanced pix2pix dehazing network. In: Proceedings of the IEEE/CVF Conference on CVPR, pp. 8152–8160

  51. Galshetwar VM, Patil PW, Chaudhary S (2022) Video enhancement with single frame. In: Computer Vision and Image Processing, pp. 206–218. Springer, Cham

  52. Ranftl R, Lasinger K, Hafner D, Schindler K, Koltun V (2020) Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE transactions on PAMI

  53. Shin J, Park H, Paik J (2022) Region-based dehazing via dual-supervised triple-convolutional network. IEEE Trans Multimedia 24:245–260

    Article  Google Scholar 

  54. Chen D, He M, Fan Q, Liao J, Zhang L, Hou D, Yuan L, Hua G (2019) Gated context aggregation network for image dehazing and deraining. In: 2019 IEEE WACV), pp. 1375–1383. IEEE

  55. Shin J, Kim M, Paik J, Lee S (2020) Radiance-reflectance combined optimization and structure-guided l0-norm for single image dehazing. IEEE Trans Multimedia 22(1):30–44

    Article  Google Scholar 

  56. Zhu Z, Wei H, Hu G, Li Y, Qi G, Mazur N (2021) A novel fast single image dehazing algorithm based on artificial multiexposure image fusion. IEEE Trans Instrum Meas 70:1–23

    Google Scholar 

  57. Cai B, Xu X, Jia K, Qing C, Tao D (2016) Dehazenet: An end-to-end system for single image haze removal. IEEE TIP 25(11):5187–5198

    MathSciNet  Google Scholar 

  58. Li B, Peng X, Wang Z, Xu J, Feng D (2017) Aod-net: All-in-one dehazing network. In: Proceedings of the IEEE Conference on ICCV, pp. 4770–4778

  59. Ren W, Ma L, Zhang J, Pan J, Cao X, Liu W, Yang M-H (2018) Gated fusion network for single image dehazing. In: Proceedings of the IEEE Conference on CVPR, pp. 3253–3261

Download references

Acknowledgements

The authors would like to thank Department of Computer Science and Engineering, PEC Chandigarh, for their support. Also, we would like to thank Dr. Prashant Patil and Mr. Ashutosh Kulkarni for his valuable suggestions to this work. A special thanks to Finolex Academy of Management and Technology, Ratnagiri for their invaluable support in my research work.

Funding

No funding was received for conducting this study.

Author information

Author notes

  1. P. Saini and S. Chaudhary have contributed equally to this work.

Authors and Affiliations

  1. Department of Computer Science and Engineering, Punjab Engineering College, Sector 12, Chandigarh, Chandigarh, 160012, India

    Vijay M. Galshetwar & Poonam Saini

  2. Amity University, Mohali, Punjab, 140306, India

    Sachin Chaudhary

Authors
  1. Vijay M. Galshetwar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Poonam Saini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sachin Chaudhary
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors have equally contributed.

Corresponding author

Correspondence to Sachin Chaudhary.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Ethics approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

The participant has consented to the submission of the case report to the journal.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Galshetwar, V.M., Saini, P. & Chaudhary, S. Drfnet: dual stream recurrent feature sharing network for video dehazing. Int. J. Mach. Learn. & Cyber. 15, 3397–3412 (2024). https://doi.org/10.1007/s13042-024-02099-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-024-02099-2

Keywords

Search

Navigation