Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

HDRD-Net: High-resolution detail-recovering image deraining network

  • 1221: Deep Learning for Image/Video Compression and Visual Quality Assessment
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image deraining aims to restore the clean scenes of rainy images, which facilitates a number of outdoor vision systems, such as autonomous driving, unmanned aerial vehicles and surveillance systems. This paper proposes a high-resolution detail-recovering image deraining network (HDRD-Net) to effectively remove rain streaks and recover lost details, as well as improving the quality of derained images. HDRD-Net consists of three sub-networks. First, we combine the residual network and Squeeze-and-Excitation block for rain streak removal. Second, we integrate the Structure Detail Context Aggregation block into the detail-recovering network to extract detail features form rainy images. Third, a dual super-resolution reconstruction network is utilized to enhance the quality of derained images. In addition, we extend the Rain100 dataset by incorporating low-resolution rainy images to construct a new Rain100++ dataset for high-resolution image deraining. Experimental results on several datasets show that HDRD-Net outperforms state-of-the-art methods in terms of rain removal, detail preservation and visual quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Anderson P, Fouad A (2003) Institute of Electrical and Electronics Engineers. Power system control and stability

  2. Bossu J, Hautiere N, Tarel JP (2011) Rain or snow detection in image sequences through use of a histogram of orientation of streaks. Int J Comput Vis 93 (3):348

    Article  Google Scholar 

  3. Brewer N, Liu N (2008) Using the shape characteristics of rain to identify and remove rain from video. In: Joint IAPR international workshops on statistical techniques in pattern recognition (SPR) and structural and syntactic pattern recognition (SSPR) (Springer), pp 451–458

  4. Chen YL, Hsu CT (2013) A generalized low-rank appearance model for spatio-temporally correlated rain streaks. In: proceedings of the IEEE international conference on computer vision, pp 1968–1975

  5. Deng S, Wei M, Wang J, Feng Y, Liang L, Xie H, Wang FL, Wang M (2020) Detail-recovery image deraining via context aggregation networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 14,560–14,569

  6. Ding X, Chen L, Zheng X, Huang Y, Zeng D (2016) Single image rain and snow removal via guided L0 smoothing filter. Multimed Tools Appl 75(5):2697

    Article  Google Scholar 

  7. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: European conference on computer vision (Springer), pp 184–199

  8. Eigen D, Krishnan D, Fergus R (2013) Restoring an image taken through a window covered with dirt or rain. In: Proceedings of the IEEE international conference on computer vision, pp 633–640

  9. Fu X, Huang J, Zeng D, Huang Y, Ding X, Paisley J (2017) Removing rain from single images via a deep detail network. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 3855–3863

  10. Fu YH, Kang LW, Lin CW, Hsu CT (2011) Single-frame-based rain removal via image decomposition. In: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE), vol 2011, pp 1453–1456

  11. Garg K, Nayar SK (2004) Detection and removal of rain from videos. In: Proceedings of the 2004 IEEE Computer society conference on computer vision and pattern recognition, 2004. CVPR 2004., vol 1 (IEEE), vol 1, pp I–I

  12. Hu X, Fu CW, Zhu L, Heng PA (2019) Depth-attentional features for single-image rain removal. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 8022–8031

  13. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  14. Hu X, Zhu L, Wang T, Fu CW, Heng PA (2021) Single-image real-time rain removal based on depth-guided non-local features. IEEE Trans Image Process 30:1759

    Article  Google Scholar 

  15. Jain V, Seung S (2008) Natural image denoising with convolutional networks. Adv Neural Inform Process Syst 21:769

    Google Scholar 

  16. Kang LW, Lin CW, Fu YH (2011) Automatic single-image-based rain streaks removal via image decomposition. IEEE Trans image process 21(4):1742

    Article  MathSciNet  MATH  Google Scholar 

  17. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1646–1654

  18. Li Y, Tan RT, Guo X, Lu J, Brown MS (2016) Rain streak removal using layer priors. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2736–2744

  19. Li X, Wu J, Lin Z, Liu H, Zha H (2018) Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: Proceedings of the european conference on computer vision (ECCV), pp 254–269

  20. Mu P, Chen J, Liu R, Fan X, Luo Z (2018) Learning bilevel layer priors for single image rain streaks removal. IEEE Signal Process Lett 26(2):307

    Article  Google Scholar 

  21. Pan J, Liu S, Sun D, Zhang J, Liu Y, Ren J, Li Z, Tang J, Lu H, Tai YW et al (2018) Learning dual convolutional neural networks for low-level vision, pp 3070–3079

  22. Qian R, Tan RT, Yang W, Su J, Liu J (2018) Attentive generative adversarial network for raindrop removal from a single image. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2482–2491

  23. Ren W, Tian J, Han Z, Chan A, Tang T (2017) Video desnowing and deraining based on matrix decomposition. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 4210–4219

  24. Santhaseelan V, Asari VK (2015) Utilizing local phase information to remove rain from video. Int J Comput Vis 112(1):71

    Article  Google Scholar 

  25. Tripathi AK, Mukhopadhyay S (2011) A probabilistic approach for detection and removal of rain from videos. IETE J Res 57(1):82

    Article  Google Scholar 

  26. Xu J, Zhao W, Liu P, Tang X (2012) Removing rain and snow in a single image using guided filter. In: 2012 IEEE International Conference on Computer Science and Automation Engineering (CSAE), vol 2 (IEEE, 2012), vol 2, pp 304–307

  27. Yang Y, Lu H (2019) Single image deraining via recurrent hierarchy enhancement network. In: Proceedings of the 27th ACM International conference on multimedia, pp 1814–1822

  28. Yang W, Tan RT, Feng J, Liu J, Guo Z, Yan S (2017) Deep joint rain detection and removal from a single image. In: Proceedings of the ieee conference on computer vision and pattern recognition, pp 1357–1366

  29. Yu W, Huang Z, Zhang W, Feng L, Xiao N (2019) Gradual network for single image de-raining. In: Proceedings of the 27th ACM International conference on multimedia, pp 1795–1804

  30. Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv:1511.07122

  31. Zhang X, Li H, Qi Y, Leow WK, Ng TK (2006) Rain removal in video by combining temporal and chromatic properties. In: 2006 IEEE international conference on multimedia and expo (IEEE 2006), pp 461–464

  32. Zhang H, Sindagi V, Patel VM (2019) Image de-raining using a conditional generative adversarial network. IEEE Trans Circuits Systems Vid Technol 30(11):3943

    Article  Google Scholar 

  33. Zhao X, Liu P, Liu J, Xianglong T (2008) The application of histogram on rain detection in video. In: 11th Joint International Conference on Information Sciences (Atlantis Press), pp 382–387

  34. Zheng X, Liao Y, Guo W, Fu X, Ding X (2013) Single-image-based rain and snow removal using multi-guided filter. In: International conference on neural information processing (Springer), pp 258–265

  35. Zhu L, Deng Z, Hu X, Xie H, Xu X, Qin J, Heng PA (2020) Learning gated non-local residual for single-image rain streak removal. IEEE Transactions on Circuits and Systems for Video Technology

  36. Zhu L, Fu CW, Lischinski D, Heng PA (2017) Joint bi-layer optimization for single-image rain streak removal. In: Proceedings of the IEEE international conference on computer vision, pp 2526–2534

Download references

Acknowledgements

The work described in this paper was fully supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (UGC/FDS16/E01/19), Hong Kong Metropolitan University Research Grant (No. RD/2021/09), the One-off Special Fund from Central and Faculty Fund in Support of Research from 2019/20 to 2021/22 (MIT02/19-20), the Research Cluster Fund (RG 78/2019-2020R), the Interdisciplinary Research Scheme of the Dean’s Research Fund 2019-20 (FLASS/DRF/IDS-2) of The Education University of Hong Kong, and the Lam Woo Research Fund (LWI20011) of Lingnan University, Hong Kong.

Author information

Authors and Affiliations

  1. Hong Kong Metropolitan University, Hong Kong SAR, China

    Dingkun Zhu, Weiming Wang & Fu Lee Wang

  2. Nanjing University of Aeronautics and Astronautics, Nanjing, China

    Sen Deng & Mingqiang Wei

  3. The Education University of Hong Kong, Hong Kong SAR, China

    Gary Cheng

  4. Lingnan University, Hong Kong SAR, China

    Haoran Xie

Authors
  1. Dingkun Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sen Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weiming Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gary Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mingqiang Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Fu Lee Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Haoran Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gary Cheng.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, D., Deng, S., Wang, W. et al. HDRD-Net: High-resolution detail-recovering image deraining network. Multimed Tools Appl 81, 42889–42906 (2022). https://doi.org/10.1007/s11042-022-13489-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13489-5

Keywords

Search

Navigation