Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Utilizing Local Phase Information to Remove Rain from Video

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In the context of extracting information from video, bad weather conditions like rain can have a detrimental effect. In this paper, a novel framework to detect and remove rain streaks from video is proposed. The first part of the proposed framework for rain removal is a technique to detect rain streaks based on phase congruency features. The variation of features from frame to frame is used to estimate the candidate rain pixels in a frame. In order to reduce the number of false candidates due to global motion, frames are registered using phase correlation. The second part of the proposed framework is a novel reconstruction technique that utilizes information from three different sources, which are intensities of the rain affected pixel, spatial neighbors, and temporal neighbors. An optimal estimate for the actual intensity of the rain affected pixel is made based on the minimization of registration error between frames. An optical flow technique using local phase information is adopted for registration. This part of the proposed framework for removing rain is modeled such that the presence of local motion will not distort the features in the reconstructed video. The proposed framework is evaluated quantitatively and qualitatively on a variety of videos with varying complexities. The effectiveness of the algorithm is quantitatively verified by computing a no-reference image quality measure on individual frames of the reconstructed video. From a variety of experiments that are performed on output videos, it is shown that the proposed technique performs better than state-of-the-art techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Alessandrini, M., Basarab, A., Liebgott, H., & Bernard, O. (2013). Myocardial motion estimation from medical images using the monogenic signal. IEEE Transactions on Image Processing, 22(3), 1084–1095.

    Article  MathSciNet  Google Scholar 

  • Barnum, P. C., Narasimhan, S., & Kanade, T. (2010). Analysis of rain and snow in frequency space. International Journal of Computer Vision, 86(2–3), 256–274.

    Article  Google Scholar 

  • Bossu, J., Hautière, N., & Tarel, J. P. (2011). Rain or snow detection in image sequences through use of a histogram of orientation of streaks. International Journal of Comput Vision, 93(3), 348–367.

    Article  Google Scholar 

  • Brewer, N. (2008). Using the shape characteristics of rain to identify and remove rain from video. Structural, syntactic, and statistical pattern recognition (pp. 451–458). Berlin: Springer.

    Chapter  Google Scholar 

  • Felsberg, M. (2007). Optical flow estimation from monogenic phase. Complex motion (pp. 1–13). Berlin: Springer.

    Chapter  Google Scholar 

  • Felsberg, M., & Sommer, G. (2001). The monogenic signal. IEEE Transactions on Signal Processing, 49(12), 3136–3144. doi:10.1109/78.969520.

    Article  MathSciNet  Google Scholar 

  • Field, D. J., et al. (1987). Relations between the statistics of natural images and the response properties of cortical cells. The Journal of the Optical Society of America A, 4(12), 2379–2394.

    Article  Google Scholar 

  • Garg, K., & Nayar, S. (2004). Detection and removal of rain from videos. In CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004, (vol. 1, pp. I-528–I-535).

  • Garg, K., & Nayar, S. (2007). Vision and rain. International Journal of Computer Vision, 75(1), 3–27.

    Article  Google Scholar 

  • Garg, K., & Nayar, S. K. (2003). Photometric model of a rain drop. Technical Report, New York: Columbia University.

  • Hase, H., Miyake, K. & Yoneda, M. (1999). Real-time snowfall noise elimination. In Proceedings of International Conference on Image Processing, ICIP 99 (vol. 2, pp. 406–409).

  • Huang, D. A., Kang, L. W., Yang, M. C., Lin, C. W., & Wang, Y. C. (2012). Context-aware single image rain removal. In 2012 IEEE International Conference on Multimedia and Expo (ICME) (pp. 164–169).

  • Kang, L. W., Lin, C. W., & Fu, Y. H. (2012a). Automatic single-image-based rain streaks removal via image decomposition. IEEE Transactions on Image Processing, 21(4), 1742–1755.

    Article  MathSciNet  Google Scholar 

  • Kang, L. W., Lin, C. W., Lin, C. T. & Lin, Y. C. (2012b). Self-learning-based rain streak removal for image/video. In 2012 IEEE International Symposium on Circuits and Systems (ISCAS) (pp. 1871–1874).

  • Kim, J. H., Lee, C., Sim, J. Y., & Kim, C. S. (2013). Single-image deraining using an adaptive nonlocal means filter. In 2013 20th IEEE International Conference on Image Processing (ICIP) (pp. 914–917).

  • Kovesi, P. (1999). Image features from phase congruency. VIDERE: Journal of Computer Vision Research, 1(3), 1–26.

    Google Scholar 

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.

    Article  Google Scholar 

  • Mechler, F., Reich, D. S., & Victor, J. D. (2002). Detection and discrimination of relative spatial phase by v1 neurons. The Journal of Neuroscience, 22, 6129–6157.

    Google Scholar 

  • Moorthy, A. K., & Bovik, A. C. (2010). A two-step framework for constructing blind image quality indices. IEEE Signal Processing Letters, 17(5), 513–516.

    Article  Google Scholar 

  • Morrone, M., & Owens, R. (1987). Feature detection from local energy. Pattern Recognition Letters, 6(5), 303–313.

    Article  Google Scholar 

  • Narasimhan, S., & Nayar, S. (2002). Vision and the atmosphere. International Journal of Computer Vision, 48(3), 233–254.

    Article  MATH  Google Scholar 

  • Oppenheim, A. V., & Lim, J. S. (1981). The importance of phase in signals. Proceedings of the IEEE, 69(5), 529–541.

    Article  Google Scholar 

  • Park, W. J., & Lee, K. H. (2008). Rain removal using kalman filter in video. In International Conference on Smart Manufacturing Application, ICSMA 2008 (pp. 494–497).

  • Reddy, B., & Chatterji, B. N. (1996). An fft-based technique for translation, rotation, and scale-invariant image registration. IEEE Transactions on Image Processing, 5(8), 1266–1271.

    Article  Google Scholar 

  • Santhaseelan, V., & Asari, V. K. (2011). Phase congruency based technique for the removal of rain from video. Image analysis and recognition (pp. 30–39). Berlin: Springer.

    Chapter  Google Scholar 

  • Santhaseelan, V., & Asari, V. K. (2012). A phase space approach for detection and removal of rain in video. In IS&T/SPIE Electronic Imaging, (pp. 830,114–830,114).

  • Shen, M., Xue, P. (2011). A fast algorithm for rain detection and removal from videos. In 2011 IEEE International Conference on Multimedia and Expo (ICME) (pp. 1–6).

  • Starik, S., & Werman, M. (2003). Simulation of rain in videos. ICCV texture workshop, 2, 406–409.

    Google Scholar 

  • Tripathi, A., & Mukhopadhyay, S. (2011). A probabilistic approach for detection and removal of rain from videos. IETE Journal of Research, 57(1), 82–91.

    Article  Google Scholar 

  • Tripathi, A., & Mukhopadhyay, S. (2012a). Removal of rain from videos: A review. Signal, Image and Video Processing, 1(8), 1–10.

    Google Scholar 

  • Tripathi, A., & Mukhopadhyay, S. (2012b). Video post processing: low-latency spatiotemporal approach for detection and removal of rain. IET Image Processing, 6(2), 181–196.

    Article  MathSciNet  Google Scholar 

  • Unser, M., Sage, D., & Ville, D. V. D. (2009). Multiresolution monogenic signal analysis using the riesz-laplace wavelet transform. IEEE Transactions on Image Processing, 18(11), 2402–2418.

    Article  MathSciNet  Google Scholar 

  • Venkatesh, S. & Owens, R. (1989). An energy feature detection scheme. In IEEE International Conference on Image Processing: Conference Proceedings ICIP’89, Sep 5–8 1989, Singapore: IEEE.

  • Viola, P., & Jones, M. J. (2004). Robust real-time face detection. International Journal of Computer Vision, 57(2), 137–154.

    Article  Google Scholar 

  • Wadhwa, N., Rubinstein, M., Durand, F. & Freeman, W. T. (2013). Phase-based video motion processing. In Proceedings SIGGRAPH on ACM Transactions Graph (vol. 32(4), pp. 80).

  • Xue, X., Jin, X., Zhang, C. & Goto, S. (2012). Motion robust rain detection and removal from videos. In 2012 IEEE 14th International Workshop on Multimedia Signal Processing (MMSP) (pp. 170–174).

  • Zhang, X., Li, H., Qi, Y., Leow, W. K. & Ng, T. K. (2006). Rain removal in video by combining temporal and chromatic properties. In 2006 IEEE International Conference on Multimedia and Expo (pp. 461–464).

Download references

Author information

Authors and Affiliations

  1. University of Dayton, 300 College Park, Dayton, OH, 45469, USA

    Varun Santhaseelan & Vijayan K. Asari

Authors
  1. Varun Santhaseelan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vijayan K. Asari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Varun Santhaseelan.

Additional information

Communicated by Srinivasa Narasimhan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Santhaseelan, V., Asari, V.K. Utilizing Local Phase Information to Remove Rain from Video. Int J Comput Vis 112, 71–89 (2015). https://doi.org/10.1007/s11263-014-0759-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-014-0759-8

Keywords

Search

Navigation