Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Effective shadow removal via multi-scale image decomposition

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Shadow removal is a fundamental and challenging problem in image processing field. Current approaches can only process shadows with simple scenes. For complex texture and illumination, the performance is less impressive. In this paper, we propose a novel shadow removal algorithm based on multi-scale image decomposition, which can recover the illumination for complex shadows with inconsistent illumination and different surface materials. Independent of shadow detection, our algorithm only requires a rough boundary distinguishing shadow regions from non-shadow regions. It first performs a multi-scale decomposition for the input image based on an illumination-sensitive smoothing process and then removes shadows in the basic layer using a local-to-global optimization strategy, which fuses all local shadow-free results in a global manner. Finally, we recover the texture details for the shadow-free basic layer and obtain the final shadow-free image. We validate the performance of the proposed method under various lighting and texture conditions and show consistent illumination between shadow and surrounding regions in the shadow removal results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Ssstrunk, S.: Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans. PAMI 34(11), 2274–2282 (2012)

    Article  Google Scholar 

  2. Arbel, E., Hel-Or, H.: Shadow removal using intensity surfaces and texture anchor points. IEEE Trans. PAMI 33(6), 1202–1216 (2011)

    Article  Google Scholar 

  3. Clarenz, U., Griebel, M., Rumpf, M., Schweitzer, M.A., Telea, A.: Feature sensitive multiscale editing on surfaces. Vis. Comput. 20(5), 329–343 (2004)

    Article  Google Scholar 

  4. Darabi, S., Shechtman, E., Barnes, C., Dan, B.G., Sen, P.: Image melding. ACM TOG 31(4), 1–10 (2012)

    Article  Google Scholar 

  5. Finlayson, G.D., Drew, M.S., Lu, C.: Intrinsic images by entropy minimization. In: ECCV, pp. 582–595 (2004)

  6. Finlayson, G.D., Hordley, S.D., Drew, M.S.: Removing shadows from images. In: ECCV, vol. 4(2353), pp. 823–836 (2002)

  7. Finlayson, G.D., Hordley, S.D., Lu, C., Drew, M.S.: On the removal of shadows from images. IEEE Trans. PAMI 28(1), 59–68 (2005)

    Article  Google Scholar 

  8. Gangnet, M., Blake, A.: Poisson image editing. In: ACM SIGGRAPH, pp. 313–318 (2003)

  9. Gryka, M., Terry, M., Brostow, G.J.: Learning to remove soft shadows. ACM TOG 34(5), 1–15 (2015)

    Article  Google Scholar 

  10. Guo, R., Dai, Q., Hoiem, D.: Single-image shadow detection and removal using paired regions. In: CVPR, pp. 2033–2040 (2011)

  11. Hu, X., Fu, C.W., Zhu, L., Qin, J., Heng, P.A.: Direction-aware spatial context features for shadow detection and removal. In: CVPR (2018)

  12. Khan, S.H., Bennamoun, M., Sohel, F., Togneri, R.: Automatic shadow detection and removal from a single image. IEEE Trans. PAMI 38(3), 431–446 (2016)

    Article  Google Scholar 

  13. Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Trans. PAMI 30(2), 228–242 (2008)

    Article  Google Scholar 

  14. Li, H., Zhang, L., Shen, H.: An adaptive nonlocal regularized shadow removal method for aerial remote sensing images. IEEE Trans. Geosci. Remote Sens. 52(1), 106–120 (2014)

    Article  Google Scholar 

  15. Liu, F., Gleicher, M.: Texture-consistent shadow removal. In: ECCV, pp. 437–450 (2008)

  16. Matting, S., Chuang, Y.Y., Dan, B.G., Curless, B., Salesin, D.H., Szeliski, R.: Shadow matting and compositing. ACM TOG 22(3), 494–500 (2003)

    Article  Google Scholar 

  17. Mohan, A., Tumblin, J., Choudhury, P.: Editing soft shadows in a digital photograph. IEEE Comput. Graph. Appl. 27(2), 23–31 (2007)

    Article  Google Scholar 

  18. Pajak, D., Čadík, M., Aydın, T.O., Okabe, M., Myszkowski, K., Seidel, H.P.: Contrast prescription for multiscale image editing. Vis. Comput. 26(6–8), 739–748 (2010)

    Article  Google Scholar 

  19. Qu, L., Tian, J., He, S., Tang, Y., Lau, R.W.H.: Deshadownet: a multi-context embedding deep network for shadow removal. In: CVPR, pp. 2308–2316 (2017)

  20. Reinhard, E., Adhikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graph. Appl. 21(5), 34–41 (2001)

    Article  Google Scholar 

  21. Shor, Y., Lischinski, D.: The shadow meets the mask: pyramid-based shadow removal. Comput. Graph. Forum 27(2), 577–586 (2008)

    Article  Google Scholar 

  22. Subr, K., Soler, C.: Edge-preserving multiscale image decomposition based on local extrema. ACM TOG 28(5), 1–9 (2009)

    Article  Google Scholar 

  23. Vicente, T.F.Y., Hoai, M., Samaras, D.: Leave-one-out kernel optimization for shadow detection and removal. IEEE Trans. PAMI PP(99), 1 (2018)

    Google Scholar 

  24. Vicente, T.F.Y., Hou, L., Yu, C.P., Hoai, M., Samaras, D.: Large-Scale Training of Shadow Detectors with Noisily-Annotated Shadow Examples. Springer, Berlin (2016)

    Book  Google Scholar 

  25. Wang, J., Li, X., Hui, L., Yang, J.: Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In: CVPR (2018)

  26. Wu, T.P., Tang, C.K.: A Bayesian approach for shadow extraction from a single image. In: ICCV, pp. 480–487 (2005)

  27. Wu, T.P., Tang, C.K., Brown, M.S., Shum, H.Y.: Natural shadow matting. ACM TOG 26(2), 8 (2007)

    Article  Google Scholar 

  28. Xiao, C., She, R., Xiao, D., Ma, K.L.: Fast shadow removal using adaptive multi-scale illumination transfer. Comput. Graph. Forum 32(8), 207–218 (2013)

    Article  Google Scholar 

  29. Xiao, C., Xiao, D., Zhang, L., Chen, L.: Efficient shadow removal using subregion matching illumination transfer. Comput. Graph. Forum 32(7), 421–430 (2013)

    Article  Google Scholar 

  30. Xiao, Y., Tsougenis, E., Tang, C.: Shadow removal from single RGB-D images. In: CVPR, pp. 3011–3018 (2014)

  31. Yagyu, S., Sakiyama, A., Tanaka, Y.: Edge preserving multiscale image decomposition with customized domain transform filters. In: Signal and Information Processing, pp. 458–462 (2016)

  32. Yang, Q., Tan, K.H., Ahuja, N.: Shadow removal using bilateral filtering. IEEE TIP 21(10), 4361–4368 (2012)

    MathSciNet  MATH  Google Scholar 

  33. Yanli, L., Xavier, G.: Online tracking of outdoor lighting variations for augmented reality with moving cameras. IEEE Trans. Vis. Comput. Graph. 18(4), 573–580 (2012)

    Article  Google Scholar 

  34. Zhang, L., Yan, Q., Liu, Z., Zou, H., Xiao, C.: Illumination decomposition for photograph with multiple light sources. IEEE Trans. Image Process. 26(9), 4114–4127 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  35. Zhang, L., Zhang, Q., Xiao, C.: Shadow remover: image shadow removal based on illumination recovering optimization. IEEE TIP 24(11), 4623–36 (2015)

    MathSciNet  MATH  Google Scholar 

  36. Zhu, J., Samuel, K.G.G., Masood, S.Z., Tappen, M.F.: Learning to recognize shadows in monochromatic natural images. In: CVPR, pp. 223–230 (2010)

Download references

Acknowledgements

This work was partly supported by The National Key Research and Development Program of China (2017YF-B1002600), the NSFC (No. 61672390), Wuhan Science and Technology Plan Project (No. 2017010201010109), Key Technological Innovation Projects of Hubei Province (2018AAA062), and China Postdoctoral Science Found (No. 070307).

Author information

Authors and Affiliations

  1. Hubei Key Laboratory of Intelligent Information Processing and Realtime Industrial System, School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, 430081, China

    Ling Zhang & Xiaolong Zhang

  2. JD.com American Technologies Corporation, Mountain View, CA, 94043, USA

    Qingan Yan

  3. School of Computer Science, Wuhan University, Wuhan, 430072, China

    Yao Zhu & Chunxia Xiao

Authors
  1. Ling Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qingan Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yao Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaolong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chunxia Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chunxia Xiao.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 13898 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, L., Yan, Q., Zhu, Y. et al. Effective shadow removal via multi-scale image decomposition. Vis Comput 35, 1091–1104 (2019). https://doi.org/10.1007/s00371-019-01685-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-019-01685-8

Keywords

Search

Navigation