Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Solving Uncalibrated Photometric Stereo Using Fewer Images by Jointly Optimizing Low-rank Matrix Completion and Integrability

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

We introduce a new, integrated approach to uncalibrated photometric stereo. We perform 3D reconstruction of Lambertian objects using multiple images produced by unknown, directional light sources. We show how to formulate a single optimization that includes rank and integrability constraints, allowing also for missing data. We then solve this optimization using the Alternating Direction Method of Multipliers (ADMM). We conduct extensive experimental evaluation on real and synthetic data sets. Our integrated approach is particularly valuable when performing photometric stereo using as few as 4–6 images, since the integrability constraint is capable of improving estimation of the linear subspace of possible solutions. We show good improvements over prior work in these cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Ackermann, J., Goesele, M.: A survey of photometric stereo techniques. Found. Trends Comput. Graph. Vis. 9(3–4), 149–254 (2015)

    Article  MATH  Google Scholar 

  2. Alldrin, N.G., Mallick, S.P., Kriegman, D.J.: Resolving the generalized bas-relief ambiguity by entropy minimization. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR’07, IEEE, pp. 1–7 (2007)

  3. Basri, R., Jacobs, D., Kemelmacher, I.: Photometric stereo with general, unknown lighting. Int. J. Comput. Vis. 72(3), 239–257 (2007)

    Article  Google Scholar 

  4. Belhumeur, P.N., Kriegman, D.J., Yuille, A.L.: The bas-relief ambiguity. Int. J. Comput. Vis. 35(1), 33–44 (1999)

    Article  Google Scholar 

  5. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  MATH  Google Scholar 

  6. Cabral, R.S., Torre, F., Costeira, J.P., Bernardino, A.: Matrix completion for multi-label image classification. In: Advances in Neural Information Processing Systems, pp. 190–198 (2011)

  7. Cai, J.F., Candès, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  8. Chandraker, M., Agarwal, S., Kriegman, D.: Shadowcuts: Photometric stereo with shadows. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR’07. IEEE, pp. 1–8 (2007)

  9. Chandraker, M.K., Kahl, C.F., Kriegman, D.J.: Reflections on the generalized bas-relief ambiguity. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, IEEE, vol. 1, pp. 788–795 (2005)

  10. Drbohlav, O., Chaniler, M.: Can two specular pixels calibrate photometric stereo? In: Tenth IEEE International Conference on Computer Vision, 2005. ICCV 2005, IEEE, vol. 2, pp. 1850–1857 (2005)

  11. Favaro, P., Papadhimitri, T.: A closed-form solution to uncalibrated photometric stereo via diffuse maxima. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 821–828 (2012)

  12. Georghiades, A.S.: Incorporating the torrance and sparrow model of reflectance in uncalibrated photometric stereo. In: Proceedings of Ninth IEEE International Conference on Computer Vision, 2003, IEEE, pp. 816–823 (2003)

  13. Goldstein, T., O’Donoghue, B., Setzer, S., Baraniuk, R.: Fast alternating direction optimization methods. SIAM J. Imaging Sci. 7(3), 1588–1623 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  14. Hayakawa, H.: Photometric stereo under a light source with arbitrary motion. JOSA A 11(11), 3079–3089 (1994)

    Article  Google Scholar 

  15. Hu, Y., Zhang, D., Ye, J., Li, X., He, X.: Fast and accurate matrix completion via truncated nuclear norm regularization. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2117–2130 (2013)

    Article  Google Scholar 

  16. Joshi, N., Kemelmacher, I., Simon, I.: Photometric stereo dataset. http://courses.cs.washington.edu/courses/cse455/10wi/projects/project4/ (2015)

  17. Mecca, R., Tankus, A., Wetzler, A., Bruckstein, A.M.: A direct differential approach to photometric stereo with perspective viewing. SIAM J. Imaging Sci. 7(2), 579–612 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  18. Oh, T.H., Kim, H., Tai, Y.W., Bazin, J.C., Kweon, I.S.: Partial sum minimization of singular values in RPCA for low-level vision. In: 2013 IEEE International Conference on Computer Vision (ICCV), IEEE, pp. 145–152 (2013)

  19. Oh, T.H., Tai, Y.W., Bazin, J.C., Kim, H., Kweon, I.S.: Partial sum minimization of singular values in robust PCA: algorithm and applications. IEEE Trans. Pattern Anal. Mach. Intell. 38(4), 744–758 (2016)

    Article  Google Scholar 

  20. Okabe, T., Sato, I., Sato, Y.: Attached shadow coding: estimating surface normals from shadows under unknown reflectance and lighting conditions. In: 2009 IEEE 12th International Conference on Computer Vision, IEEE, pp. 1693–1700 (2009)

  21. Okatani, T., Yoshida, T., Deguchi, K.: Efficient algorithm for low-rank matrix factorization with missing components and performance comparison of latest algorithms. In: 2011 IEEE International Conference on Computer Vision (ICCV), IEEE, pp. 842–849 (2011)

  22. Papadhimitri, T., Favaro, P.: A new perspective on uncalibrated photometric stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1474–1481 (2013)

  23. Phong, B.T.: Illumination for computer generated pictures. Commun. ACM 18(6), 311–317 (1975)

    Article  Google Scholar 

  24. Quéau, Y., Lauze, F., Durou, J.D.: Solving uncalibrated photometric stereo using total variation. J. Math. Imaging Vis. 52(1), 87–107 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  25. Shi, B., Matsushita, Y., Wei, Y., Xu, C., Tan, P.: Self-calibrating photometric stereo. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 1118–1125 (2010)

  26. Shi, B., Wu, Z., Mo, Z., Duan, D., Yeung, S.K., Tan, P.: A benchmark dataset and evaluation for non-lambertian and uncalibrated photometric stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3707–3716 (2016)

  27. Sunkavalli, K., Zickler, T., Pfister, H.: Visibility subspaces: uncalibrated photometric stereo with shadows. In: European Conference on Computer Vision, Springer, pp. 251–264 (2010)

  28. Tan, P.: Phong reflectance model. In: Computer Vision, Springer, pp. 592–594 (2014)

  29. Tan, .P, Mallick, S.P., Quan, L., Kriegman, D.J., Zickler, T.: Isotropy, reciprocity and the generalized bas-relief ambiguity. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR’07, IEEE, pp. 1–8 (2007)

  30. Woodham, R.J.: Photometric method for determining surface orientation from multiple images. Opt. Eng. 19(1), 191 (1980)

    Article  Google Scholar 

  31. Wu, L., Ganesh, A., Shi, B., Matsushita, Y., Wang, Y., Ma, Y.: Robust photometric stereo via low-rank matrix completion and recovery. In: Computer Vision–ACCV 2010, Springer, pp. 703–717 (2011)

  32. Xiong, Y., Chakrabarti, A., Basri, R., Gortler, S.J., Jacobs, D.W., Zickler, T.: From shading to local shape. IEEE Trans. Pattern Anal. Mach. Intell. 37(1), 67–79 (2015)

    Article  Google Scholar 

  33. Yuille, A., Snow, D.: Shape and albedo from multiple images using integrability. In: Proceedings of 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1997. IEEE, pp. 158–164 (1997)

Download references

Author information

Authors and Affiliations

  1. Center for Automation Research, University of Maryland, College Park, MD, USA

    Soumyadip Sengupta, Hao Zhou & David Jacobs

  2. TU Dresden, Dresden, Germany

    Walter Forkel

  3. Department of Computer Science and Applied Mathematics, Weizmann Institute of Science, Rehovot, Israel

    Ronen Basri

  4. Department of Computer Science, University of Maryland, College Park, MD, USA

    Tom Goldstein

Authors
  1. Soumyadip Sengupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hao Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Walter Forkel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ronen Basri
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tom Goldstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. David Jacobs
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Soumyadip Sengupta.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sengupta, S., Zhou, H., Forkel, W. et al. Solving Uncalibrated Photometric Stereo Using Fewer Images by Jointly Optimizing Low-rank Matrix Completion and Integrability. J Math Imaging Vis 60, 563–575 (2018). https://doi.org/10.1007/s10851-017-0772-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-017-0772-y

Keywords

Search

Navigation