Nothing Special   »   [go: up one dir, main page]
Skip to main content
SpringerLink
Log in

Differentiable Programming for Hyperspectral Unmixing Using a Physics-Based Dispersion Model

  • Conference paper
  • First Online:
Computer Vision – ECCV 2020 (ECCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12372))

Included in the following conference series:

Abstract

Hyperspectral unmixing is an important remote sensing task with applications including material identification and analysis. Characteristic spectral features make many pure materials identifiable from their visible-to-infrared spectra, but quantifying their presence within a mixture is a challenging task due to nonlinearities and factors of variation. In this paper, spectral variation is considered from a physics-based approach and incorporated into an end-to-end spectral unmixing algorithm via differentiable programming. The dispersion model is introduced to simulate realistic spectral variation, and an efficient method to fit the parameters is presented. Then, this dispersion model is utilized as a generative model within an analysis-by-synthesis spectral unmixing algorithm. Further, a technique for inverse rendering using a convolutional neural network to predict parameters of the generative model is introduced to enhance performance and speed when training data is available. Results achieve state-of-the-art on both infrared and visible-to-near-infrared (VNIR) datasets, and show promise for the synergy between physics-based models and deep learning in hyperspectral unmixing in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Azinovic, D., Li, T.M., Kaplanyan, A., Niessner, M.: Inverse path tracing for joint material and lighting estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2447–2456 (2019)

    Google Scholar 

  2. Bandfield, J.L.: Global mineral distributions on mars. J. Geophys. Res. Planets 107(E6), 1–9 (2002)

    Article  Google Scholar 

  3. Bateson, C.A., Asner, G.P., Wessman, C.A.: Endmember bundles: a new approach to incorporating endmember variability into spectral mixture analysis. IEEE Trans. Geosci. Remote Sens. 38(2), 1083–1094 (2000)

    Article  Google Scholar 

  4. Baydin, A.G., Pearlmutter, B.A., Radul, A.A., Siskind, J.M.: Automatic differentiation in machine learning: a survey. J. Mach. Learn. Res. 18(1), 5595–5637 (2017)

    MathSciNet  MATH  Google Scholar 

  5. Bhojanapalli, S., Kyrillidis, A., Sanghavi, S.: Dropping convexity for faster semi-definite optimization. In: Conference on Learning Theory, pp. 530–582 (2016)

    Google Scholar 

  6. Borsoi, R.A., et al.: Spectral variability in hyperspectral data unmixing: a comprehensive review. arXiv preprint arXiv:2001.07307 (2020)

  7. Burns, R.G.: Crystal field spectra and evidence of cation ordering in olivine minerals. Am. Mineral. J. Earth Planet. Mater. 55(9–10), 1608–1632 (1970)

    Google Scholar 

  8. Chang, C.I.: Hyperspectral Imaging: Techniques for Spectral Detection and Classification, vol. 1. Springer, New York (2003). https://doi.org/10.1007/978-1-4419-9170-6

    Book  Google Scholar 

  9. Chen, F., Zhang, Y.: Sparse hyperspectral unmixing based on constrained lp-l 2 optimization. IEEE Geosci. Remote Sens. Lett. 10(5), 1142–1146 (2013)

    Article  Google Scholar 

  10. Chen, Y., Jiang, H., Li, C., Jia, X., Ghamisi, P.: Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Trans. Geosci. Remote Sens. 54(10), 6232–6251 (2016)

    Article  Google Scholar 

  11. Cheng, G., Li, Z., Han, J., Yao, X., Guo, L.: Exploring hierarchical convolutional features for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 56(11), 6712–6722 (2018)

    Article  Google Scholar 

  12. Chouzenoux, E., Legendre, M., Moussaoui, S., Idier, J.: Fast constrained least squares spectral unmixing using primal-dual interior-point optimization. IEEE J. Sel. Top. Appl. Earth Observations Remote Sens. 7(1), 59–69 (2014)

    Article  Google Scholar 

  13. Christensen, P., et al.: Mars global surveyor thermal emission spectrometer experiment: investigation description and surface science results. J. Geophys. Res. Planets 106(E10), 23823–23871 (2001)

    Article  Google Scholar 

  14. Christensen, P.R., et al.: Lauretta: The OSIRIS-REx thermal emission spectrometer (OTES) instrument. Space Sci. Rev. 214(5), 87 (2018)

    Article  Google Scholar 

  15. Combe, J.P., et al.: Analysis of OMEGA/mars express data hyperspectral data using a multiple-endmember linear spectral unmixing model (MELSUM): methodology and first results. Planet. Space Sci. 56(7), 951–975 (2008)

    Article  Google Scholar 

  16. Du, X., Zare, A., Gader, P., Dranishnikov, D.: Spatial and spectral unmixing using the beta compositional model. IEEE J. Sel. Top. Appl. Earth Observations Remote Sens. 7(6), 1994–2003 (2014)

    Article  Google Scholar 

  17. Duchi, J.C., Ruan, F.: Solving (most) of a set of quadratic equalities: composite optimization for robust phase retrieval. Inf. Infer. J. IMA 8(3), 471–529 (2019)

    MathSciNet  Google Scholar 

  18. Engel, J., Hantrakul, L.H., Gu, C., Roberts, A.: DDSP: differentiable digital signal processing. In: International Conference on Learning Representations (2020)

    Google Scholar 

  19. Feely, K.C., Christensen, P.R.: Quantitative compositional analysis using thermal emission spectroscopy: application to igneous and metamorphic rocks. J. Geophys. Res. Planets 104(E10), 24195–24210 (1999)

    Article  Google Scholar 

  20. Gader, P., Zare, A., Close, R., Aitken, J., Tuell, G.: MUUFL Gulfport hyperspectral and LIDAR airborne data set. Tech. rep. REP-2013-57. University Florida, Gainesville, FL, USA0 (2013)

    Google Scholar 

  21. Gkioulekas, I., Levin, A., Zickler, T.: An evaluation of computational imaging techniques for heterogeneous inverse scattering. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 685–701. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_42

    Chapter  Google Scholar 

  22. Gkioulekas, I., Zhao, S., Bala, K., Zickler, T., Levin, A.: Inverse volume rendering with material dictionaries. ACM Trans. Graph. (TOG) 32(6), 162 (2013)

    Article  Google Scholar 

  23. Goudge, T.A., Mustard, J.F., Head, J.W., Salvatore, M.R., Wiseman, S.M.: Integrating CRISM and TES hyperspectral data to characterize a halloysite-bearing deposit in Kashira crater, mars. Icarus 250, 165–187 (2015)

    Article  Google Scholar 

  24. Heinz, D.C., et al.: Fully constrained least squares linear spectral mixture analysis method for material quantification in hyperspectral imagery. IEEE Trans. Geosci. Remote Sens. 39(3), 529–545 (2001)

    Article  Google Scholar 

  25. Heylen, R., Parente, M., Gader, P.: A review of nonlinear hyperspectral unmixing methods. IEEE J. Sel. Top. Appl. Earth Observations Remote Sens. 7(6), 1844–1868 (2014)

    Article  Google Scholar 

  26. Hu, W., Huang, Y., Wei, L., Zhang, F., Li, H.: Deep convolutional neural networks for hyperspectral image classification. J. Sens. 2015, 12 (2015)

    Article  Google Scholar 

  27. Jain, P., Kar, P., et al.: Non-convex optimization for machine learning. Found. Trends Mach. Learn. 10(3–4), 142–336 (2017)

    Article  MATH  Google Scholar 

  28. Keshava, N., Mustard, J.F.: Spectral unmixing. IEEE Signal Process. Mag. 19(1), 44–57 (2002)

    Article  Google Scholar 

  29. Klingelhöfer, G., et al.: Jarosite and hematite at Meridiani planum from opportunity’s mössbauer spectrometer. Science 306(5702), 1740–1745 (2004)

    Article  Google Scholar 

  30. Larkin, P.: Infrared and Raman Spectroscopy: Principles and Spectral Interpretation. Elsevier (2017)

    Google Scholar 

  31. Lee, J.D., Simchowitz, M., Jordan, M.I., Recht, B.: Gradient descent converges to minimizers. arXiv preprint arXiv:1602.04915 (2016)

  32. Lee, S., Tien, C.: Optical constants of soot in hydrocarbon flames. In: Symposium (International) on Combustion, vol. 18, pp. 1159–1166. Elsevier (1981)

    Google Scholar 

  33. Li, S., Song, W., Fang, L., Chen, Y., Ghamisi, P., Benediktsson, J.A.: Deep learning for hyperspectral image classification: an overview. IEEE Trans. Geosci. Remote Sens. 57(9), 6690–6709 (2019)

    Article  Google Scholar 

  34. Li, T.M., Aittala, M., Durand, F., Lehtinen, J.: Differentiable monte carlo ray tracing through edge sampling. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37(6), 222:1–222:11 (2018)

    Google Scholar 

  35. Li, Y., Zhang, H., Shen, Q.: Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network. Remote Sens. 9(1), 67 (2017)

    Article  Google Scholar 

  36. Liu, P., Zhang, H., Eom, K.B.: Active deep learning for classification of hyperspectral images. IEEE J. Sel. Top. Appl. Earth Observations Remote Sens. 10(2), 712–724 (2016)

    Article  Google Scholar 

  37. Loubet, G., Holzschuch, N., Jakob, W.: Reparameterizing discontinuous integrands for differentiable rendering. Trans. Graph. (Proc. SIGGRAPH Asia) 38(6) (2019). https://doi.org/10.1145/3355089.3356510

  38. Lu, G., Fei, B.: Medical hyperspectral imaging: a review. J. Biomed. Opt. 19(1), 010901 (2014)

    Article  Google Scholar 

  39. Moersch, J., Christensen, P.R.: Thermal emission from particulate surfaces: a comparison of scattering models with measured spectra. J. Geophys. Res. Planets 100(E4), 7465–7477 (1995)

    Article  Google Scholar 

  40. Nimier-David, M., Vicini, D., Zeltner, T., Jakob, W.: Mitsuba 2: a retargetable forward and inverse renderer. Trans. Graph. (Proc. SIGGRAPH Asia) 38(6) (2019). https://doi.org/10.1145/3355089.3356498

  41. Paszke, A., et al.: Automatic differentiation in pytorch (2017)

    Google Scholar 

  42. Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, pp. 8024–8035 (2019)

    Google Scholar 

  43. Ramsey, M.S., Christensen, P.R.: Mineral abundance determination: quantitative deconvolution of thermal emission spectra. J. Geophys. Res. Solid Earth 103(B1), 577–596 (1998)

    Article  Google Scholar 

  44. Ramsey, M.S., Christensen, P.R.: Mineral abundance determination: Quantitative deconvolution of thermal emission spectra: application to analysis of martian atmospheric particulates. J. Geophys. Res. Solid Earth 103, 577–596 (2000)

    Article  Google Scholar 

  45. Ravi, N., et al.: Pytorch3D. https://github.com/facebookresearch/pytorch3d (2020)

  46. Roberts, D.A., Gardner, M., Church, R., Ustin, S., Scheer, G., Green, R.: Mapping chaparral in the santa monica mountains using multiple endmember spectral mixture models. Remote Sens. Environ. 65(3), 267–279 (1998)

    Article  Google Scholar 

  47. Rogers, A., Aharonson, O.: Mineralogical composition of sands in meridiani planum determined from mars exploration rover data and comparison to orbital measurements. J. Geophys. Res. Planets 113, E6 (2008)

    Article  Google Scholar 

  48. Salisbury, J.W., D’Aria, D.M., Sabins Jr., F.F.: Thermal infrared remote sensing of crude oil slicks. Remote Sens. Environ. 45(2), 225–231 (1993)

    Article  Google Scholar 

  49. Spitzer, W., Kleinman, D.: Infrared lattice bands of quartz. Phys. Rev. 121(5), 1324 (1961)

    Article  Google Scholar 

  50. Stein, D.: Application of the normal compositional model to the analysis of hyperspectral imagery. In: IEEE Workshop on Advances in Techniques for Analysis of Remotely Sensed Data, 2003, pp. 44–51. IEEE (2003)

    Google Scholar 

  51. Sunshine, J.M., Pieters, C.M.: Determining the composition of olivine from reflectance spectroscopy. J. Geophys. Res. Planets 103(E6), 13675–13688 (1998)

    Article  Google Scholar 

  52. Tsai, C.Y., Sankaranarayanan, A.C., Gkioulekas, I.: Beyond volumetric albedo – a surface optimization framework for non-line-of-sight imaging. In: IEEE International Conference Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  53. Wang, F., Decker, J., Wu, X., Essertel, G., Rompf, T.: Backpropagation with callbacks: foundations for efficient and expressive differentiable programming. In: Advances in Neural Information Processing Systems, pp. 10180–10191 (2018)

    Google Scholar 

  54. Wang, F., Zheng, D., Decker, J., Wu, X., Essertel, G.M., Rompf, T.: Demystifying differentiable programming: shift/reset the penultimate backpropagator. In: Proceedings of the ACM on Programming Languages, vol. 3, no. ICFP, pp. 1–31 (2019)

    Google Scholar 

  55. Wenrich, M.L., Christensen, P.R.: Optical constants of minerals derived from emission spectroscopy: application to quartz. J. Geophys. Res. Solid Earth 101(B7), 15921–15931 (1996)

    Article  Google Scholar 

  56. Yang, X., Ye, Y., Li, X., Lau, R.Y., Zhang, X., Huang, X.: Hyperspectral image classification with deep learning models. IEEE Trans. Geosci. Remote Sens. 56(9), 5408–5423 (2018)

    Article  Google Scholar 

  57. Yu, Y., Smith, W.A.: InverseRenderNet: learning single image inverse rendering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3155–3164 (2019)

    Google Scholar 

  58. Zare, A., Gader, P.: Sparsity promoting iterated constrained endmember detection in hyperspectral imagery. IEEE Geosci. Remote Sens. Lett. 4(3), 446–450 (2007)

    Article  Google Scholar 

  59. Zare, A., Gader, P.: Hyperspectral band selection and endmember detection using sparsity promoting priors. IEEE Geosci. Remote Sens. Lett. 5(2), 256–260 (2008)

    Article  Google Scholar 

  60. Zare, A., Ho, K.: Endmember variability in hyperspectral analysis: addressing spectral variability during spectral unmixing. IEEE Signal Process. Mag. 31(1), 95–104 (2013)

    Article  Google Scholar 

  61. Zhang, C., Wu, L., Zheng, C., Gkioulekas, I., Ramamoorthi, R., Zhao, S.: A differential theory of radiative transfer. ACM Trans. Graph. (TOG) 38(6), 1–16 (2019)

    Google Scholar 

  62. Zhang, L., Zhang, L., Du, B.: Deep learning for remote sensing data: a technical tutorial on the state of the art. IEEE Geosci. Remote Sens. Mag. 4(2), 22–40 (2016)

    Article  Google Scholar 

  63. Zhang, S., Li, J., Li, H.C., Deng, C., Plaza, A.: Spectral-spatial weighted sparse regression for hyperspectral image unmixing. IEEE Trans. Geosci. Remote Sens. 56(6), 3265–3276 (2018)

    Article  Google Scholar 

  64. Zhao, W., Du, S.: Spectral-spatial feature extraction for hyperspectral image classification: a dimension reduction and deep learning approach. IEEE Trans. Geosci. Remote Sens. 54(8), 4544–4554 (2016)

    Article  Google Scholar 

  65. Zhou, Y., Rangarajan, A., Gader, P.D.: A Gaussian mixture model representation of endmember variability in hyperspectral unmixing. IEEE Trans. Image Process. 27(5), 2242–2256 (2018)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by NSF grant IIS-1909192 as well as GPU resources from ASU Research Computing. We thank Dr. Alina Zare, Christopher Haberle, and Dr. Deanna Rogers for their helpful discussions, and Kim Murray (formerly Kim Feely) for providing the laboratory measurements and analysis contributing to this paper.

Author information

Authors and Affiliations

  1. Arizona State University, Tempe, USA

    John Janiczek, Parth Thaker, Gautam Dasarathy, Philip Christensen & Suren Jayasuriya

  2. Northern Arizona University, Flagstaff, USA

    Christopher S. Edwards

Authors
  1. John Janiczek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Parth Thaker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gautam Dasarathy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christopher S. Edwards
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Philip Christensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Suren Jayasuriya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Suren Jayasuriya .

Editor information

Editors and Affiliations

  1. University of Oxford, Oxford, UK

    Andrea Vedaldi

  2. Graz University of Technology, Graz, Austria

    Horst Bischof

  3. University of Freiburg, Freiburg im Breisgau, Germany

    Thomas Brox

  4. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Jan-Michael Frahm

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 28243 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Janiczek, J., Thaker, P., Dasarathy, G., Edwards, C.S., Christensen, P., Jayasuriya, S. (2020). Differentiable Programming for Hyperspectral Unmixing Using a Physics-Based Dispersion Model. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12372. Springer, Cham. https://doi.org/10.1007/978-3-030-58583-9_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58583-9_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58582-2

  • Online ISBN: 978-3-030-58583-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation