Abstract
Non-line-of-sight (NLOS) imaging techniques use light that diffusely reflects off of visible surfaces (e.g., walls) to see around corners. One approach involves using pulsed lasers and ultrafast sensors to measure the travel time of multiply scattered light. Unlike existing NLOS techniques that generally require densely raster scanning points across the entirety of a relay wall, we explore a more efficient form of NLOS scanning that reduces both acquisition times and computational requirements. We propose a circular and confocal non-line-of-sight (\(\text {C}^2\text {NLOS}\)) scan that involves illuminating and imaging a common point, and scanning this point in a circular path along a wall. We observe that (1) these \(\text {C}^2\text {NLOS}\) measurements consist of a superposition of sinusoids, which we refer to as a transient sinogram, (2) there exists computationally efficient reconstruction procedures that transform these sinusoidal measurements into 3D positions of hidden scatterers or NLOS images of hidden objects, and (3) despite operating on an order of magnitude fewer measurements than previous approaches, these \(\text {C}^2\text {NLOS}\) scans provide sufficient information about the hidden scene to solve these different NLOS imaging tasks. We show results from both simulated and real \(\text {C}^2\text {NLOS}\) scans (Project page: https://marikoisogawa.github.io/project/c2nlos).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Adib, F., Hsu, C.Y., Mao, H., Katabi, D., Durand, F.: Capturing the human figure through a wall. ACM Trans. Graph. (TOG) 34(6), 1–13 (2015)
Ahn, B., Dave, A., Veeraraghavan, A., Gkioulekas, I., Sankaranarayanan, A.C.: Convolutional approximations to the general non-line-of-sight imaging operator. In: IEEE International Conference on Computer Vision (ICCV), pp. 7888–7898 (2019)
Ballard, D.H.: Generalizing the Hough transform to fetect arbitrary shapes. In: Readings in Computer Vision, pp. 714–725 (1987)
Bouman, K.L., et al.: Turning corners into cameras: principles and methods. In: IEEE International Conference on Computer Vision (ICCV), pp. 2289–2297 (2017)
Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)
Buttafava, M., Zeman, J., Tosi, A., Eliceiri, K., Velten, A.: Non-line-of-sight imaging using a time-gated single photon avalanche diode. Opt. Express 23(16), 20997–21011 (2015)
Chan, S., Warburton, R.E., Gariepy, G., Leach, J., Faccio, D.: Non-line-of-sight tracking of people at long range. Opt. Express 25(9), 10109–10117 (2017)
Chandran, S., Jayasuriya, S.: Adaptive lighting for data-driven non-line-of-sight 3D localization and object identification. In: The British Machine Vision Conference (BMVC) (2019)
Galindo, M., Marco, J., O’Toole, M., Wetzstein, G., Gutierrez, D., Jarabo, A.: A dataset for benchmarking time-resolved non-line-of-sight imaging. In: ACM SIGGRAPH 2019 Posters, pp. 1–2 (2019)
Gupta, O., Willwacher, T., Velten, A., Veeraraghavan, A., Raskar, R.: Reconstruction of hidden 3D shapes using diffuse reflections. Opt. Express 20(17), 19096–19108 (2012)
Heide, F., Heidrich, W., Hullin, M.B.: Diffuse mirrors: 3D reconstruction from diffuse indirect illumination using inexpensive time-of-flight sens. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3222–3229 (2014)
Heide, F., O’Toole, M., Zang, K., Lindell, D.B., Diamond, S., Wetzstein, G.: Non-line-of-sight imaging with partial occluders and surface normals. ACM Trans. Graph. 38(3), 1–10 (2019)
Isogawa, M., Yuan, Y., O’Toole, M., Kitani, K.M.: Optical non-line-of-sight physics-based 3D human pose estimation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7013–7022 (2020)
Jarabo, A., Marco, J., Muñoz, A., Buisan, R., Jarosz, W., Gutierrez, D.: A framework for transient rendering. ACM Trans. Graph. (ToG) 33(6), 1–10 (2014)
Kak, A.C., Slaney, M., Wang, G.: Principles of computerized tomographic imaging. Med. Phys. 29(1), 107–107 (2002)
Kirmani, A., Hutchison, T., Davis, J., Raskar, R.: Looking around the corner using transient imaging. In: IEEE International Conference on Computer Vision (ICCV), pp. 159–166 (2009)
Klein, J., Peters, C., Laurenzis, M., Hullin, M.: Tracking objects outside the line of sight using 2D intensity images. Sci. Rep. 6(32491), 1–9 (2016)
Klein, J., Peters, C., Laurenzis, M., Hullin, M.: Non-line-of-sight MoCap. In: ACM SIGGRAPH Emerging Technologies, pp. 18:1–18:2 (2017)
La Manna, M., Kine, F., Breitbach, E., Jackson, J., Sultan, T., Velten, A.: Error backprojection algorithms for non-line-of-sight imaging. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1615–1626 (2019)
Li, T., Fan, L., Zhao, M., Liu, Y., Katabi, D.: Making the invisible visible: action recognition through walls and occlusions. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 872–881 (2019)
Lindell, D.B., Wetzstein, G., Koltun, V.: Acoustic non-line-of-sight imaging. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6780–6789 (2019)
Lindell, D.B., Wetzstein, G., O’Toole, M.: Wave-based non-line-of-sight imaging using fast F-K migration. ACM Trans. Graph. (TOG) 38(4), 116 (2019)
Maeda, T., Satat, G., Swedish, T., Sinha, L., Raskar, R.: Recent advances in imaging around corners. In: arXiv (2019)
Maeda, T., Wang, Y., Raskar, R., Kadambi, A.: Thermal non-line-of-sight imaging. In: IEEE International Conference on Computational Photography (ICCP), pp. 1–11 (2019)
Metzler, C.A., Lindell, D.B., Wetzstein, G.: Keyhole imaging: non-line-of-sight imaging and tracking of moving objects along a single optical path at long standoff distances. In: arXiv (2019)
O’Toole, M., Heide, F., Lindell, D.B., Zang, K., Diamond, S., Wetzstein, G.: Reconstructing transient images from single-photon sensors. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2289–2297 (2017)
O’Toole, M., Lindell, D.B., Wetzstein, G.: Confocal non-line-of-sight imaging based on the light-cone transform. Nature 555(7696), 338 (2018)
Pediredla, A., Dave, A., Veeraraghavan, A.: SNLOS: Non-line-of-sight scanning through temporal focusing. In: IEEE International Conference on Computational Photography (ICCP), pp. 1–13 (2019)
Redo-Sanchez, A., et al.: Terahertz time-gated spectral imaging for content extraction through layered structures. Nat. Commun. 7, 12665 (2016)
Saunders, C., Murray-Bruce, J., Goyal, V.K.: Computational periscopy with an ordinary digital camera. Nature 565(7740), 472–475 (2019)
Tancik, M., Satat, G., Raskar, R.: Flash photography for data-driven hidden scene recovery. In: arXiv (2018)
Tsai, C.Y., Sankaranarayanan, A.C., Gkioulekas, I.: Beyond volumetric albedo - a surface optimization framework for non-line-of-sight imaging. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Velten, A., Willwacher, T., Gupta, O., Veeraraghavan, A., Bawendi, M.G., Raskar, R.: Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun. 3, 745 (2012)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Xin, S., Nousias, S., Kutulakos, K.N., Sankaranarayanan, A.C., Narasimhan, S.G., Gkioulekas, I.: A theory of Fermat paths for non-line-of-sight shape reconstruction. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Zou, C.C., Ge, S.: A Hough transform-based method for fast detection of fixed period sinusoidal curves in images. In: International Conference on Signal Processing (ICSP), vol. 1, pp. 909–912 (2002)
Acknowledgements
We thank Ioannis Gkioulekas for helpful discussions and feedback on this work. M. Isogawa is supported by NTT Corporation. M. O’Toole is supported by the DARPA REVEAL program.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary material 1 (mp4 42570 KB)
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Isogawa, M., Chan, D., Yuan, Y., Kitani, K., O’Toole, M. (2020). Efficient Non-Line-of-Sight Imaging from Transient Sinograms. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12352. Springer, Cham. https://doi.org/10.1007/978-3-030-58571-6_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-58571-6_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58570-9
Online ISBN: 978-3-030-58571-6
eBook Packages: Computer ScienceComputer Science (R0)