Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Autonomous reconstruction of unknown indoor scenes guided by time-varying tensor fields

Published: 20 November 2017 Publication History

Abstract

Autonomous reconstruction of unknown scenes by a mobile robot inherently poses the question of balancing between exploration efficacy and reconstruction quality. We present a navigation-by-reconstruction approach to address this question, where moving paths of the robot are planned to account for both global efficiency for fast exploration and local smoothness to obtain high-quality scans. An RGB-D camera, attached to the robot arm, is dictated by the desired reconstruction quality as well as the movement of the robot itself. Our key idea is to harness a time-varying tensor field to guide robot movement, and then solve for 3D camera control under the constraint of the 2D robot moving path. The tensor field is updated in real time, conforming to the progressively reconstructed scene. We show that tensor fields are well suited for guiding autonomous scanning for two reasons: first, they contain sparse and controllable singularities that allow generating a locally smooth robot path, and second, their topological structure can be used for globally efficient path routing within a partially reconstructed scene. We have conducted numerous tests with a mobile robot, and demonstrate that our method leads to a smooth exploration and high-quality reconstruction of unknown indoor scenes.

Supplementary Material

ZIP File (a202-xu.zip)
Supplemental material.
MP4 File (a202-xu.mp4)

References

[1]
Sameer Agarwal, Noah Snavely, Steven M Seitz, and Richard Szeliski. 2010. Bundle adjustment in the large. In European conference on computer vision. Springer, 29--42.
[2]
Shi Bai, Jinkun Wang, Fanfei Chen, and Brendan Englot. 2016. Information-theoretic exploration with Bayesian optimization. In Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on. IEEE, 1816--1822.
[3]
Xiang Bai and Longin Jan Latecki. 2008. Path similarity skeleton graph matching. IEEE T-PAMI 30, 7 (2008), 1282--1292.
[4]
Harry Blum. 1967. A Transformation for Extracting New Descriptors of Shape. In Models for the Perception of Speech and Visual Form, Weiant Wathen-Dunn (Ed.). MIT Press, 362--380.
[5]
Johann Borenstein and Yoram Koren. 1989. Real-time obstacle avoidance for fast mobile robots. IEEE Trans. Sys., Man, and Cyber. 19, 5 (1989), 1179--1187.
[6]
Benjamin Charrow, Gregory Kahn, Sachin Patil, Sikang Liu, Ken Goldberg, Pieter Abbeel, Nathan Michael, and Vijay Kumar. 2015. Information-theoretic planning with trajectory optimization for dense 3D mapping. In Proceedings of Robotics: Science and Systems.
[7]
Guoning Chen, Gregory Esch, Peter Wonka, Pascal Müller, and Eugene Zhang. 2008. Interactive procedural street modeling. In ACM Trans. on Graph. (SIGGRAPH), Vol. 27. 103.
[8]
Guoning Chen, Vivek Kwatra, Li-Yi Wei, Charles D Hansen, and Eugene Zhang. 2012. Design of 2d time-varying vector fields. IEEE Trans. Vis. & Computer Graphics 18, 10 (2012), 1717--1730.
[9]
Jiawen Chen, Dennis Bautembach, and Shahram Izadi. 2013. Scalable real-time volumetric surface reconstruction. ACM Trans. on Graph. (SIGGRAPH) 32, 4 (2013), 113.
[10]
Sungjoon Choi, Qian-Yi Zhou, and Vladlen Koltun. 2015. Robust reconstruction of indoor scenes. In Proc. CVPR. 5556--5565.
[11]
Angela Dai, Matthias Nießner, Michael Zollöfer, Shahram Izadi, and Christian Theobalt. 2016. BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration. arXiv preprint arXiv:1604.01093 (2016).
[12]
Thierry Delmarcelle and Lambertus Hesselink. 1994. The topology of symmetric, second-order tensor fields. In Proc. IEEE Visualization. 140--147.
[13]
Tamal K Dey. 2006. Curve and surface reconstruction: algorithms with mathematical analysis. Vol. 23. Cambridge University Press.
[14]
Herbert Edelsbrunner, David Letscher, and Afra Zomorodian. 2002. Topological persistence and simplification. Discrete and Computational Geometry 28, 4 (2002), 511--533.
[15]
Xinyi Fan, Linguang Zhang, Benedict Brown, and Szymon Rusinkiewicz. 2016. Automated View and Path Planning for Scalable Multi-Object 3D Scanning. ACM Trans. on Graph. (SIGGRAPH Asia) 35, 6 (2016), 239.
[16]
Fetch. 2016. Fetch Robotics. http://fetchrobotics.com/research. (2016).
[17]
Gurobi. 2016. Gurobi Solver. http://www.gurobi.com/LP. (2016).
[18]
Aaron Hertzmann and Denis Zorin. 2000. Illustrating smooth surfaces. In Proc. of SIGGRAPH. 517--526.
[19]
Armin Hornung, Kai M. Wurm, Maren Bennewitz, Cyrill Stachniss, and Wolfram Burgard. 2013. OctoMap: An Efficient Probabilistic 3D Mapping Framework Based on Octrees. Autonomous Robots 34, 3 (2013), 189--206.
[20]
Oussama Khatib. 1986. Real-time obstacle avoidance for manipulators and mobile robots. Int. J. Robotics Research 5, 1 (1986), 90--98.
[21]
Thomas Kollar and Nicholas Roy. 2008. Trajectory optimization using reinforcement learning for map exploration. Int. J. Robotics Research 27, 2 (2008), 175--196.
[22]
Yoram Koren and Johann Borenstein. 1991. Potential field methods and their inherent limitations for mobile robot navigation. In Proc. ICRA. 1398--1404.
[23]
Michael Krainin, Brian Curless, and Dieter Fox. 2011. Autonomous Generation of Complete 3D Object Models Using Next Best View Manipulation Planning. In Proc. ICRA. 5031--5037.
[24]
Simon Kriegel, Christian Rink, Tim Bodenmüller, Alexander Narr, Michael Suppa, and Gerd Hirzinger. 2012. Next-best-scan planning for autonomous 3D modeling. In Proc. IROS. 2850--2856.
[25]
Kok-Lim Low and Anselmo Lastra. 2006. An adaptive hierarchical next-best-view algorithm for 3d reconstruction of indoor scenes. In Proceedings of 14th Pacific Conference on Computer Graphics and Applications (Pacific Graphics 2006).
[26]
Richard A Newcombe, Andrew J Davison, Shahram Izadi, Pushmeet Kohli, Otmar Hilliges, Jamie Shotton, David Molyneaux, Steve Hodges, David Kim, and Andrew Fitzgibbon. 2011. KinectFusion: Real-time dense surface mapping and tracking. In Proc. IEEE Int. Symp. on Mixed and Augmented Reality. 127--136.
[27]
Matthias Nieser, Jonathan Palacios, Konrad Polthier, and Eugene Zhang. 2012. Hexagonal global parameterization of arbitrary surfaces. IEEE Transactions on Visualization and Computer Graphics 18, 6 (2012), 865--878.
[28]
M. Nießner, M. Zollhöfer, S. Izadi, and M. Stamminger. 2013. Real-time 3D Reconstruction at Scale using Voxel Hashing. ACM Trans. on Graph. (SIGGRAPH Asia) 32, 6 (2013), 169.
[29]
Kyel Ok, Sameer Ansari, Billy Gallagher, William Sica, Frank Dellaert, and Mike Stilman. 2013. Path planning with uncertainty: Voronoi uncertainty fields. In Proc. ICRA. 4596--4601.
[30]
Jonathan Palacios and Eugene Zhang. 2007. Rotational symmetry field design on surfaces. ACM Trans.on Graph. (SIGGRAPH) 26, 3 (2007), 55.
[31]
Manikandasriram Srinivasan Ramanagopal and Jerome Le Ny. 2016. Motion Planning Strategies for Autonomously Mapping 3D Structures. arXiv preprint arXiv:1602.06667 (2016).
[32]
Nicolas Ray, Bruno Vallet, Laurent Alonso, and Bruno Levy. 2009. Geometry-aware direction field processing. ACM Trans. on Graph. 29, 1 (2009), 1.
[33]
ROS. 2014. ROS Wiki. http://wiki.ros.org/. (2014).
[34]
Robbie Shade and Paul Newman.2011. Choosing where to go: Complete 3D exploration with stereo. In Proc. ICRA. 2806--2811.
[35]
S. Song, L. Zhang, and J. Xiao. 2015. Robot In a Room: Toward Perfect Object Recognition in Closed Environments. arXiv preprint arXiv:1507.02703 (2015).
[36]
Sebastian Thrun and others. 2002. Robotic mapping: A survey. Exploring artificial intelligence in the new millennium 1 (2002), 1--35.
[37]
Joan Vallvé and Juan Andrade-Cetto. 2015. Potential information fields for mobile robot exploration. Robotics and Autonomous Systems 69 (2015), 68--79.
[38]
Amir Vaxman, Marcel Campen, Olga Diamanti, Daniele Panozzo, David Bommes, Klaus Hildebrandt, and Mirela Ben-Chen. 2016. Directional Field Synthesis, Design, and Processing. Computer Graphics Forum 35, 2 (2016), 545--572.
[39]
Wenping Wang, Bert Jüttler, Dayue Zheng, and Yang Liu. 2008. Computation of rotation minimizing frames. ACM Trans. on Graph. 27, 1 (2008), 2.
[40]
Thomas Whelan, Stefan Leutenegger, Renato F Salas-Moreno, Ben Glocker, and Andrew J Davison. 2015. ElasticFusion: Dense SLAM without a pose graph. In Proc. Robotics: Science and Systems.
[41]
Melonee Wise, Michael Ferguson, Derek King, Eric Diehr, and David Dymesich. 2016. Fetch and freight: Standard platforms for service robot applications. In Workshop on Autonomous Mobile Service Robots.
[42]
Shihao Wu, Wei Sun, Pinxin Long, Hui Huang, Daniel Cohen-Or, Minglun Gong, Oliver Deussen, and Baoquan Chen. 2014. Quality-driven poisson-guided autoscanning. ACM Trans. on Graph. (SIGGRAPH Asia) 33, 6 (2014), 203.
[43]
Kai Xu, Hui Huang, Yifei Shi, Hao Li, Pinxin Long, Jiannong Caichen, Wei Sun, and Baoquan Chen. 2015. Autoscanning for Coupled Scene Reconstruction and Proactive Object Analysis. ACM Trans. on Graph. 34, 6 (2015), 177.
[44]
Kai Xu, Yifei Shi, Lintao Zheng, Junyu Zhang, Min Liu, Hui Huang, Hao Su, Daniel Cohen-Or, and Baoquan Chen. 2016. 3D Attention-Driven Depth Acquisition for Object Identification. ACM Transactions on Graphics (TOG) 35, 6 (2016), 238.
[45]
Kai Xu, Hao Zhang, Daniel Cohen-Or, and Yueshan Xiong. 2009. Dynamic harmonic fields for surface processing. Computers & Graphics 33, 3 (2009), 391--398.
[46]
Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, and Thomas Funkhouser. 2016. 3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions. arXiv preprint arXiv:1603.08182 (2016).
[47]
Eugene Zhang, James Hays, and Greg Turk. 2007. Interactive tensor field design and visualization on surfaces. IEEE Trans. Vis. & Computer Graphics 13, 1 (2007), 94--107.
[48]
Eugene Zhang, Konstantin Mischaikow, and Greg Turk. 2006. Vector field design on surfaces. ACM Trans. on Graph. 25, 4 (2006), 1294--1326.
[49]
Yizhong Zhang, Weiwei Xu, Yiying Tong, and Kun Zhou. 2014. Online Structure Analysis for Real-time Indoor Scene Reconstruction. ACM Trans. on Graph. 34, 5 (2014), 159.

Cited By

View all
  • (2024)X-SLAM: Scalable Dense SLAM for Task-aware Optimization using CSFDACM Transactions on Graphics10.1145/365823343:4(1-15)Online publication date: 19-Jul-2024
  • (2024)Asymptotic Topology of 3D Linear Symmetric Tensor Fields2024 IEEE Topological Data Analysis and Visualization (TopoInVis)10.1109/TopoInVis64104.2024.00010(55-64)Online publication date: 13-Oct-2024
  • (2024)Skeleton Disk-Graph Roadmap: A Sparse Deterministic Roadmap for Safe 2D Navigation and ExplorationIEEE Robotics and Automation Letters10.1109/LRA.2023.33341039:1(555-562)Online publication date: Jan-2024
  • Show More Cited By

Index Terms

  1. Autonomous reconstruction of unknown indoor scenes guided by time-varying tensor fields

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Graphics
    ACM Transactions on Graphics  Volume 36, Issue 6
    December 2017
    973 pages
    ISSN:0730-0301
    EISSN:1557-7368
    DOI:10.1145/3130800
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 20 November 2017
    Published in TOG Volume 36, Issue 6

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. autonomous reconstruction
    2. camera trajectory optimization
    3. indoor scene reconstruction
    4. path planning
    5. tensor fields

    Qualifiers

    • Research-article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)48
    • Downloads (Last 6 weeks)4
    Reflects downloads up to 14 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)X-SLAM: Scalable Dense SLAM for Task-aware Optimization using CSFDACM Transactions on Graphics10.1145/365823343:4(1-15)Online publication date: 19-Jul-2024
    • (2024)Asymptotic Topology of 3D Linear Symmetric Tensor Fields2024 IEEE Topological Data Analysis and Visualization (TopoInVis)10.1109/TopoInVis64104.2024.00010(55-64)Online publication date: 13-Oct-2024
    • (2024)Skeleton Disk-Graph Roadmap: A Sparse Deterministic Roadmap for Safe 2D Navigation and ExplorationIEEE Robotics and Automation Letters10.1109/LRA.2023.33341039:1(555-562)Online publication date: Jan-2024
    • (2024)Autonomous view planning methods for 3D scanningAutomation in Construction10.1016/j.autcon.2024.105291160(105291)Online publication date: Apr-2024
    • (2024)THP: Tensor-field-driven hierarchical path planning for autonomous scene exploration with depth sensorsComputational Visual Media10.1007/s41095-022-0312-610:6(1121-1135)Online publication date: 18-May-2024
    • (2023)Online Scene CAD Recomposition via Autonomous ScanningACM Transactions on Graphics10.1145/361833942:6(1-16)Online publication date: 5-Dec-2023
    • (2023)ScanBot: Autonomous Reconstruction via Deep Reinforcement LearningACM Transactions on Graphics10.1145/359211342:4(1-16)Online publication date: 26-Jul-2023
    • (2023)Virtual Reality Solutions Employing Artificial Intelligence Methods: A Systematic Literature ReviewACM Computing Surveys10.1145/356502055:10(1-29)Online publication date: 2-Feb-2023
    • (2023)Active scene reconstruction via self-rotation driven by optimized information theoryFourteenth International Conference on Graphics and Image Processing (ICGIP 2022)10.1117/12.2680387(125)Online publication date: 27-Jun-2023
    • (2023)Where to Render: Studying Renderability for IBR of Large-Scale Scenes2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR55154.2023.00051(356-366)Online publication date: Mar-2023
    • Show More Cited By

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media