Abstract
Recent work in robotic manipulation focuses on object retrieval in cluttered spaces under occlusion. Nevertheless, the majority of efforts lack an analysis of conditions for the completeness of the approaches or the methods apply only when objects can be removed from the workspace. This work formulates the general, occlusion-aware manipulation task, and focuses on safe object reconstruction in a confined space with in-place rearrangement. It proposes a framework that ensures safety with completeness guarantees. Furthermore, an algorithm, which is an instantiation of this abstract framework for monotone instances is developed and evaluated empirically by comparing against a random and a greedy baseline on randomly generated experiments in simulation. Even for cluttered scenes with realistic objects, the proposed algorithm significantly outperforms the baselines and maintains a high success rate across experimental conditions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Code and videos: https://sites.google.com/scarletmail.rutgers.edu/occlusion-manipulation.
- 2.
Please refer to the Appendix for an analysis of object extraction. Appendix can be found at: https://arxiv.org/abs/2205.11719.
- 3.
The proof can be found in the Appendix: https://arxiv.org/abs/2205.11719.
- 4.
See Appendix for a detailed discussion: https://arxiv.org/abs/2205.11719.
- 5.
In-place relocation refers to rearrangement of objects within the workspace, i.e., not utilizing buffer space for the rearrangement that is external to the workspace.
- 6.
The details are included in the Appendix: https://arxiv.org/abs/2205.11719.
- 7.
Code and videos: https://sites.google.com/scarletmail.rutgers.edu/occlusion-manipulation.
References
J. Ahn, J. Lee, S. H. Cheong, C. Kim, and C. Nam. An integrated approach for determining objects to be relocated and their goal positions inside clutter for object retrieval. In ICRA, pages 6408–6414, 2021
W. Bejjani, W. C. Agboh, M. R. Dogar, and M. Leonetti. Occlusion-aware search for object retrieval in clutter. In IROS, pages 4678–4685, 2021
Bohg, J., Hausman, K., Sankaran, B., Brock, O., Kragic, D., Schaal, S., Sukhatme, G.S.: Interactive perception: Leveraging action in perception and perception in action. IEEE TRO 33(6), 1273–1291 (2017)
B. Calli, A. Singh, A. Walsman, S. Srinivasa, P. Abbeel, and A. M. Dollar. The ycb object and model set: Towards common benchmarks for manipulation research. In ICAR, pages 510–517, 2015
S. H. Cheong, B. Y. Cho, J. Lee, C. Kim, and C. Nam. Where to relocate?: Object rearrangement inside cluttered and confined environments for robotic manipulation. In ICRA, pages 7791–7797, 2020
D. Coleman, I. Sucan, S. Chitta, and N. Correll. Reducing the barrier to entry of complex robotic software: a moveit! case study. arXiv:1404.3785, 2014
E. Coumans and Y. Bai. Pybullet, a python module for physics simulation for games, robotics and machine learning. http://pybullet.org, 2016-2021
M. Danielczuk, A. Kurenkov, A. Balakrishna, M. Matl, D. Wang, R. Martín-Martín, A. Garg, S. Savarese, and K. Goldberg. Mechanical search: Multi-step retrieval of a target object occluded by clutter. In ICRA, pages 1614–1621, 2019
E. D. Demaine, S. P. Fekete, and R. J. Lang. Circle packing for origami design is hard. arXiv:1008.1224, 2010
Dogar, M.R., Koval, M.C., Tallavajhula, A., Srinivasa, S.S.: Object search by manipulation. Autonomous Robots 36(1), 153–167 (2014)
K. Gao, D. Lau, B. Huang, K. E. Bekris, and J. Yu. Fast high-quality tabletop rearrangement in bounded workspace. In ICRA, 2022
Garrett, C.R., Chitnis, R., Holladay, R., Kim, B., Silver, T., Kaelbling, L.P., Lozano-Pérez, T.: Integrated task and motion planning. Annual Review of Control, Robotics, and Autonomous Systems 4, 265–293 (2021)
C. R. Garrett, C. Paxton, T. Lozano-Pérez, L. P. Kaelbling, and D. Fox. Online replanning in belief space for partially observable task and motion problems. In ICRA, pages 5678–5684, 2020
M. Gupta, T. Rühr, M. Beetz, and G. S. Sukhatme. Interactive environment exploration in clutter. In IROS, pages 5265–5272, 2013
H. Huang, M. Danielczuk, C. M. Kim, L. Fu, Z. Tam, J. Ichnowski, A. Angelova, B. Ichter, and K. Goldberg. Mechanical search on shelves using a novel "bluction" tool. In 2022 International Conference on Robotics and Automation (ICRA), pages 6158–6164. IEEE, 2022
H. Huang, M. Dominguez-Kuhne, V. Satish, M. Danielczuk, K. Sanders, J. Ichnowski, A. Lee, A. Angelova, V. Vanhoucke, and K. Goldberg. Mechanical search on shelves using lateral access x-ray. In IROS, pages 2045–2052, 2020
K. Huang and T. Hermans. Building 3d object models during manipulation by reconstruction-aware trajectory optimization. arXiv:1905.03907, 2019
K. N. Kumar, I. Essa, and S. Ha. Graph-based cluttered scene generation and interactive exploration using deep reinforcement learning. In ICRA, 2022
A. Kurenkov, J. Taglic, R. Kulkarni, M. Dominguez-Kuhne, A. Garg, R. Martín-Martín, and S. Savarese. Visuomotor mechanical search: Learning to retrieve target objects in clutter. In IROS, pages 8408–8414, 2020
J. K. Li, D. Hsu, and W. S. Lee. Act to see and see to act: Pomdp planning for objects search in clutter. In IROS, pages 5701–5707, 2016
Liu, Z., Liu, W., Qin, Y., Xiang, F., Gou, M., Xin, S., Roa, M.A., Calli, B., Su, H., Sun, Y., et al.: Ocrtoc: A cloud-based competition and benchmark for robotic grasping and manipulation. IEEE RA-L 7(1), 486–493 (2021)
S. Lu, R. Wang, Y. Miao, C. Mitash, and K. Bekris. Online model reconstruction and reuse for lifelong improvement of robot manipulation. In ICRA, 2022
Mitash, C., Shome, R., Wen, B., Boularias, A., Bekris, K.: Task-driven perception and manipulation for constrained placement of unknown objects. IEEE RA-L 5(4), 5605–5612 (2020)
Nam, C., Cheong, S.H., Lee, J., Kim, D.H., Kim, C.: Fast and resilient manipulation planning for object retrieval in cluttered and confined environments. IEEE TRO 37(5), 1539–1552 (2021)
R. A. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A. J. Davison, P. Kohi, J. Shotton, S. Hodges, and A. Fitzgibbon. Kinectfusion: Real-time dense surface mapping and tracking. In ISMAR, pages 127–136, 2011
T. Novkovic, R. Pautrat, F. Furrer, M. Breyer, R. Siegwart, and J. Nieto. Object finding in cluttered scenes using interactive perception. In ICRA, 2020
A. Price, L. Jin, and D. Berenson. Inferring occluded geometry improves performance when retrieving an object from dense clutter. In ISRR, 2019
A. H. Qureshi, A. Mousavian, C. Paxton, M. C. Yip, and D. Fox. Nerp: Neural rearrangement planning for unknown objects. In RSS, 2021
M. Stilman, J.-U. Schamburek, J. Kuffner, and T. Asfour. Manipulation planning among movable obstacles. In ICRA, pages 3327–3332, 2007
R. Wang, K. Gao, J. Yu, and K. Bekris. Lazy rearrangement planning in confined spaces. In ICAPS, 2022
R. Wang, Y. Miao, and K. E. Bekris. Efficient and high-quality prehensile rearrangement in cluttered and confined spaces. In ICRA, 2022
R. Wang, C. Mitash, S. Lu, D. Boehm, and K. E. Bekris. Safe and effective picking paths in clutter given discrete distributions of object poses. In IROS, 2020
Y. Xiao, S. Katt, A. ten Pas, S. Chen, and C. Amato. Online planning for target object search in clutter under partial observability. In ICRA, 2019
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary material 1 (mp4 6199 KB)
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Miao, Y., Wang, R., Bekris, K. (2023). Safe, Occlusion-Aware Manipulation for Online Object Reconstruction in Confined Spaces. In: Billard, A., Asfour, T., Khatib, O. (eds) Robotics Research. ISRR 2022. Springer Proceedings in Advanced Robotics, vol 27. Springer, Cham. https://doi.org/10.1007/978-3-031-25555-7_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-25555-7_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25554-0
Online ISBN: 978-3-031-25555-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)