Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3610419.3610506acmotherconferencesArticle/Chapter ViewAbstractPublication PagesairConference Proceedingsconference-collections
research-article

Deep Learning-based 6D pose estimation of texture less objects for Industrial Cobots

Published: 02 November 2023 Publication History

Abstract

Robotics and artificial intelligence have led to a significant improvement in the automation of various industrial processes. 6D pose estimation for industrial robotic arms refers to the process of accurately determining the position and orientation of an object in 3D space relative to a robotic arm, typically in an industrial or manufacturing context. This is important for tasks such as object recognition, grasping, and manipulation, where the robot needs to know the precise position and orientation of the object in order to perform its task but the accuracy of 6D pose estimation remains a challenge. To address this, researchers created deep learning algorithms that assess the posture of objects. However, these techniques still face significant challenges in real-world industrial applications, such as lighting conditions, occlusions, and the large number of objects that need to be recognized in a short amount of time. This research aims to present an efficient method of 6D pose estimation for texture less objects. Synthetic data generation provides better generalization of the real-world data sources, making it more time and cost-effective than collecting real-world data. The major challenge is bridging the reality gap between virtual and real environments, and the data must be photorealistic for it to be used in a real time model. Domain randomization and powerful simulators are employed to replicate a real environment in order to solve these issues.

References

[1]
Arayici, Yusuf & Hamilton, A & Gamito, Pedro. (2006). Modelling 3D Scanned Data to Visualise and Analyse the Built Environment for Regeneration. 17.
[2]
R. Kaskman, S. Zakharov, I. Shugurov and S. Ilic, "HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objects," 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea (South), 2019, pp. 2767-2776.
[3]
Hinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., Bradski, G., Konolige, K., Navab, N.: Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes. In: ACCV (2012)
[4]
T. Hodan, P. Haluza, Š. Obdržálek, J. Matas, M. Lourakis and X. Zabulis, "T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-Less Objects," 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), Santa Rosa, CA, USA, 2017, pp. 880-888.
[5]
F. Sadeghi and S. Levine. CAD2RL: Real single-image flight without a single real image. In Robotics: Science and Systems (RSS), 2017
[6]
Stefan Hinterstoisser, Vincent Lepetit, Paul Wohlhart, and Kurt Konolige. 2019. On Pre-trained Image Features and Synthetic Images for Deep Learning. In Computer Vision – ECCV 2018 Workshops: Munich, Germany, September 8-14, 2018, Proceedings, Part I. Springer-Verlag, Berlin, Heidelberg, 682–697. https://doi.org/10.1007/978-3-030-11009-3_42
[7]
J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba and P. Abbeel, "Domain randomization for transferring deep neural networks from simulation to the real world," 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 2017, pp. 23-30.
[8]
J. Tremblay, "Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA, 2018
[9]
Xiang, Yu & Schmidt, Tanner & Narayanan, Venkatraman & Fox, Dieter. (2018). PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes. 10.15607/RSS.2018.XIV.019.
[10]
C. Song, J. Song and Q. Huang, "HybridPose: 6D Object Pose Estimation Under Hybrid Representations," 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 428-437
[11]
Y. Lin, J. Tremblay, S. Tyree, P. A. Vela and S. Birchfield, "Single-Stage Keypoint- Based Category-Level Object Pose Estimation from an RGB Image," 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA, 2022, pp. 1547-1553.
[12]
Ali, Ihtisham & Suominen, & Gotchev, Atanas & Morales,. (2019). Methods for Simultaneous Robot-World-Hand–Eye Calibration: A Comparative Study. Sensors. 19. 2837. 10.3390/s19122837.
[13]
Dong, Mingshuai & Wei, Shimin & yu, Xiuli & Yin, Jianqin. (2021). Mask-GD Segmentation Based Robotic Grasp Detection.
[14]
Hu, Z., Hou, R., Niu, J., Yu, X., Ren, T., & Li, Q. (2021). Object Pose Estimation for Robotic Grasping based on Multi-view Keypoint Detection. In 2021 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom) (pp. 1295-1302).
[15]
Duan H, Wang P, Huang Y, Xu G, Wei W, Shen X. Robotics Dexterous Grasping: The Methods Based on Point Cloud and Deep Learning. Front Neurorobot. 2021;15 658280.
[16]
Francis, Anthony G., Brandon Kinman, Krista Ann Reymann, Laura Downs, Nathan Koenig, Ryan M. Hickman, Thomas B. McHugh, and Vincent Olivier Vanhoucke. "Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items." (2022).
[17]
Panerati, Jacopo & Zheng, Hehui & Zhou, Siqi & Xu, James & Prorok, Amanda & Schoellig, Angela. (2021). Learning to Fly—a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control.
[18]
Tremblay, Jonathan & To, Thang & Sundaralingam, Balakumar & Xiang, Yu & Fox, Dieter & Birchfield, Stan. (2018). Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects.
[19]
V. Lepetit, F. Moreno-Noguer, and P. Fua. EPnP: An accurate O(n) solution to the PnP problem. International Journal Computer Vision, 81(2), 2009.
[20]
S.-E. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh. Convolutional pose machines. In CVPR, 2016.
[21]
K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
[22]
Lin W, Liang P, Luo G, Zhao Z, Zhang C. Research of Online Hand-Eye Calibration Method Based on ChArUco Board. Sensors (Basel). 2022 May 17;22(10):3805. 35632215; PMCID: PMC9147787.
[23]
Du, G., Wang, K., Lian, S. Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review. Artif Intell Rev54, 1677–1734 (2021).
[24]
Morrical, Nate & Tremblay, Jonathan & Lin, Yunzhi & Tyree, Stephen & Birchfield, Stan & Pascucci, Valerio & Wald, Ingo. (2021). NViSII: A Scriptable Tool for Photorealistic Image Generation.

Index Terms

  1. Deep Learning-based 6D pose estimation of texture less objects for Industrial Cobots

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    AIR '23: Proceedings of the 2023 6th International Conference on Advances in Robotics
    July 2023
    583 pages
    ISBN:9781450399807
    DOI:10.1145/3610419
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 02 November 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. 6D Pose
    2. Calibration
    3. DOPE
    4. Deep learning
    5. Domain randomization
    6. Gripping
    7. Keywords- Synthetic data

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    AIR 2023

    Acceptance Rates

    Overall Acceptance Rate 69 of 140 submissions, 49%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 38
      Total Downloads
    • Downloads (Last 12 months)38
    • Downloads (Last 6 weeks)2
    Reflects downloads up to 19 Nov 2024

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media