Abstract
This paper presents a computer vision method that, taking information from an RGB-D camera, performs real time 3D object recognition to be used in cognitive robotics experiments, where the real time constraints are key. To this end, we have implemented and tested an algorithm that combines a deep neural network (YOLOv3 tiny) that processes RGB images and provides object recognition and 2D localization, with a point cloud analysis method to compute the third dimension. The proposed approach has been tested in real-time manipulation experiments with the UR5e robotic arm through ROS, and using a GPU to execute the method, showing that this combination allows for an efficient real-time learning using cognitive models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Asada, M., et al.: Cognitive developmental robotics: a survey. IEEE Trans. Auton. Ment. Dev. 1(1), 12–34 (2009)
Weng, J.: Developmental robotics: theory and experiments. Int. J. Humanoid Rob. 1(02), 199–236 (2004)
Doncieux, S., et al.: Open-ended learning: a conceptual framework based on representational redescription. Front. Neurorobot. 12, 59 (2018)
Romero, A., Piater, J., Bellas, F., Duro, R.J.: ANN-based representation learning in a lifelong open-ended learning cognitive architecture. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN) 2022, pp. 1–8. IEEE (2022)
Thrun, S., Mitchell, T.M.: Lifelong robot learning. Robot. Auton. Syst. 15(1–2), 25–46 (1995)
Dong, S., Wang, P., Abbas, K.: A survey on deep learning and its applications. Comput. Sci. Rev. 40, 100379 (2021)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)
Zhao, Z.Q., Zheng, P., Xu, S.T., Wu, X.: Object detection with deep learning: a review. IEEE Trans. Neural Networks Learn. Syst. 30(11), 3212–3232 (2019)
Arnold, E., Al-Jarrah, O.Y., Dianati, M., Fallah, S., Oxtoby, D., Mouzakitis, A.: A survey on 3d object detection methods for autonomous driving applications. IEEE Trans. Intell. Transp. Syst. 20(10), 3782–3795 (2019)
Dhillon, A., Verma, G.K.: Convolutional neural network: a review of models, methodologies and applications to object detection. Prog. Artif. Intell. 9(2), 85–112 (2019). https://doi.org/10.1007/s13748-019-00203-0
Xiao, Y., et al.: A review of object detection based on deep learning. Multimedia Tools and Applications 79(33–34), 23729–23791 (2020). https://doi.org/10.1007/s11042-020-08976-6
Guo, Y., Wang, H., Hu, Q., Liu, H., Liu, L., Bennamoun, M.: Deep learning for 3d point clouds: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 43(12), 4338–4364 (2020)
Zhou, Y., Tuzel, O.: Voxelnet: end-to-end learning for point cloud based 3D object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4490–4499 (2018)
Linder, T., Pfeiffer, K.Y., Vaskevicius, N., Schirmer, R., Arras, K.O.: Accurate detection and 3D localization of humans using a novel YOLO-based RGB-D fusion approach and synthetic training data. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 1000–1006. IEEE (2020)
Fernandes, D., et al.: Point-cloud based 3D object detection and classification methods for self-driving applications: a survey and taxonomy. Information Fusion 68, 161–191 (2021)
Romero, A., Bellas, F., Becerra, J.A., Duro, R.J.: Motivation as a tool for designing lifelong learning robots. Integr. Comput.-Aided Eng. 27(4), 353–372 (2020)
Becerra, J.A., Romero, A., Bellas, F., Duro, R.J.: Motivational engine and long-term memory coupling within a cognitive architecture for lifelong open-ended learning. Neurocomputing 452, 341–354 (2021)
Redmon, J.: Darknet: Open-Source Neural Networks in C. http://pjreddie.com/darknet/. Accessed 06 June 2022
Jiang, P., Ergu, D., Liu, F., Cai, Y., Ma, B.: A review of yolo algorithm developments. Procedia Comput. Sci. 199, 1066–1073 (2022)
YOLO algorithm. https://pjreddie.com/darknet/yolo/. Accessed 06 June 2022
ROS Noetic wiki. https://wiki.ros.org/noetic. Accessed 06 June 2022
Bjelonic, M.: Darknet_ros. http://wiki.ros.org/darknet_ros. Accessed 04 June 2022
Martin Rico, F.: Darknet_ros_3d. https://github.com/IntelligentRoboticsLabs/gb_visual_detection_3d. Accessed 06 June 2022
Ragel, R., Maza, I., Caballero, F., Ollero, A.: Comparison of motion planning techniques for a multi-rotor UAS equipped with a multi-joint manipulator arm. In: 2015 Workshop on Research, Education and Development of Unmanned Aerial Systems (RED-UAS), pp. 133–141 (2015)
Janson, L., Schmerling, E., Clark, A., Pavone, M.: Fast marching tree: a fast marching sampling-based method for optimal motion planning in many dimensions. Int. J. Robot. Res. 34(7), 883–921 (2015)
Acknowledgments
The authors wish to acknowledge the support received from the CITIC research center, funded by Xunta de Galicia and European Regional Development Fund by grant ED431G 2019/01, and to the Horizon Programme of the European Union through grant number 2019-1-ES01-KA201-065742.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Vidal-Soroa, D., Furelos, P., Bellas, F., Becerra, J.A. (2023). An Approach to 3D Object Detection in Real-Time for Cognitive Robotics Experiments. In: Tardioli, D., Matellán, V., Heredia, G., Silva, M.F., Marques, L. (eds) ROBOT2022: Fifth Iberian Robotics Conference. ROBOT 2022. Lecture Notes in Networks and Systems, vol 589. Springer, Cham. https://doi.org/10.1007/978-3-031-21065-5_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-21065-5_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-21064-8
Online ISBN: 978-3-031-21065-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)