Abstract
This paper presents a robot system for performing pick and place operations with deformable objects. The system uses a structured light scanner to capture a point cloud of the object to be grasped. This point cloud is then analyzed to determine a pick and place action. Finally, the determined action is executed by the robot to solve the task. The robotic placement strategy contains several free parameters, which should be chosen in a context-specific manner. To determine these parameters we rely on simulation-based optimization of the individual use cases. The entire system is tested extensively in real world trials. First, the reliability of the grasp is evaluated for 7 different types of pork cuts. Then the validity of the simulation-based optimization of the placement strategy is evaluated for 2 of the most different pork cuts, to show the generality of the overall approach.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Balaguer, B., Carpin, S.: Combining imitation and reinforcement learning to fold deformable planar objects. In: IROS, pp. 1405–1412. IEEE. http://dblp.uni-trier.de/db/conf/iros/iros2011.html#BalaguerC11 (2011)
Berkenkamp, F., Krause, A., Schoellig, A.P.: Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics. arXiv:1602.04450 (2016)
Bodenhagen, L., Fugl, A.R., Jordt, A., Willatzen, M., Andersen, K.A., Olsen, M.M., Koch, R., Petersen, H.G., Krüger, N.: An adaptable robot vision system performing manipulation actions with flexible objects. IEEE Trans. Autom. Sci. Eng. 11(3), 749–765 (2014)
Buch, J.P., Laursen, J.S., Sørensen, L.C., Ellekilde, L.P., Kraft, D., Schultz, U.P., Petersen, H.G.: Applying simulation and a domain-specific language for an adaptive action library. In: International Conference on Simulation, Modeling, and Programming for Autonomous Robots, pp. 86–97. Springer (2014)
Caelles, S., Maninis, K.K., Pont-Tuset, J., Leal-Taixé, L., Cremers, D., Van Gool, L.: One-shot video object segmentation. In: Computer Vision and Pattern Recognition (CVPR) (2017)
Calandra, R., Seyfarth, A., Peters, J., Deisenroth, M.P.: Bayesian optimization for learning gaits under uncertainty. Ann. Math. Artif. Intell. 76(1-2), 5–23 (2016)
Costa, A., Nannicini, G.: Rbfopt: an open-source library for black-box optimization with costly function evaluations. Optimization Online (4538) (2014)
Eberly, D.: Thin plate splines. Geometric Tools Inc 2002, 116 (2002)
Eiríksson, E.R., Wilm, J., Pedersen, D.B., Aanæs, H.: Precision and accuracy parameters in structured light 3-d scanning. International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences 40 (2015)
Ellekilde, L.P., Jorgensen, J.A.: Robwork: A flexible toolbox for robotics research and education. In: 2010 41st International Symposium on and 2010 6th German Conference on Robotics of Robotics (ISR), (ROBOTIK), pp. 1–7. VDE (2010)
Geng, J.: Structured-light 3d surface imaging: a tutorial. Adv. Opt. Photon. 3(2), 128–160 (2011)
Gonzalez-Jorge, H., Rodríguez-Gonzálvez, P., Martínez-Sánchez, J., González-Aguilera, D., Arias, P., Gesto, M., Díaz-Vilariño, L.: Metrological comparison between kinect i and kinect ii sensors. Measurement 70, 21–26 (2015)
Gupta, M., Nayar, S.K.: Micro phase shifting. Proc. IEEE CVPR, pp. 813–820 (2012)
Gupta, M., Yin, Q., Nayar, S.K.: Structured light in sunlight. In: The IEEE International Conference on Computer Vision (ICCV) (2013)
Hemker, T., Stelzer, M., von Stryk, O., Sakamoto, H.: Efficient walking speed optimization of a humanoid robot. Int. J. Robot. Res. 28(2), 303–314 (2009)
Jensen, S., Wilm, J., Aanæs, cH.: An error analysis of structured light scanning of biological tissue, pp. 135–145 Springer (2017)
Jørgensen, T.B., Holm, P.H.S., Petersen, H.G., Krüger, N.: Intelligent Robotics and Applications: 8th International Conference, ICIRA 2015, Portsmouth, UK, August 24-27, 2015. Springer International Publishing, Cham (2015)
Jørgensen, T.B., Debrabant, K., Krüger, N.: Robust optimization of robotic pick and place operations for deformable objects through simulation. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 3863–3870 (2016)
Jørgensen, T.B., Pedersen, M.M., Hansen, N.W., Hansen, B.R., Kruger, N.: A flexible suction based grasp tool and associated grasp strategies for handling meat. International Conference on Mechatronics and Robotics Engineering accepted (2017)
Jørgensen, T.B., Wolniakowski, A., Petersen, H.G., Debrabant, K., Kruger, N.: Robust optimization with applications to design of context specific robot solutions. Robotics and Computer Integrated Manufacturing Submitted (2017)
Kabsch, W.: A solution for the best rotation to relate two sets of vectors. Acta Crystallogr., Sect. A: Cryst. Phys., Diffr., Theor. Gen. Crystallogr. 32(5), 922–923 (1976)
Koh, Y.J., Kim, C.S.: Primary object segmentation in videos based on region augmentation and reduction. http://openaccess.thecvf.com/content_cvpr_2017/papers/Koh_Primary_Object_Segmentation_CVPR_2017_paper.pdf (2017)
Kruse, D., Radke, R.J., Wen, J.T.: Human-robot collaborative handling of highly deformable materials. In: American Control Conference (ACC), 2017, pp. 1511–1516. IEEE (2017)
Li, Y., Wang, Y., Case, M., Chang, S.F., Allen, P.K.: Real-time pose estimation of deformable objects using a volumetric approach. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), pp. 1046–1052. IEEE (2014)
Long, P., Khalil, W., Martinet, P.: Robotic deformable object cutting: from simulation to experimental validation (2014)
Loshchilov, I., Schoenauer, M., Sebag, M.: Adaptive coordinate descent (2011)
Mesit, J., Guha, R., Chaudhry, S.: 3d soft body simulation using mass-spring system with internal pressure force and simplified implicit integration. J. Comput. 2(8), 34–43 (2007)
Misimi, E., Øye, E.R., Eilertsen, A., Mathiassen, J.R., AAsebø, O.B., Gjerstad, T., Buljo, J., Skotheim, Ø.: Gribbot–robotic 3d vision-guided harvesting of chicken fillets. Comput. Electron. Agric. 121, 84–100 (2016)
Nabil, E., Belhassen-Chedli, B., Grigore, G.: Soft material modeling for robotic task formulation and control in the muscle separation process. Robot. Comput. Integr. Manuf. 32, 37–53 (2015)
Pont-Tuset, J., Perazzi, F., Caelles, S., Arbeláez, P., Sorkine-Hornung, A., Van Gool, L.: The 2017 davis challenge on video object segmentation. arXiv:1704.00675 (2017)
Posdamer, J., Altschuler, M.: Surface measurement by space-encoded projected beam systems. Comput. Graphics Image Process. 18, 1–17 (1982). https://doi.org/10.1016/0146-664X(82)90096-X
Rusu, R.B., Cousins, S.: 3D is here: Point Cloud Library (PCL). In: IEEE International Conference on Robotics and Automation (ICRA). Shanghai, China (2011)
Rusu, R.B., Cousins, S.: Region growing segmentation. http://pointclouds.org/documentation/tutorials/region_growing_segmentation.php (2017)
Schulman, J., Lee, A., Ho, J., Abbeel, P.: Tracking deformable objects with point clouds. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), pp. 1130–1137. IEEE (2013)
Tappen, M.F., Freeman, W.T.: Comparison of graph cuts with belief propagation for stereo, using identical mrf parameters. In: Null, p. 900. IEEE (2003)
Tesch, M., Schneider, J., Choset, H.: Using response surfaces and expected improvement to optimize snake robot gait parameters. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1069–1074. IEEE (2011)
Voigtlaender, P., Leibe, B.: Online adaptation of convolutional neural networks for video object segmentation. In: BMVC (2017)
Wilm, J., Olesen, O.V., Larsen, R.: Slstudio: open-source framework for real-time structured light. Proceedings of the 4th International Conference on Image Processing Theory, Tools and Application (ipta 2014) p. 7002001. https://doi.org/10.1109/IPTA.2014.7002001 (2014)
Wolniakowski, A., Jorgensen, J.A., Miatliuk, K., Petersen, H.G., Kruger, N.: Task and context sensitive optimization of gripper design using dynamic grasp simulation. In: 2015 20th International Conference on Methods and Models in Automation and Robotics (MMAR), pp. 29–34. IEEE (2015)
Zoumponos, G.T., Aspragathos, N.A.: A fuzzy strategy for the robotic folding of fabrics with machine vision feedback. Industrial Robot: An International Journal 37(3), 302–308 (2010)
Acknowledgments
The financial support from the The Danish Innovation Foundation through the strategic platform “MADE-Platform for Future Production” and from the EU project ReconCell (FP7-ICT-680431) is gratefully acknowledged.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Jørgensen, T.B., Jensen, S.H.N., Aanæs, H. et al. An Adaptive Robotic System for Doing Pick and Place Operations with Deformable Objects. J Intell Robot Syst 94, 81–100 (2019). https://doi.org/10.1007/s10846-018-0958-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-018-0958-6