Contact Edit: Artist Tools for Intuitive Modeling of Hand-Object Interactions
Authors: 
Arjun Sriram Lakshmipathy, Nicole Feng, Yu Xi Lee, Moshe Mahler, Nancy PollardAuthors Info & Claims
Article No.: 45, Pages 1 - 20
Abstract
Posing high-contact interactions is challenging and time-consuming, with hand-object interactions being especially difficult due to the large number of degrees of freedom (DOF) of the hand and the fact that humans are experts at judging hand poses. This paper addresses this challenge by elevating contact areas to first-class primitives. We provide end-to-end art-directable (EAD) tools to model interactions based on contact areas, directly manipulate contact areas, and compute corresponding poses automatically. To make these operations intuitive and fast, we present a novel axis-based contact model that supports real-time approximately isometry-preserving operations on triangulated surfaces, permits movement between surfaces, and is both robust and scalable to large areas. We show that use of our contact model facilitates high quality posing even for unconstrained, high-DOF custom rigs intended for traditional keyframe-based animation pipelines. We additionally evaluate our approach with comparisons to prior art, ablation studies, user studies, qualitative assessments, and extensions to full-body interaction.
Supplementary Material
References
[1]
H. Biermann, I. Martin, F. Bernardini, and D. Zorin. 2002. Cut-and-Paste Editing of Multiresolution Surfaces. 21, 3 (jul 2002), 312--321.
[2]
S. Brahmbhatt, C. Ham, C. C. Kemp, and J. Hays. 2019a. ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://contactdb.cc.gatech.edu
[3]
S. Brahmbhatt, A. Handa, J. Hays, and D. Fox. 2019b. ContactGrasp: Functional Multi-finger Grasp Synthesis from Contact. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
[4]
S. Brahmbhatt, C. Tang, C. D. Twigg, C. C. Kemp, and J. Hays. 2020. ContactPose: A Dataset of Grasps with Object Contact and Hand Pose. In The European Conference on Computer Vision (ECCV). https://contactpose.cc.gatech.edu
[5]
X. Cheng, E. Huang, Y. Hou, and M. T. Mason. 2021. Contact Mode Guided Sampling-Based Planning for Quasistatic Dexterous Manipulation in 2D. In 2021 IEEE International Conference on Robotics and Automation (ICRA). 6520--6526.
[6]
S. Christen, M. Kocabas, E. Aksan, J. Hwangbo, J. Song, and O. Hilliges. 2021. D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions. arXiv preprint arXiv:2112.03028 (2021).
[7]
K. Crane, C. Weischedel, and M. Wardetzky. 2017. The Heat Method for Distance Computation. Commun. ACM 60, 11 (Oct. 2017), 90--99.
[8]
C. Ferrari and J. Canny. 1992. Planning optimal grasps. In Proceedings 1992 IEEE International Conference on Robotics and Automation. 2290--2295 vol.3.
[9]
A. Gehre, M. M. Bronstein, L. Kobbelt, and J. Solomon. 2018. Interactive Curve Constrained Functional Maps. Computer Graphics Forum (2018).
[10]
P. Grady, C. Tang, C. D. Twigg, M. Vo, S. Brahmbhatt, and C. C. Kemp. 2021. ContactOpt: Optimizing Contact to Improve Grasps. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). http://www.pgrady.net/contactopt/
[11]
M. Guay, M.P. Cani, and R. Ronfard. 2013. The Line of Action: An Intuitive Interface for Expressive Character Posing. ACM Trans. Graph. 32, 6, Article 205 (nov 2013), 8 pages.
[12]
H. Hamer, J. Gall, R. Urtasun, and L. Van Gool. 2011. Data-driven animation of hand-object interactions. In 2011 IEEE International Conference on Automatic Face Gesture Recognition (FG). 360--367.
[13]
P. Harish, M. Mahmudi, B. Callennec, and R. Boulic. 2016. Parallel inverse kinematics for multithreaded architectures. ACM Transactions on Graphics (TOG) 35, 2 (2016), 1--13.
[14]
C. Hazard, N.S. Pollard, and S. Coros. 2020. Automated design of robotic hands for in-hand manipulation tasks. International Journal of Humanoid Robotics 17, 01 (2020), 1950029.
[15]
T. Igarashi and J. F. Hughes. 2002. Clothing Manipulation. In Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology (Paris, France) (UIST '02). Association for Computing Machinery, New York, NY, USA, 91--100.
[16]
H. Jiang, S. Liu, J. Wang, and X. Wang. 2021. Hand-Object Contact Consistency Reasoning for Human Grasps Generation. In Proceedings of the International Conference on Computer Vision (ICCV).
[17]
S.G. Johnson. 2017. The NLopt nonlinear-optimization package. http://ab-initio.mit.edu/nlopt
[18]
A. Lakshmipathy, D. Bauer, C. Bauer, and N. S. Pollard. 2022. Contact Transfer: A Direct, User-Driven Method for Human to Robot Transfer of Grasps and Manipulations. arXiv:2110.15532 [cs.RO]
[19]
A. Lakshmipathy, D. Bauer, and N. S. Pollard. 2021. Contact Tracing: A Low Cost Reconstruction Framework for Surface Contact Interpolation. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
[20]
Y. Lee, S. Lee, and S.H. Lee. 2017. Multifinger interaction between remote users in avatar-mediated telepresence. Computer Animation and Virtual Worlds 28 (05 2017), e1778.
[21]
Y. Li, J. L. Fu, and N. S. Pollard. 2007. Data-Driven Grasp Synthesis Using Shape Matching and Task-Based Pruning. IEEE Transactions on Visualization and Computer Graphics 13, 4 (2007), 732--747.
[22]
C. K. Liu. 2008. Synthesis of Interactive Hand Manipulation. In Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Dublin, Ireland) (SCA '08). Eurographics Association, Goslar, DEU, 163--171.
[23]
J. S. B. Mitchell, D. M. Mount, and C. H. Papadimitriou. 1987. The Discrete Geodesic Problem. SIAM J. Comput. 16, 4 (1987), 647--668. arXiv:https://doi.org/10.1137/0216045
[24]
I. Mordatch, Z. Popović, and E. Todorov. 2012. Contact-invariant optimization for hand manipulation. In Proceedings of the ACM SIGGRAPH/Eurographics symposium on computer animation. 137--144.
[25]
R. M. Murray, S. S. Sastry, and L. Zexiang. 1994. A Mathematical Introduction to Robotic Manipulation (1st ed.). CRC Press, Inc., USA.
[26]
N.S. Pollard. 1997. Parallel algorithms for synthesis of whole-hand grasps. In Proceedings of International Conference on Robotics and Automation, Vol. 1. 373--378 vol.1.
[27]
N. S. Pollard and V. B. Zordan. 2005. Physically Based Grasping Control from Example. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Los Angeles, California) (SCA '05). Association for Computing Machinery, New York, NY, USA, 311--318.
[28]
K. Polthier and M. Schmies. 2006. Straightest Geodesics on Polyhedral Surfaces. In ACM SIGGRAPH 2006 Courses (Boston, Massachusetts) (SIGGRAPH '06). Association for Computing Machinery, New York, NY, USA, 30--38.
[29]
M. A. Roa and R. Suarez. 2009. Computation of Independent Contact Regions for Grasping 3-D Objects. IEEE Transactions on Robotics 25, 4 (2009), 839--850.
[30]
R. M. Sanso and D. Thalmann. 1994. A Hand Control and Automatic Grasping System for Synthetic Actors. Computer Graphics Forum (1994).
[31]
R. Schmidt, C. Grimm, and B. Wyvill. 2006. Interactive Decal Compositing with Discrete Exponential Maps. 25, 3 (jul 2006), 605--613.
[32]
N. Sharp, K. Crane, et al. 2019a. geometry-central. www.geometry-central.net.
[33]
N. Sharp, Y. Soliman, and K. Crane. 2019b. The Vector Heat Method. ACM Trans. Graph. 38, 3 (2019).
[34]
B. Siciliano and O. Khatib. 2007. Springer Handbook of Robotics. Springer-Verlag, Berlin, Heidelberg.
[35]
B. Smith, C. Wu, H. Wen, P. Peluse, Y. Sheikh, J. K. Hodgins, and T. Shiratori. 2020. Constraining Dense Hand Surface Tracking with Elasticity. 39, 6, Article 219 (nov 2020), 14 pages.
[36]
O. Taheri, N. Ghorbani, M. J. Black, and D. Tzionas. 2020. GRAB: A Dataset of Whole-Body Human Grasping of Objects. In European Conference on Computer Vision (ECCV). https://grab.is.tue.mpg.de
[37]
D. Turpin, L. Wang, E. Heiden, Y.C. Chen, M. Macklin, S. Tsogkas, S. Dickinson, and A. Garg. 2022. Grasp'D: Differentiable Contact-Rich Grasp Synthesis for Multi-Fingered Hands. In Computer Vision - ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23--27, 2022, Proceedings, Part VI (Tel Aviv, Israel). Springer-Verlag, Berlin, Heidelberg, 201--221.
[38]
Y. Wu, J. Wang, Y. Zhang, S. Zhang, O. Hilliges, F. Yu, and S. Tang. 2021. SAGA: Stochastic Whole-Body Grasping with Contact. (12 2021).
[39]
K. Yamane and Y. Nakamura. 2003. Natural motion animation through constraining and deconstraining at will. IEEE Transactions on visualization and computer graphics 9, 3 (2003), 352--360.
[40]
Y. Ye and C. K. Liu. 2012. Synthesis of detailed hand manipulations using contact sampling. ACM Trans. Graph. 31, 4 (2012), 1--10.
[41]
H. Zhang, Y. Ye, T. Shiratori, and T. Komura. 2021. ManipNet: Neural Manipulation Synthesis with a Hand-Object Spatial Representation. 40, 4, Article 121 (jul 2021), 14 pages.
[42]
J. Zhao and N. I. Badler. 1994. Inverse kinematics positioning using nonlinear programming for highly articulated figures. ACM Transactions on Graphics (TOG) 13, 4 (1994), 313--336.
[43]
W. Zhao, J. Zhang, J. Min, and J. Chai. 2013. Robust Realtime Physics-Based Motion Control for Human Grasping. 32, 6, Article 207 (nov 2013), 12 pages.
Index Terms
- Contact Edit: Artist Tools for Intuitive Modeling of Hand-Object Interactions
Recommendations
Data-driven Fingertip Appearance for Interactive Hand Simulation
MIG '13: Proceedings of Motion on GamesContact on a finger pad results in deformation that redistributes blood within the fingertip tissue in a manner correlated to the pressure. We build a data-driven model that relates contact information to the visible changes of the finger nail and ...
Mechanical Design of an Anthropomorphic Hand for Shapeless Object Grasping
CERMA '10: Proceedings of the 2010 IEEE Electronics, Robotics and Automotive Mechanics ConferenceIn this paper is presented the design of a mechanical hand, for tele-operation tasks, which counts with movements similar to those made by a human hand, due to its number of joints (22 DOF) and structure built from the wrist. The relative movements ...
Object handing with a robotic hand equipped with optical three-axis tactile sensors
RA '07: Proceedings of the 13th IASTED International Conference on Robotics and ApplicationsIn object handing, a robot should decide proper grasping force on the basis of the hardness of an object and the degree of surface slipperiness, which are estimated by three-axis tactile sensors mounted on its fingers. In the present paper, we produce a ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © 2023 Owner/Author(s).
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 26 July 2023
Published in TOG Volume 42, Issue 4
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 450Total Downloads
- Downloads (Last 12 months)323
- Downloads (Last 6 weeks)33
Reflects downloads up to 01 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in