Abstract
This paper proposes a generalizable model to synthesize high-fidelity clothing wrinkle deformation in 3D by learning from real data. Given the complex deformation behaviors of real-world clothing, this task presents significant challenges, primarily due to the lack of accurate ground-truth data. Obtaining high-fidelity 3D deformations requires special equipment like a multi-camera system, which is not easily scalable. To address this challenge, we decompose the clothing into a base surface and fine wrinkles; and introduce a new method that can generate wrinkles as high-frequency 3D displacement from coarse clothing deformation. Our method is conditioned by Green-Lagrange strain field—a local rotation-invariant measurement that is independent of body and clothing topology, enhancing its generalizability. Using limited real data (e.g., 3K) of garment meshes, we train a diffusion model that can generate high-fidelity wrinkles from a coarse clothing mesh, conditioned on its strain field. Practically, we obtain the coarse clothing mesh using a body-conditioned VAE, ensuring compatibility of the deformation with the body pose. In our experiments, we demonstrate that our generative wrinkle model outperforms existing methods by synthesizing high-fidelity wrinkle deformation from novel body poses and clothing while preserving the quality comparable to the one from training data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aharoni, H., Todorova, D.V., Albarrán, O., Goehring, L., Kamien, R.D., Katifori, E.: The smectic order of wrinkles. Nat. Commun. 15809 (2017)
Bertiche, H., Madadi, M., Escalera, S.: Pbns: physically based neural simulation for unsupervised garment pose space deformation. ACM TOG 1–14 (2021)
Bertiche, H., Madadi, M., Escalera, S.: Neural cloth simulation. ACM TOG (2022)
Bhatnagar, B.L., Sminchisescu, C., Theobalt, C., Pons-Moll, G.: Loopreg: self-supervised learning of implicit surface correspondences, pose and shape for 3d human mesh registration. In: NeurIPS, pp. 12909–12922 (2020)
Bouaziz, S., Martin, S., Liu, T., Kavan, L., Pauly, M.: Projective dynamics: fusing constraint projections for fast simulation. ACM TOG (2014)
Chen, L., et al.: Deep deformation detail synthesis for thin shell models. In: Computer Graphics Forum (2023)
Chen, Z., Chen, H.Y., Kaufman, D.M., Skouras, M., Vouga, E.: Fine wrinkling on coarsely meshed thin shells. ACM TOG 1–32 (2021)
Chen, Z., Kaufman, D., Skouras, M., Vouga, E.: Complex wrinkle field evolution. ACM TOG 1–19 (2023)
Chung, H., Kim, J., Mccann, M.T., Klasky, M.L., Ye, J.C.: Diffusion posterior sampling for general noisy inverse problems. In: ICLR (2022)
Du, T., et al.: Diffpd: differentiable projective dynamics. ACM TOG 1–21 (2021)
Feng, X., Huang, W., Xu, W., Wang, H.: Learning-based bending stiffness parameter estimation by a drape tester. ACM TOG 1–16 (2022)
Furukawa, Y., Ponce, J.: Dense 3d motion capture from synchronized video streams. In: CVPR, pp. 1–8 (2008)
Grigorev, A., Black, M.J., Hilliges, O.: Hood: hierarchical graphs for generalized modelling of clothing dynamics. In: CVPR, pp. 16965–16974 (2023)
Guo, J., Li, J., Narain, R., Park, H.S.: Inverse simulation: reconstructing dynamic geometry of clothed humans via optimal control. In: CVPR, pp. 14698–14707 (2021)
Guo, J., et al.: Diffusion shape prior for wrinkle-accurate cloth registration. arXiv preprint arXiv:2311.05828 (2023)
Halimi, O., et al.: Pattern-based cloth registration and sparse-view animation. ACM TOG 1–17 (2022)
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: NeurIPS (2017)
Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In: NeurIPS, pp. 6840–6851 (2020)
Ho, J., Salimans, T.: Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 (2022)
Hu, Y., et al.: Difftaichi: differentiable programming for physical simulation. In: ICLR (2020)
Işık, M., et al.: Humanrf: high-fidelity neural radiance fields for humans in motion. ACM TOG 1–12 (2023)
Jafarian, Y., Park, H.S.: Learning high fidelity depths of dressed humans by watching social media dance videos. In: CVPR, pp. 12753–12762 (2021)
Kavan, L., Gerszewski, D., Bargteil, A.W., Sloan, P.P.: Physics-inspired upsampling for cloth simulation in games. ACM TOG 1–10 (2011)
Kim, B., et al.: Chupa: carving 3d clothed humans from skinned shape priors using 2d diffusion probabilistic models. arXiv preprint arXiv:2305.11870 (2023)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. In: ICLR (2014)
Lähner, Z., Cremers, D., Tung, T.: DeepWrinkles: accurate and realistic clothing modeling. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11208, pp. 698–715. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01225-0_41
Le, B.H., Deng, Z.: Smooth skinning decomposition with rigid bones. ACM TOG 1–10 (2012)
Li, J., et al.: An implicit frictional contact solver for adaptive cloth simulation. ACM TOG 1–15 (2018)
Li, Y., Du, T., Wu, K., Xu, J., Matusik, W.: Diffcloth: differentiable cloth simulation with dry frictional contact. ACM TOG 1–20 (2022)
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., Black, M.J.: Smpl: a skinned multi-person linear model. ACM TOG 1–16 (2015)
Ma, Q., et al.: Learning to dress 3d people in generative clothing. In: CVPR, pp. 6469–6478 (2020)
Mahmood, N., Ghorbani, N., Troje, N.F., Pons-Moll, G., Black, M.J.: Amass: archive of motion capture as surface shapes. In: ICCV, pp. 5442–5451 (2019)
Müller, M., Chentanez, N.: Wrinkle meshes. In: Symposium on Computer Animation, pp. 85–91 (2010)
Müller, M., Heidelberger, B., Hennix, M., Ratcliff, J.: Position based dynamics. J. Vis. Commun. Image Representation 109–118 (2007)
Narain, R., Pfaff, T., O’Brien, J.F.: Folding and crumpling adaptive sheets. ACM TOG 1–8 (2013)
Narain, R., Samii, A., O’brien, J.F.: Adaptive anisotropic remeshing for cloth simulation. ACM TOG 1–10 (2012)
Pan, X., et al.: Predicting loose-fitting garment deformations using bone-driven motion networks. In: ACM SIGGRAPH 2022 Conference Proceedings, pp. 1–10 (2022)
Patel, C., Liao, Z., Pons-Moll, G.: Tailornet: predicting clothing in 3d as a function of human pose, shape and garment style. In: CVPR, pp. 7365–7375 (2020)
Peng, S., et al.: Neural body: implicit neural representations with structured latent codes for novel view synthesis of dynamic humans. In: CVPR (2021)
Pons-Moll, G., Pujades, S., Hu, S., Black, M.J.: Clothcap: seamless 4d clothing capture and retargeting. ACM TOG 1–15 (2017)
Qiao, Y.L., Liang, J., Koltun, V., Lin, M.C.: Scalable differentiable physics for learning and control. In: ICML, pp. 7847–7856 (2020)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: MICCAI, pp. 234–241 (2015)
Saito, S., Yang, J., Ma, Q., Black, M.J.: Scanimate: weakly supervised learning of skinned clothed avatar networks. In: CVPR, pp. 2886–2897 (2021)
Santesteban, I., Otaduy, M.A., Casas, D.: Learning-based animation of clothing for virtual try-on. Comput. Graph. Forum 355–366 (2019)
Santesteban, I., Otaduy, M.A., Casas, D.: Snug: self-supervised neural dynamic garments. In: CVPR, pp. 8140–8150 (2022)
Shao, R., Zheng, Z., Zhang, H., Sun, J., Liu, Y.: DiffuStereo: high quality human reconstruction via diffusion-based stereo using sparse cameras. In: Avidan, S., Brostow, G., Cisse, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision – ECCV 2022. ECCV 2022. LNCS, vol. 13692, pp. 702–720. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19824-3_41
Sohn, K., Lee, H., Yan, X.: Learning structured output representation using deep conditional generative models. In: NeurIPS (2015)
Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models. In: ICLR (2021)
Stein, O., Jacobson, A., Wardetzky, M., Grinspun, E.: A smoothness energy without boundary distortion for curved surfaces. ACM TOG 1–17 (2020)
Wang, H., O’Brien, J.F., Ramamoorthi, R.: Data-driven elastic models for cloth: modeling and measurement. ACM TOG 1–12 (2011)
Yoon, J.S., Yu, Z., Park, J., Park, H.S.: Humbi: a large multiview dataset of human body expressions and benchmark challenge. IEEE TPAMI, pp. 623–640 (2021)
Zhang, C., Pujades, S., Black, M.J., Pons-Moll, G.: Detailed, accurate, human shape estimation from clothed 3d scan sequences. In: CVPR, pp. 4191–4200 (2017)
Zhang, H., et al.: Pymaf: 3d human pose and shape regression with pyramidal mesh alignment feedback loop. In: ICCV, pp. 11446–11456 (2021)
Zhang, J.E., Dumas, J., Fei, Y., Jacobson, A., James, D.L., Kaufman, D.M.: Progressive simulation for cloth quasistatics. ACM TOG 1–16 (2022)
Zhang, M., Ceylan, D., Mitra, N.J.: Motion guided deep dynamic 3d garments. ACM TOG 1–12 (2022)
Zhang, M., Wang, T., Ceylan, D., Mitra, N.J.: Deep detail enhancement for any garment. In: Computer Graphics Forum, pp. 399–411 (2021)
Zhong, Y.D., Han, J., Brikis, G.O.: Differentiable physics simulations with contacts: do they have correct gradients wrt position, velocity and control? arXiv preprint arXiv:2207.05060 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Guo, J., Yoon, J.S., Saito, S., Shiratori, T., Park, H.S. (2025). High-Fidelity Modeling of Generalizable Wrinkle Deformation. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15139. Springer, Cham. https://doi.org/10.1007/978-3-031-73004-7_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-73004-7_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-73003-0
Online ISBN: 978-3-031-73004-7
eBook Packages: Computer ScienceComputer Science (R0)