Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Towards Garment Sewing Pattern Reconstruction from a Single Image

Published: 05 December 2023 Publication History

Abstract

Garment sewing pattern represents the intrinsic rest shape of a garment, and is the core for many applications like fashion design, virtual try-on, and digital avatars. In this work, we explore the challenging problem of recovering garment sewing patterns from daily photos for augmenting these applications. To solve the problem, we first synthesize a versatile dataset, named SewFactory, which consists of around 1M images and ground-truth sewing patterns for model training and quantitative evaluation. SewFactory covers a wide range of human poses, body shapes, and sewing patterns, and possesses realistic appearances thanks to the proposed human texture synthesis network. Then, we propose a two-level Transformer network called Sewformer, which significantly improves the sewing pattern prediction performance. Extensive experiments demonstrate that the proposed framework is effective in recovering sewing patterns and well generalizes to casually-taken human photos. Code, dataset, and pre-trained models will be released.

Supplementary Material

ZIP File (papers_220s4-file4.zip)
supplemental

References

[1]
Badour Albahar, Jingwan Lu, Jimei Yang, Zhixin Shu, Eli Shechtman, and Jia-Bin Huang. 2021. Pose with Style: Detail-preserving pose-guided image synthesis with conditional stylegan. ACM Transactions on Graphics (TOG) 40, 6 (2021), 1--11.
[2]
Thiemo Alldieck, Marcus Magnor, Bharat Lal Bhatnagar, Christian Theobalt, and Gerard Pons-Moll. 2019. Learning to reconstruct people in clothing from a single RGB camera. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1175--1186.
[3]
Autodesk, INC. 2019. Maya. https:/autodesk.com/maya
[4]
Seungbae Bang, Maria Korosteleva, and Sung-Hee Lee. 2021. Estimating Garment Patterns from Static Scan Data. Computer Graphics Forum (Eurographics) 40, 6 (2021), 273--287.
[5]
Hugo Bertiche, Meysam Madadi, and Sergio Escalera. 2020. CLOTH3D: clothed 3d humans. In European Conference on Computer Vision. Springer, 344--359.
[6]
Bharat Lal Bhatnagar, Garvita Tiwari, Christian Theobalt, and Gerard Pons-Moll. 2019. Multi-garment net: Learning to dress 3d people from images. In proceedings of the IEEE/CVF international conference on computer vision. 5420--5430.
[7]
Bindita Chaudhuri, Nikolaos Sarafianos, Linda Shapiro, and Tony Tung. 2021. Semi-supervised synthesis of high-resolution editable textures for 3d humans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7991--8000.
[8]
Xipeng Chen, Guangrun Wang, Dizhong Zhu, Xiaodan Liang, Philip HS Torr, and Liang Lin. 2022a. Structure-Preserving 3D Garment Modeling with Neural Sewing Machines. arXiv preprint arXiv:2211.06701 (2022).
[9]
Xiaowu Chen, Bin Zhou, Fei-Xiang Lu, Lin Wang, Lang Bi, and Ping Tan. 2015. Garment modeling with a depth camera. ACM Transactions on Graphics 34, 6 (2015), 203--1.
[10]
Zhiqin Chen, Kangxue Yin, and Sanja Fidler. 2022b. AUV-Net: Learning Aligned UV Maps for Texture Transfer and Synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1465--1474.
[11]
Kwang-Jin Choi and Hyeong-Seok Ko. 2002. Stable but Responsive Cloth. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques. Association for Computing Machinery, New York, NY, USA, 604--611.
[12]
Enric Corona, Albert Pumarola, Guillem Alenya, Gerard Pons-Moll, and Francesc Moreno-Noguer. 2021. Smplicit: Topology-aware generative model for clothed people. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11875--11885.
[13]
Jianglin Fu, Shikai Li, Yuming Jiang, Kwan-Yee Lin, Chen Qian, Chen Change Loy, Wayne Wu, and Ziwei Liu. 2022. Stylegan-human: A data-centric odyssey of human generation. In European Conference on Computer Vision. Springer, 1--19.
[14]
Iliyan Georgiev, Thiago Ize, Mike Farnsworth, Ramón Montoya-Vozmediano, Alan King, Brecht Van Lommel, Angel Jimenez, Oscar Anson, Shinji Ogaki, Eric Johnston, et al. 2018. Arnold: A brute-force production path tracer. ACM Transactions on Graphics (TOG) 37, 3 (2018), 1--12.
[15]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative adversarial networks. Commun. ACM 63, 11 (2020), 139--144.
[16]
Chihiro Goto and Nobuyuki Umetani. 2021. Data-driven Garment Pattern Estimation from 3D Geometries. In Eurographics. The Eurographics Association.
[17]
Artur Grigorev, Karim Iskakov, Anastasia Ianina, Renat Bashirov, Ilya Zakharkin, Alexander Vakhitov, and Victor Lempitsky. 2021. Stylepeople: A generative model of fullbody human avatars. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5151--5160.
[18]
Rıza Alp Güler, Natalia Neverova, and Iasonas Kokkinos. 2018. Densepose: Dense human pose estimation in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7297--7306.
[19]
Xintong Han, Xiaojun Hu, Weilin Huang, and Matthew R Scott. 2019. Clothflow: A flow-based model for clothed person generation. In Proceedings of the IEEE/CVF international conference on computer vision. 10471--10480.
[20]
Nils Hasler, Bodo Rosenhahn, and Hans-Peter Seidel. 2007. Reverse engineering garments. In International Conference on Computer Vision/Computer Graphics Collaboration Techniques and Applications. Springer, 200--211.
[21]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.
[22]
Charlie Hewitt, Tadas Baltrušaitis, Erroll Wood, Lohit Petikam, Louis Florentin, and Hanz Cuevas Velasquez. 2023. Procedural Humans for Computer Vision. arXiv preprint arXiv:2301.01161 (2023).
[23]
Zeng Huang, Yuanlu Xu, Christoph Lassner, Hao Li, and Tony Tung. 2020. Arch: Animatable reconstruction of clothed humans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3093--3102.
[24]
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1125--1134.
[25]
Moon-Hwan Jeong, Dong-Hoon Han, and Hyeong-Seok Ko. 2015. Garment capture from a photograph. Computer Animation and Virtual Worlds 26, 3--4 (2015), 291--300.
[26]
Boyi Jiang, Juyong Zhang, Yang Hong, Jinhao Luo, Ligang Liu, and Hujun Bao. 2020. Bcnet: Learning body and cloth shape from a single image. In European Conference on Computer Vision. Springer, 18--35.
[27]
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision. Springer, 694--711.
[28]
Tero Karras, Samuli Laine, and Timo Aila. 2019. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 4401--4410.
[29]
James Kennedy and Russell Eberhart. 1995. Particle swarm optimization. In Proceedings of ICNN'95-international conference on neural networks, Vol. 4. IEEE, 1942--1948.
[30]
Maria Korosteleva and Sung-Hee Lee. 2021. Generating Datasets of 3D Garments with Sewing Patterns. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, Vol. 1.
[31]
Maria Korosteleva and Sung-Hee Lee. 2022. NeuralTailor: Reconstructing Sewing Pattern Structures from 3D Point Clouds of Garments. ACM Transactions on Graphics (SIGGRAPH) 41, 4 (2022), 16 pages.
[32]
Christoph Lassner, Gerard Pons-Moll, and Peter V Gehler. 2017. A generative model of people in clothing. In Proceedings of the IEEE International Conference on Computer Vision. 853--862.
[33]
Verica Lazova, Eldar Insafutdinov, and Gerard Pons-Moll. 2019. 360-degree textures of people in clothing from a single image. In 2019 International Conference on 3D Vision (3DV). IEEE, 643--653.
[34]
Kaixuan Liu, Xianyi Zeng, Pascal Bruniaux, Xuyuan Tao, Xiaofeng Yao, Victoria Li, and Jianping Wang. 2018. 3D interactive garment pattern-making technology. Computer-Aided Design 104 (2018), 113--124.
[35]
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision. 10012--10022.
[36]
Ziwei Liu, Ping Luo, Shi Qiu, Xiaogang Wang, and Xiaoou Tang. 2016. Deepfashion: Powering robust clothes recognition and retrieval with rich annotations. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1096--1104.
[37]
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. 2015. SMPL: A skinned multi-person linear model. ACM transactions on graphics (TOG) 34, 6 (2015), 1--16.
[38]
Qianli Ma, Jinlong Yang, Anurag Ranjan, Sergi Pujades, Gerard Pons-Moll, Siyu Tang, and Michael J Black. 2020. Learning to dress 3d people in generative clothing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6469--6478.
[39]
Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Gerard Pons-Moll, and Michael J Black. 2019. AMASS: Archive of motion capture as surface shapes. In Proceedings of the IEEE/CVF international conference on computer vision. 5442--5451.
[40]
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. 2021. Sdedit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations.
[41]
Gyeongsik Moon, Hyeongjin Nam, Takaaki Shiratori, and Kyoung Mu Lee. 2022. 3D clothed human reconstruction in the wild. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23--27, 2022, Proceedings, Part II. Springer, 184--200.
[42]
Rahul Narain, Armin Samii, and James F O'brien. 2012. Adaptive anisotropic remeshing for cloth simulation. ACM transactions on graphics (TOG) 31, 6 (2012), 1--10.
[43]
Chaitanya Patel, Zhouyingcheng Liao, and Gerard Pons-Moll. 2020. Tailornet: Predicting clothing in 3d as a function of human pose, shape and garment style. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7365--7375.
[44]
Albert Pumarola, Jordi Sanchez-Riera, Gary Choi, Alberto Sanfeliu, and Francesc Moreno-Noguer. 2019. 3dpeople: Modeling the geometry of dressed humans. In Proceedings of the IEEE/CVF international conference on computer vision. 2242--2251.
[45]
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10684--10695.
[46]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, 234--241.
[47]
Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, and Hao Li. 2019. Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization. In Proceedings of the IEEE/CVF international conference on computer vision. 2304--2314.
[48]
Shunsuke Saito, Tomas Simon, Jason Saragih, and Hanbyul Joo. 2020. Pifuhd: Multilevel pixel-aligned implicit function for high-resolution 3d human digitization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 84--93.
[49]
Kripasindhu Sarkar, Vladislav Golyanik, Lingjie Liu, and Christian Theobalt. 2021a. Style and pose control for image synthesis of humans from a single monocular view. arXiv preprint arXiv:2102.11263 (2021).
[50]
Kripasindhu Sarkar, Lingjie Liu, Vladislav Golyanik, and Christian Theobalt. 2021b. Humangan: A generative model of human images. In 2021 International Conference on 3D Vision (3DV). IEEE, 258--267.
[51]
Kripasindhu Sarkar, Dushyant Mehta, Weipeng Xu, Vladislav Golyanik, and Christian Theobalt. 2020. Neural re-rendering of humans from a single image. In European Conference on Computer Vision. Springer, 596--613.
[52]
Ryan Schmidt, Cindy Grimm, and Brian Wyvill. 2006. Interactive decal compositing with discrete exponential maps. In ACM SIGGRAPH 2006 Papers. 605--613.
[53]
Nicholas Sharp and Keenan Crane. 2018. Variational surface cutting. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1--13.
[54]
Garvita Tiwari, Bharat Lal Bhatnagar, Tony Tung, and Gerard Pons-Moll. 2020. Sizer: A dataset and model for parsing 3d clothing and learning size sensitive 3d clothing. In European Conference on Computer Vision. Springer, 1--18.
[55]
Lokender Tiwari and Brojeshwar Bhowmick. 2021. DeepDraper: Fast and Accurate 3D Garment Draping over a 3D Human Body. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 1416--1426.
[56]
Gül Varol, Javier Romero, Xavier Martin, Naureen Mahmood, Michael J. Black, Ivan Laptev, and Cordelia Schmid. 2017. Learning from Synthetic Humans. In CVPR.
[57]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
[58]
Miao Wang, Guo-Ye Yang, Ruilong Li, Run-Ze Liang, Song-Hai Zhang, Peter M Hall, and Shi-Min Hu. 2019. Example-guided style-consistent image synthesis from semantic labeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1495--1504.
[59]
Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. 2018b. Pixel2mesh: Generating 3d mesh models from single rgb images. In Proceedings of the European conference on computer vision (ECCV). 52--67.
[60]
Tuanfeng Y. Wang, Duygu Ceylan, Jovan Popovic, and Niloy J. Mitra. 2018a. Learning a Shared Shape Space for Multimodal Garment Design. ACM Trans. Graph. 37, 6 (2018), 1:1--1:14.
[61]
Shuchen Weng, Wenbo Li, Dawei Li, Hongxia Jin, and Boxin Shi. 2020. Misc: Multi-condition injection and spatially-adaptive compositing for conditional person image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7741--7749.
[62]
Wenqi Xian, Patsorn Sangkloy, Varun Agrawal, Amit Raj, Jingwan Lu, Chen Fang, Fisher Yu, and James Hays. 2018. Texturegan: Controlling deep image synthesis with texture patches. In Proceedings of the IEEE conference on computer vision and pattern recognition. 8456--8465.
[63]
Donglai Xiang, Timur Bagautdinov, Tuur Stuyck, Fabian Prada, Javier Romero, Weipeng Xu, Shunsuke Saito, Jingfan Guo, Breannan Smith, Takaaki Shiratori, et al. 2022. Dressing avatars: Deep photorealistic appearance for physically simulated clothing. ACM Transactions on Graphics (TOG) 41, 6 (2022), 1--15.
[64]
Yuliang Xiu, Jinlong Yang, Xu Cao, Dimitrios Tzionas, and Michael J. Black. 2023. ECON: Explicit Clothed humans Optimized via Normal integration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
[65]
Xiangyu Xu, Hao Chen, Francesc Moreno-Noguer, Laszlo A Jeni, and Fernando De la Torre. 2020. 3D Human Shape and Pose from a Single Low-Resolution Image with Self-Supervised Learning. In ECCV.
[66]
Xiangyu Xu and Chen Change Loy. 2021. 3D human texture estimation from a single image with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 13849--13858.
[67]
Xiangyu Xu, Deqing Sun, Jinshan Pan, Yujin Zhang, Hanspeter Pfister, and Ming-Hsuan Yang. 2017. Learning to super-resolve blurry face and text images. In Proceedings of the IEEE international conference on computer vision. 251--260.
[68]
Shan Yang, Zherong Pan, Tanya Amert, Ke Wang, Licheng Yu, Tamara Berg, and Ming C Lin. 2018. Physics-inspired garment recovery from a single-view image. ACM Transactions on Graphics 37, 5 (2018), 1--14.
[69]
Zhuoqian Yang, Shikai Li, Wayne Wu, and Bo Dai. 2022. 3DHumanGAN: Towards Photo-Realistic 3D-Aware Human Image Generation. arXiv preprint arXiv:2212.07378 (2022).
[70]
Chao Zhang, Sergi Pujades, Michael J Black, and Gerard Pons-Moll. 2017. Detailed, accurate, human shape estimation from clothed 3D scan sequences. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4191--4200.
[71]
Pan Zhang, Bo Zhang, Dong Chen, Lu Yuan, and Fang Wen. 2020. Cross-domain correspondence learning for exemplar-based image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5143--5153.
[72]
Fang Zhao, Shengcai Liao, Kaihao Zhang, and Ling Shao. 2020. Human parsing based texture transfer from single image to 3D human via cross-view consistency. Advances in Neural Information Processing Systems 33 (2020), 14326--14337.
[73]
Fang Zhao, Wenhao Wang, Shengcai Liao, and Ling Shao. 2021. Learning anchored unsigned distance functions with gradient direction alignment for single-view garment reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 12674--12683.
[74]
Heming Zhu, Yu Cao, Hang Jin, Weikai Chen, Dong Du, Zhangye Wang, Shuguang Cui, and Xiaoguang Han. 2020. Deep Fashion3D: A dataset and benchmark for 3D garment reconstruction from single images. In European Conference on Computer Vision.

Cited By

View all
  • (2024)Simulation Technology for Apparel Designアパレル設計を支えるシミュレーション技術Journal of Japan Society of Kansei Engineering10.5057/kansei.22.1_1122:1(11-14)Online publication date: 31-Mar-2024
  • (2024)Models and means of clothing elements patterns classification using machine learningUkrainian Journal of Information Technology10.23939/ujit2024.01.0376:1(37-47)Online publication date: 2024
  • (2024)SpecGaussian with Latent Features: A High-quality Modeling of the View-dependent Appearance for 3D Gaussian SplattingProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681059(6270-6278)Online publication date: 28-Oct-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 42, Issue 6
December 2023
1565 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/3632123
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 December 2023
Published in TOG Volume 42, Issue 6

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. garment reconstruction
  2. sewing patterns
  3. transformer

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)332
  • Downloads (Last 6 weeks)42
Reflects downloads up to 14 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Simulation Technology for Apparel Designアパレル設計を支えるシミュレーション技術Journal of Japan Society of Kansei Engineering10.5057/kansei.22.1_1122:1(11-14)Online publication date: 31-Mar-2024
  • (2024)Models and means of clothing elements patterns classification using machine learningUkrainian Journal of Information Technology10.23939/ujit2024.01.0376:1(37-47)Online publication date: 2024
  • (2024)SpecGaussian with Latent Features: A High-quality Modeling of the View-dependent Appearance for 3D Gaussian SplattingProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681059(6270-6278)Online publication date: 28-Oct-2024
  • (2024)Spice·E: Structural Priors in 3D Diffusion using Cross-Entity AttentionACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657461(1-11)Online publication date: 13-Jul-2024
  • (2024)CNS-Edit: 3D Shape Editing via Coupled Neural Shape OptimizationACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657412(1-12)Online publication date: 13-Jul-2024
  • (2024)Inverse Garment and Pattern Modeling with a Differentiable SimulatorComputer Graphics Forum10.1111/cgf.1524943:7Online publication date: 7-Nov-2024
  • (2024)A Partition Based Method for Spectrum-Preserving Mesh SimplificationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.334161030:10(6839-6850)Online publication date: 1-Oct-2024
  • (2024)New Insights in Smooth Occluding Contours for Nonphotorealistic RenderingIEEE Computer Graphics and Applications10.1109/MCG.2023.333878444:1(76-85)Online publication date: 26-Jan-2024

View Options

Get Access

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media