Cited By
View all- Zhong SGatti EHardwick JRibul MCho YObrist M(2025) LLM-mediated domain-specific voice agents: the case of TextileBot Behaviour & Information Technology10.1080/0144929X.2025.2456667(1-33)Online publication date: 3-Feb-2025
Design Digital Multisensory Textile Experiences
Author: 
Shu ZhongAuthors Info & Claims
Pages 642 - 646
Abstract
The rise of Machine Learning (ML) is gradually digitalizing and reshaping the fashion industry, which is under pressure to achieve Net Zero. However, the integration of ML/AI for sustainable and circular practices remains limited due to a lack of domain-specific knowledge and data. My doctoral research aims to bridge this gap by designing digital multisensory textile experiences that enhance the understanding of the textile domain for both AI systems and humans. To this end, I develop TextileNet, the first fashion dataset using textile taxonomies for textile materials identification and classification via computer vision, and TextileBot, a domain-specific conversational agent. TextileBot integrates textile taxonomies with large language models (LLMs) to engage consumers in sustainable practices. Additionally, my research explores how multisensory experiences can improve user understanding and how AI perceives textiles. The overarching goal is to embed human expertise into machines, design immersive multisensory experiences, and facilitate natural human-AI interactions that promote sustainable practices.
References
[1]
Merav Allouch, Amos Azaria, and Rina Azoulay. 2021. Conversational agents: Goals, technologies, vision and challenges. Sensors 21, 24 (2021), 8448.
[2]
Kerrice Bailey, Aman Basu, and Sapna Sharma. 2022. The Environmental Impacts of Fast Fashion on Water Quality: A Systematic Review. Water 14, 7 (2022), 1073.
[3]
Ms Aayushi Bansal, Dr Rewa Sharma, and Dr Mamta Kathuria. 2022. A systematic review on data scarcity problem in deep learning: solution and applications. ACM Computing Surveys (CSUR) 54, 10s (2022), 1–29.
[4]
Amanda Baughan, Xuezhi Wang, Ariel Liu, Allison Mercurio, Jilin Chen, and Xiao Ma. 2023. A Mixed-Methods Approach to Understanding User Trust after Voice Assistant Failures. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–16.
[5]
Rodrigo Bavaresco, Diórgenes Silveira, Eduardo Reis, Jorge Barbosa, Rodrigo Righi, Cristiano Costa, Rodolfo Antunes, Marcio Gomes, Clauter Gatti, Mariangela Vanzin, 2020. Conversational agents in business: A systematic literature review and future research directions. Computer Science Review 36 (2020), 100239.
[6]
Wen-Huang Cheng, Sijie Song, Chieh-Yun Chen, Shintami Chusnul Hidayati, and Jiaying Liu. 2020. Fashion Meets Computer Vision: A Survey. arXiv preprint arXiv:2003.13988 (2020).
[7]
Desiree Cho, Shu Zhong, Madhan Kumar Vasudevan, and Marianna Obrist. 2024. Mindful Touch: Mid-Air Haptics Facilitates Novices’ Experiences of Audio-Guided Mindfulness Meditation. Available at SSRN 4768789 (2024).
[8]
Mariachiara Colucci, Annamaria Tuan, and Marco Visentin. 2020. An empirical investigation of the drivers of CSR talk and walk in the fashion industry. Journal of Cleaner Production 248 (2020), 119200.
[9]
Kirsti Cura, Niko Rintala, Taina Kamppuri, Eetta Saarimäki, and Pirjo Heikkilä. 2021. Textile recognition and sorting for recycling at an automated line using near infrared spectroscopy. Recycling 6, 1 (2021), 11.
[10]
Textile Exchange. 2021. Preferred Fiber & Materials Market Report 2021. TextileExchange.
[11]
The Ellen MacArthur Foundation. [n. d.]. Fashion and a circular economy | Ellen MacArthur Foundation. https://ellenmacarthurfoundation.org/topics/fashion/overview
[12]
Nancy Frey and Douglas Fisher. 2008. Teaching visual literacy: Using comic books, graphic novels, anime, cartoons, and more to develop comprehension and thinking skills. Corwin Press.
[13]
Alexander Frummet, David Elsweiler, and Bernd Ludwig. 2022. “What Can I Cook with these Ingredients?”-Understanding Cooking-Related Information Needs in Conversational Search. ACM Transactions on Information Systems (TOIS) 40, 4 (2022), 1–32.
[14]
Sheng Guo, Weilin Huang, Xiao Zhang, Prasanna Srikhanta, Yin Cui, Yuan Li, Hartwig Adam, Matthew R Scott, and Serge Belongie. 2019. The imaterialist fashion attribute dataset. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops. 0–0.
[15]
Fu Jia, Shiyuan Yin, Lujie Chen, and Xiaowei Chen. 2020. The circular economy in the textile and apparel industry: A systematic literature review. Journal of Cleaner Production 259 (2020), 120728.
[16]
Jon Kabat-Zinn. 1982. An outpatient program in behavioral medicine for chronic pain patients based on the practice of mindfulness meditation: Theoretical considerations and preliminary results. General hospital psychiatry 4, 1 (1982), 33–47.
[17]
Jean Kaddour, Joshua Harris, Maximilian Mozes, Herbie Bradley, Roberta Raileanu, and Robert McHardy. 2023. Challenges and Applications of Large Language Models. arXiv preprint arXiv:2307.10169 (2023).
[18]
Asimananda Khandual and Swikruti Pradhan. 2019. Fashion brands and consumers approach towards sustainable fashion. In Fast fashion, fashion brands and sustainable consumption. Springer, 37–54.
[19]
Sheetal Kusal, Shruti Patil, Jyoti Choudrie, Ketan Kotecha, Sashikala Mishra, and Ajith Abraham. 2022. AI-based Conversational Agents: A Scoping Review from Technologies to Future Directions. IEEE Access (2022).
[20]
Mina Lee, Megha Srivastava, Amelia Hardy, John Thickstun, Esin Durmus, Ashwin Paranjape, Ines Gerard-Ursin, Xiang Lisa Li, Faisal Ladhak, Frieda Rong, 2022. Evaluating Human-Language Model Interaction. arXiv preprint arXiv:2212.09746 (2022).
[21]
Peter Lee, Sebastien Bubeck, and Joseph Petro. 2023. Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine. New England Journal of Medicine 388, 13 (2023), 1233–1239.
[22]
Jared R Lindahl, Nathan E Fisher, David J Cooper, Rochelle K Rosen, and Willoughby B Britton. 2017. The varieties of contemplative experience: A mixed-methods study of meditation-related challenges in Western Buddhists. PloS one 12, 5 (2017), e0176239.
[23]
Ziwei Liu, Ping Luo, Shi Qiu, Xiaogang Wang, and Xiaoou Tang. 2016. Deepfashion: Powering robust clothes recognition and retrieval with rich annotations. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1096–1104.
[24]
Ellen MacArthur 2013. Towards the circular economy. Journal of Industrial Ecology 2 (2013), 23–44.
[25]
Ellen MacArthur 2014. Towards the circular economy: Accelerating the scale-up across global supply chains. In World Economic Forum.
[26]
Selina Meyer, David Elsweiler, Bernd Ludwig, Marcos Fernandez-Pichel, and David E Losada. 2022. Do We Still Need Human Assessors? Prompt-Based GPT-3 User Simulation in Conversational AI. In Proceedings of the 4th Conference on Conversational User Interfaces. 1–6.
[27]
Subramanian Senthilkannan Muthu. 2018. Circular Economy in Textiles and Apparel: Processing, Manufacturing, and Design. Woodhead Publishing.
[28]
Kirsi Niinimäki, Greg Peters, Helena Dahlbo, Patsy Perry, Timo Rissanen, and Alison Gwilt. 2020. The environmental price of fast fashion. Nature Reviews Earth & Environment 1, 4 (2020), 189–200.
[29]
Donald A Norman and Stephen W Draper. 1986. User centered system design: New perspectives on human-computer interaction. (1986).
[30]
Nynne Nørup, Kaj Pihl, Anders Damgaard, and Charlotte Scheutz. 2019. Evaluation of a European textile sorting centre: Material flow analysis and life cycle inventory. Resources, Conservation and Recycling 143 (2019), 310–319.
[31]
AK Roy Choudhury. 2014. Environmental impacts of the textile industry and its assessment through life cycle assessment. In Roadmap to sustainable textiles and clothing. Springer, 1–39.
[32]
Katie Seaborn, Norihisa P Miyake, Peter Pennefather, and Mihoko Otake-Matsuura. 2021. Voice in human–agent interaction: A survey. ACM Computing Surveys (CSUR) 54, 4 (2021), 1–43.
[33]
Madhan Kumar Vasudevan, Shu Zhong, Jan Kučera, Desiree Cho, and Marianna Obrist. 2023. MindTouch: Effect of Mindfulness Meditation on Mid-Air Tactile Perception. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–12.
[34]
Carlos Velasco and Marianna Obrist. 2020. Multisensory Experiences: Where the senses meet technology. Oxford University Press.
[35]
JD Zamfirescu-Pereira, Heather Wei, Amy Xiao, Kitty Gu, Grace Jung, Matthew G Lee, Bjoern Hartmann, and Qian Yang. 2023. Herding AI Cats: Lessons from Designing a Chatbot by Prompting GPT-3. (2023).
[36]
Shu Zhong, Elia Gatti, Youngjun Cho, and Marianna Obrist. 2024. Exploring Human-AI Perception Alignment in Sensory Experiences: Do LLMs Understand Textile Hand?arXiv preprint arXiv:2406.06587 (2024).
[37]
Shu Zhong, Elia Gatti, James Hardwick, Miriam Ribul, Youngjun Cho, and Marianna Obrist. 2024. LLM-Mediated Domain-Specific Voice Agents: The Case of TextileBot. arXiv preprint arXiv:2406.10590 (2024).
[38]
Shu Zhong, Miriam Ribul, Youngjun Cho, and Marianna Obrist. 2023. TextileNet: A Material Taxonomy-based Fashion Textile Dataset. arXiv preprint arXiv:2301.06160 (2023).
[39]
Xingxing Zou, Xiangheng Kong, Waikeung Wong, Congde Wang, Yuguang Liu, and Yang Cao. 2019. Fashionai: A hierarchical dataset for fashion understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 0–0.
Index Terms
- Design Digital Multisensory Textile Experiences
Recommendations
Feeling Textiles through AI: An exploration into Multimodal Language Models and Human Perception Alignment
ICMI '24: Proceedings of the 26th International Conference on Multimodal InteractionHuman-artificial intelligence (AI) alignment ensures that AI systems align with human goals and behaviors. This paper introduces perceptual alignment as a critical aspect of this alignment, focusing on the concurrence between human judgments and AI ...
Threads of Traceability: Textile IDs in the Fabric of Sustainable Fashion
DIS '24: Proceedings of the 2024 ACM Designing Interactive Systems ConferenceTextile fabrication, an ancient human technology, has evolved over millennia, transitioning from a focus on affordability and speed to a current emphasis on sustainability. With Textile ID, we envision a digital garment passport that seamlessly ...
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing SystemsA surge of interest in explainable AI (XAI) has led to a vast collection of algorithmic work on the topic. While many recognize the necessity to incorporate explainability features in AI systems, how to address real-world user needs for understanding AI ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
November 2024
725 pages
ISBN:9798400704628
DOI:10.1145/3678957
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 November 2024
Check for updates
Author Tags
Qualifiers
- Extended-abstract
- Research
- Refereed limited
Funding Sources
Conference
ICMI '24
ICMI '24: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION
November 4 - 8, 2024
San Jose, Costa Rica
Acceptance Rates
Overall Acceptance Rate 453 of 1,080 submissions, 42%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 53Total Downloads
- Downloads (Last 12 months)53
- Downloads (Last 6 weeks)13
Reflects downloads up to 16 Feb 2025
Other Metrics
Citations
Cited By
View all- Zhong SGatti EHardwick JRibul MCho YObrist M(2025) LLM-mediated domain-specific voice agents: the case of TextileBot Behaviour & Information Technology10.1080/0144929X.2025.2456667(1-33)Online publication date: 3-Feb-2025
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format