A Review of Explainable Fashion Compatibility Modeling Methods
Article No.: 274, Pages 1 - 29
Abstract
The paper reviews methods used in the fashion compatibility recommendation domain. We select methods based on reproducibility, explainability, and novelty aspects and then organize them chronologically and thematically. We presented general characteristics of publicly available datasets that are related to the fashion compatibility recommendation task. Finally, we analyzed the representation bias of datasets, fashion-based algorithms’ sustainability, and explainable model assessment. The paper describes practical problem explanations, methodologies, and published datasets that may serve as an inspiration for further research. The proposed structure of the survey organizes knowledge in the fashion recommendation domain and will be beneficial for those who want to learn the topic from scratch, expand their knowledge, or find a new field for research. Furthermore, the information included in this paper could contribute to developing an effective and ethical fashion-based recommendation system.
References
[1]
Kenan E. Ak, Joo Hwee Lim, Jo Yew Tham, and Ashraf A. Kassim. 2018. Efficient multi-attribute similarity learning towards attribute-based fashion search. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, Lake Tahoe, NV, USA, 1671–1679.
[2]
Debopriyo Banerjee, Lucky Dhakad, Harsh Maheshwari, Muthusamy Chelliah, Niloy Ganguly, and Arnab Bhattacharya. 2022. Recommendation of compatible outfits conditioned on style. In European Conference on Information Retrieval. Springer, Stavanger, Norway, 35–50.
[3]
Aurore Bardey, Madison Booth, Giuliana Heger, and Jonas Larsson. 2022. Finding yourself in your wardrobe: An exploratory study of lived experiences with a capsule wardrobe. International Journal of Market Research 64, 1 (2022), 113–131.
[4]
Sean Bell and Kavita Bala. 2015. Learning visual similarity for product design with convolutional neural networks. ACM Transactions on Graphics (TOG) 34, 4 (2015), 1–10.
[5]
Roberto Cahuantzi, Xinye Chen, and Stefan Güttel. 2021. A comparison of LSTM and GRU networks for learning symbolic sequences. arXiv preprint arXiv:2107.02248 1, 02248 (2021).
[6]
Suthee Chaidaroon, Yi Fang, Min Xie, and Alessandro Magnani. 2019. Neural compatibility ranking for text-based fashion matching. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, New York, NY, USA, 1229–1232.
[7]
Huiyuan Chen, Yusan Lin, Fei Wang, and Hao Yang. 2021. Tops, bottoms, and shoes: Building capsule wardrobes via cross-attention tensor network. In Fifteenth ACM Conference on Recommender Systems. Association for Computing Machinery, New York, NY, USA, 453–462.
[8]
Wen Chen, Pipei Huang, Jiaming Xu, Xin Guo, Cheng Guo, Fei Sun, Chao Li, Andreas Pfadler, Huan Zhao, and Binqiang Zhao. 2019. POG: Personalized outfit generation for fashion recommendation at Alibaba iFashion. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Association for Computing Machinery, New York, NY, USA, 2662–2670.
[9]
Ching-I Cheng and Damon Shing-Min Liu. 2007. A decision making framework for dressing consultant. In 2007 IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making. IEEE, Honolulu, HI, USA, 267–271.
[10]
Davide Chicco. 2021. Siamese neural networks: An overview. Artificial Neural Networks 1 (2021), 73–94.
[11]
Antonia Creswell, Tom White, Vincent Dumoulin, Kai Arulkumaran, Biswa Sengupta, and Anil A. Bharath. 2018. Generative adversarial networks: An overview. IEEE Signal Processing Magazine 35, 1 (2018), 53–65.
[12]
Guillem Cucurull, Perouz Taslakian, and David Vazquez. 2019. Context-aware visual compatibility prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Long Beach, CA, USA, 12617–12626.
[13]
Zeyu Cui, Zekun Li, Shu Wu, Xiao-Yu Zhang, and Liang Wang. 2019. Dressing as a whole: Outfit compatibility learning based on node-wise graph neural networks. In The World Wide Web Conference. Association for Computing Machinery, New York, NY, USA, 307–317.
[14]
Yashar Deldjoo, Fatemeh Nazary, Arnau Ramisa, Julian Mcauley, Giovanni Pellegrini, Alejandro Bellogin, and Tommaso Di Noia. 2022. A review of modern fashion recommender systems. arXiv preprint arXiv:2202.02757 1, 02757 (2022).
[15]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Miami, FL, USA, 248–255.
[16]
Xue Dong, Xuemeng Song, Fuli Feng, Peiguang Jing, Xin-Shun Xu, and Liqiang Nie. 2019. Personalized capsule wardrobe creation with garment and user modeling. In Proceedings of the 27th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, USA, 302–310.
[17]
Xue Dong, Xuemeng Song, Na Zheng, Jianlong Wu, Hongjun Dai, and Liqiang Nie. 2022. TryonCM2: Try-on-enhanced fashion compatibility modeling framework. IEEE Transactions on Neural Networks and Learning Systems 0, 0 (2022), 1–12.
[18]
Guangyu Gao, Liling Liu, Li Wang, and Yihang Zhang. 2019. Fashion clothes matching scheme based on Siamese Network and AutoEncoder. Multimedia Systems 25, 6 (2019), 593–602.
[19]
Wei Gong and Laila Khalid. 2021. Aesthetics, personalization and recommendation: A survey on deep learning in fashion. arXiv preprint arXiv:2101.08301 1, 08301 (2021).
[20]
Jianping Gou, Baosheng Yu, Stephen J. Maybank, and Dacheng Tao. 2021. Knowledge distillation: A survey. International Journal of Computer Vision 129, 6 (2021), 1789–1819.
[21]
Xiaoling Gu, Fei Gao, Min Tan, and Pai Peng. 2020. Fashion analysis and understanding with artificial intelligence. Information Processing & Management 57, 5 (2020), 102276.
[22]
Weili Guan, Haokun Wen, Xuemeng Song, Chung-Hsing Yeh, Xiaojun Chang, and Liqiang Nie. 2021. Multimodal compatibility modeling via exploring the consistent and complementary correlations. In Proceedings of the 29th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, USA, 2299–2307.
[23]
Xianjing Han, Xuemeng Song, Yiyang Yao, Xin-Shun Xu, and Liqiang Nie. 2019. Neural compatibility modeling with probabilistic knowledge distillation. IEEE Transactions on Image Processing 29 (2019), 871–882.
[24]
Xintong Han, Zuxuan Wu, Yu-Gang Jiang, and Larry S. Davis. 2017. Learning fashion compatibility with bidirectional LST Ms. In Proceedings of the 25th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, USA, 1078–1086.
[25]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, USA, 770–778.
[26]
Shintami Chusnul Hidayati, Cheng-Chun Hsu, Yu-Ting Chang, Kai-Lung Hua, Jianlong Fu, and Wen-Huang Cheng. 2018. What dress fits me best? Fashion recommendation on the clothing style for personal body shape. In Proceedings of the 26th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, USA, 438–446.
[27]
Yuxin Hou, Eleonora Vig, Michael Donoser, and Loris Bazzani. 2021. Learning attribute-driven disentangled representations for interactive fashion retrieval. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, Montreal, Canada, 12147–12157.
[28]
Yang Hu, Xi Yi, and Larry S. Davis. 2015. Collaborative fashion recommendation: A functional tensor factorization approach. In Proceedings of the 23rd ACM International Conference on Multimedia. IEEE, Montreal, Canada, 129–138.
[29]
Kazuya Kawakami. 2008. Supervised Sequence Labelling with Recurrent Neural Networks. Ph. D. Dissertation. Technical University of Munich.
[30]
Donghyun Kim, Kuniaki Saito, Samarth Mishra, Stan Sclaroff, Kate Saenko, and Bryan A. Plummer. 2021. Self-supervised visual attribute learning for fashion compatibility. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, Montreal, Canada, 1057–1066.
[31]
Seongjae Kim, Jinseok Seol, Holim Lim, and Sang-goo Lee. 2021. False negative distillation and contrastive learning for personalized outfit recommendation. arXiv preprint arXiv:2110.06483 1, 06483 (2021).
[32]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25 (2012).
[33]
Zhanghui Kuang, Yiming Gao, Guanbin Li, Ping Luo, Yimin Chen, Liang Lin, and Wayne Zhang. 2019. Fashion retrieval via graph reasoning networks on a similarity pyramid. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, Seoul, Korea (South), 3066–3075.
[34]
Katrien Laenen and Marie-Francine Moens. 2020. A comparative study of outfit recommendation methods with a focus on attention-based fusion. Information Processing & Management 57, 6 (2020), 102316.
[35]
Jui-Hsin Lai, Bo Wu, Xin Wang, Dan Zeng, Tao Mei, and Jingen Liu. 2019. Theme-matters: Fashion compatibility learning via theme attention. arXiv preprint arXiv:1912.06227 1, 06227 (2019).
[36]
Xingchen Li, Xiang Wang, Xiangnan He, Long Chen, Jun Xiao, and Tat-Seng Chua. 2020. Hierarchical fashion graph network for personalized outfit recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, New York, NY, USA, 159–168.
[37]
Yuncheng Li, Liangliang Cao, Jiang Zhu, and Jiebo Luo. 2017. Mining fashion outfit composition using an end-to-end deep learning approach on set data. IEEE Transactions on Multimedia 19, 8 (2017), 1946–1955.
[38]
Zekun Li, Zeyu Cui, Shu Wu, Xiaoyu Zhang, and Liang Wang. 2019. Semi-supervised compatibility learning across categories for clothing matching. In 2019 IEEE International Conference on Multimedia and Expo (ICME). Association for Computing Machinery, New York, NY, USA, 484–489.
[39]
Min Lin, Qiang Chen, and Shuicheng Yan. 2013. Network in network. arXiv preprint arXiv:1312.4400 1, 1 (2013).
[40]
Yusan Lin, Maryam Moosaei, and Hao Yang. 2020. OutfitNet: Fashion outfit recommendation with attention-based multiple instance learning. In Proceedings of The Web Conference 2020. Association for Computing Machinery, New York, NY, USA, 77–87.
[41]
Yujie Lin, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Jun Ma, and Maarten De Rijke. 2019. Explainable outfit recommendation with joint outfit matching and comment generation. IEEE Transactions on Knowledge and Data Engineering 32, 8 (2019), 1502–1516.
[42]
Yen-Liang Lin, Son Tran, and Larry S. Davis. 2020. Fashion outfit complementary item retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Seattle, WA, USA, 3311–3319.
[43]
Jinhuan Liu, Xuemeng Song, Zhumin Chen, and Jun Ma. 2019. Neural fashion experts: I know how to make the complementary clothing matching. Neurocomputing 359 (2019), 249–263.
[44]
Linlin Liu, Haijun Zhang, Yuzhu Ji, and Q. M. Jonathan Wu. 2019. Toward AI fashion design: An attribute-GAN model for clothing match. Neurocomputing 341 (2019), 156–167.
[45]
Linlin Liu, Haijun Zhang, and Dongliang Zhou. 2021. Clothing generation by multi-modal embedding: A compatibility matrix-regularized GAN model. Image and Vision Computing 107 (2021), 104097.
[46]
Zhi Lu, Yang Hu, Yan Chen, and Bing Zeng. 2021. Personalized outfit recommendation with learnable anchors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Nashville, TN, USA, 12722–12731.
[47]
Zhi Lu, Yang Hu, Yunchao Jiang, Yan Chen, and Bing Zeng. 2019. Learning binary code for personalized fashion recommendation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Seattle, WA, USA, 10562–10570.
[48]
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. 2015. Image-based recommendations on styles and substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, New York, NY, USA, 43–52.
[49]
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 1, 1 (2013).
[50]
Seyed Omid Mohammadi and Ahmad Kalhor. 2021. Smart fashion: A review of AI applications in the fashion & apparel industry. arXiv preprint arXiv:2111.00905 1, 00905 (2021).
[51]
Maryam Moosaei, Yusan Lin, Ablaikhan Akhazhanov, Huiyuan Chen, Fei Wang, and Hao Yang. 2022. OutfitGAN: Learning compatible items for generative fashion outfits. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, New Orleans, LA, USA, 2273–2277.
[52]
Takuma Nakamura and Ryosuke Goto. 2018. Outfit generation and style extraction via bidirectional LSTM and autoencoder. arXiv preprint arXiv:1807.03133 1, 03133 (2018).
[53]
Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). EMNLP, Doha, Qatar, 1532–1543.
[54]
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012. BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 1 (2012), 1–14.
[55]
Ludger Rüschendorf. 1985. The Wasserstein distance and approximation theorems. Probability Theory and Related Fields 70, 1 (1985), 117–129.
[56]
Dikshant Sagar, Jatin Garg, Prarthana Kansal, Sejal Bhalla, Rajiv Ratn Shah, and Yi Yu. 2020. PAI-BPR: Personalized outfit recommendation scheme with attribute-wise interpretability. In 2020 IEEE Sixth International Conference on Multimedia Big Data (BigMM). IEEE, New Delhi, India, 221–230.
[57]
Hiroyuki Shinnou, Masayuki Asahara, Kanako Komiya, and Minoru Sasaki. 2017. NWJC2Vec: Word embedding data constructed from NINJAL web Japanese corpus. Journal of Natural Language Processing 24, 5 (2017), 705–720.
[58]
Xuemeng Song, Fuli Feng, Xianjing Han, Xin Yang, Wei Liu, and Liqiang Nie. 2018. Neural compatibility modeling with attentive knowledge distillation. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. Association for Computing Machinery, New York, NY, USA, 5–14.
[59]
Xuemeng Song, Fuli Feng, Jinhuan Liu, Zekun Li, Liqiang Nie, and Jun Ma. 2017. NeuroStylist: Neural compatibility modeling for clothing matching. In Proceedings of the 25th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, USA, 753–761.
[60]
Xuemeng Song, Xianjing Han, Yunkai Li, Jingyuan Chen, Xin-Shun Xu, and Liqiang Nie. 2019. GP-BPR: Personalized compatibility modeling for clothing matching. In Proceedings of the 27th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, USA, 320–328.
[61]
Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A. Alemi. 2017. Inception-v4, Inception-ResNet and the impact of residual connections on learning. In Thirty-First AAAI Conference on Artificial Intelligence. AAAI, San Francisco, CA, USA, 4278–4284.
[62]
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Boston, MA, USA, 1–9.
[63]
Reuben Tan, Mariya I. Vasileva, Kate Saenko, and Bryan A. Plummer. 2019. Learning similarity conditions without explicit supervision. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, Seoul, Korea, 10373–10382.
[64]
Mariya I. Vasileva, Bryan A. Plummer, Krishna Dusad, Shreya Rajpal, Ranjitha Kumar, and David Forsyth. 2018. Learning type-aware embeddings for fashion compatibility. In Proceedings of the European Conference on Computer Vision (ECCV). Springer, Munich, Germany, 390–405.
[65]
Andreas Veit, Serge Belongie, and Theofanis Karaletsos. 2017. Conditional similarity networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Honolulu, HI, USA, 830–838.
[66]
Andreas Veit, Balazs Kovacs, Sean Bell, Julian McAuley, Kavita Bala, and Serge Belongie. 2015. Learning visual clothing style with heterogeneous dyadic co-occurrences. In Proceedings of the IEEE International Conference on Computer Vision. IEEE, Santiago, Chile, 4642–4650.
[67]
Jianfeng Wang, Xiaochun Cheng, Ruomei Wang, and Shaohui Liu. 2021. Learning outfit compatibility with graph attention network and visual-semantic embedding. In 2021 IEEE International Conference on Multimedia and Expo (ICME). IEEE, Shenzhen, China, 1–6.
[68]
Zhen Wang and Hongyan Quan. 2019. Fashion outfit composition combining sequential learning and deep aesthetic network. In 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, Budapest, Hungary, 1–7.
[69]
Xin Wang, Bo Wu, and Yueqi Zhong. 2019. Outfit compatibility prediction and diagnosis with multi-layered comparison network. In Proceedings of the 27th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, USA, 329–337.
[70]
Jane E. Workman and Cathryn M. Studak. 2006. Fashion consumers and fashion problem recognition style. International Journal of Consumer Studies 30, 1 (2006), 75–84.
[71]
Xun Yang, Yunshan Ma, Lizi Liao, Meng Wang, and Tat-Seng Chua. 2019. TransNFCM: Translation-based neural fashion compatibility modeling. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. AAAI, Honolulu, Hawaii, USA, 403–410.
[72]
Xin Yang, Xuemeng Song, Fuli Feng, Haokun Wen, Ling-Yu Duan, and Liqiang Nie. 2021. Attribute-wise explainable fashion compatibility modeling. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 17, 1 (2021), 1–21.
[73]
Xuewen Yang, Dongliang Xie, Xin Wang, Jiangbo Yuan, Wanying Ding, and Pengyun Yan. 2020. Learning tuple compatibility for conditional outfit recommendation. In Proceedings of the 28th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, USA, 2636–2644.
[74]
Ruiping Yin, Kan Li, Jie Lu, and Guangquan Zhang. 2019. Enhancing fashion recommendation with visual compatibility relationship. In The World Wide Web Conference. Association for Computing Machinery, New York, NY, USA, 3434–3440.
[75]
Huijing Zhan, Jie Lin, Kenan Emir Ak, Boxin Shi, Ling-Yu Duan, and Alex C. Kot. 2021. Attentive attribute-aware fashion knowledge graph for outfit preference prediction. IEEE Transactions on Multimedia 24 (2021), 819–831.
[76]
Heming Zhang, Xuewen Yang, Jianchao Tan, Chi-Hao Wu, Jue Wang, and C.-C. Jay Kuo. 2020. Learning color compatibility in fashion outfits. arXiv preprint arXiv:2007.02388 01 (2020).
[77]
Ye Zhang and Byron Wallace. 2015. A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification. arXiv preprint arXiv:1510.03820 1, 1 (2015).
[78]
Na Zheng, Xuemeng Song, Qingying Niu, Xue Dong, Yibing Zhan, and Liqiang Nie. 2021. Collocation and try-on network: Whether an outfit is compatible. In Proceedings of the 29th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, USA, 309–317.
Index Terms
- A Review of Explainable Fashion Compatibility Modeling Methods
Recommendations
Computational Technologies for Fashion Recommendation: A Survey
Fashion recommendation is a key research field in computational fashion research and has attracted considerable interest in the computer vision, multimedia, and information retrieval communities in recent years. Due to the great demand for applications, ...
Learning Fashion Compatibility with Bidirectional LSTMs
MM '17: Proceedings of the 25th ACM international conference on MultimediaThe ubiquity of online fashion shopping demands effective recommendation services for customers. In this paper, we study two types of fashion recommendation: (i) suggesting an item that matches existing components in a set to form a stylish outfit (a ...
Enhancing Fashion Recommendation with Visual Compatibility Relationship
WWW '19: The World Wide Web ConferenceWith the increasing of online shopping services, fashion recommendation plays an important role in daily online shopping scenes. A lot of recommender systems have been developed with visual information. However, few works take into account compatibility ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
November 2024
977 pages
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 June 2024
Online AM: 11 May 2024
Accepted: 23 April 2024
Revised: 28 October 2023
Received: 12 October 2022
Published in CSUR Volume 56, Issue 11
Check for updates
Author Tags
Qualifiers
- Survey
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 376Total Downloads
- Downloads (Last 12 months)376
- Downloads (Last 6 weeks)61
Reflects downloads up to 22 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in