Nothing Special   »   [go: up one dir, main page]

skip to main content
survey

Recommender Systems Leveraging Multimedia Content

Published: 28 September 2020 Publication History

Abstract

Recommender systems have become a popular and effective means to manage the ever-increasing amount of multimedia content available today and to help users discover interesting new items. Today’s recommender systems suggest items of various media types, including audio, text, visual (images), and videos. In fact, scientific research related to the analysis of multimedia content has made possible effective content-based recommender systems capable of suggesting items based on an analysis of the features extracted from the item itself. The aim of this survey is to present a thorough review of the state-of-the-art of recommender systems that leverage multimedia content, by classifying the reviewed papers with respect to their media type, the techniques employed to extract and represent their content features, and the recommendation algorithm. Moreover, for each media type, we discuss various domains in which multimedia content plays a key role in human decision-making and is therefore considered in the recommendation process. Examples of the identified domains include fashion, tourism, food, media streaming, and e-commerce.

References

[1]
Charu C. Aggarwal. 2016a. Content-based recommender systems. In Recommender Systems. Springer, 139--166.
[2]
Charu C. Aggarwal. 2016b. Ensemble-based and hybrid recommender systems. In Recommender Systems. Springer, 199--224.
[3]
Taleb Alashkar, Songyao Jiang, and Yun Fu. 2017a. Rule-based facial makeup recommendation system. In Proceedings of the 12th IEEE International Conference on Automatic Face 8 Gesture Recognition (FG’17). IEEE, 325--330.
[4]
Taleb Alashkar, Songyao Jiang, Shuyang Wang, and Yun Fu. 2017b. Examples-rules guided deep neural network for makeup recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence. 941--947.
[5]
Massimiliano Albanese, Angelo Chianese, Antonio d’Acierno, Vincenzo Moscato, and Antonio Picariello. 2010. A multimedia recommender integrating object features and user behavior. Multimedia Tools Applic. 50 (2010), 563--585.
[6]
Massimiliano Albanese, Antonio d’Acierno, Vincenzo Moscato, Fabio Persia, and Antonio Picariello. 2013. A multimedia recommender system. ACM Trans. Internet Technol. 13, 1 (2013).
[7]
J. Allen. 1977. Short term spectral analysis, synthesis, and modification by discrete Fourier transform. IEEE Trans. Acoust. Speech Sig. Proc. 25, 3 (June 1977), 235--238.
[8]
Fernando Amat, Ashok Chandrashekar, Tony Jebara, and Justin Basilico. 2018. Artwork personalization at Netflix. In Proceedings of the 12th ACM Conference on Recommender Systems. ACM, 487--488.
[9]
Ivana Andjelkovic, Denis Parra, and John O’Donovan. 2019. Moodplay: Interactive music recommendation based on Artists’ mood similarity. Int. J. Hum.-comput. Stud. 121 (2019), 142--159.
[10]
Vito Walter Anelli, Yashar Deldjoo, Tommaso Di Noia, and Antonio Ferrara. 2019. Towards effective device-aware federated learning. In Proceedings of the International Conference of the Italian Association for Artificial Intelligence. Springer, 477--491.
[11]
D. Azucar, D. Marengo, and M. Settanni. 2018. Predicting the big 5 personality traits from digital footprints on social media: A meta-analysis. Personal. Indiv. Dif. 124 (2018), 150--159.
[12]
Ricardo Baeza-Yates and Berthier Ribeiro-Neto. 2011. Modern Information Retrieval—The Concepts and Technology Behind Search (2nd ed.). Addison-Wesley, Pearson, Harlow, England.
[13]
Tadas Baltrusaitis, Chaitanya Ahuja, and Louis-Philippe Morency. 2019. Multimodal machine learning: A survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intell. 41, 2 (2019), 423--443.
[14]
Ilaria Bartolini, Vincenzo Moscato, Ruggero G. Pensa, Antonio Penta, Antonio Picariello, Carlo Sansone, and Maria Luisa Sapino. 2013. Recommending multimedia objects in cultural heritage applications. In Proceedings of the International Conference on Image Analysis and Processing. Springer, 257--267.
[15]
Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. 2006. Surf: Speeded up robust features. In Proceedings of the European Conference on Computer Vision. Springer, 404--417.
[16]
Sergio Benini, Luca Canini, and Riccardo Leonardi. 2011. A connotative space for supporting movie affective recommendation. IEEE Trans. Multimedia 13, 6 (2011), 1356--1370.
[17]
Thierry Bertin-Mahieux, Daniel P. W. Ellis, Brian Whitman, and Paul Lamere. 2011. The million song dataset. In Proceedings of the 12th International Society for Music Information Retrieval Conference. 591--596.
[18]
Jesús Bobadilla, Fernando Ortega, Antonio Hernando, and Abraham Gutiérrez. 2013. Recommender systems survey. Knowledge-based Syst. 46 (2013), 109--132.
[19]
Geoffray Bonnin and Dietmar Jannach. 2014. Automated generation of music playlists: Survey and experiments. ACM Comput. Surv. 47, 2 (Nov. 2014).
[20]
Steven Bourke, Kevin McCarthy, and Barry Smyth. 2011. The social camera: A case-study in contextual image recommendation. In Proceedings of the 16th International Conference on Intelligent User Interfaces. ACM, 13--22.
[21]
Sabri Boutemedjet and Djemel Ziou. 2006. A generative graphical model for collaborative filtering of visual content. In Proceedings of the Industrial Conference on Data Mining. Springer, 404--415.
[22]
Sabri Boutemedjet and Djemel Ziou. 2008. A graphical model for context-aware visual content recommendation. IEEE Trans. Multimedia 10, 1 (2008), 52--62.
[23]
Sabri Boutemedjet, Djemel Ziou, and Nizar Bouguila. 2008. Unsupervised feature selection for accurate recommendation of high-dimensional image data. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 177--184.
[24]
Jiajun Bu, Shulong Tan, Chun Chen, Can Wang, Hao Wu, Lijun Zhang, and Xiaofei He. 2010. Music recommendation by unified hypergraph: Combining social media information and music content. In Proceedings of the ACM Multimedia Conference. ACM, 391--400.
[25]
Robin Burke. 2002. Hybrid recommender systems: Survey and experiments. User Model. User-adapt. Interact. 12, 4 (2002), 331--370.
[26]
Luca Canini, Sergio Benini, and Riccardo Leonardi. 2013. Affective recommendation of movies based on selected connotative features. IEEE Trans. Circ. Syst. Vid. Technol. 23, 4 (2013), 636--647.
[27]
Erion Çano and Maurizio Morisio. 2019. Hybrid Recommender Systems: A Systematic Literature Review. arxiv:cs.IR/1901.03888 (2019).
[28]
Fabio Celli, Elia Bruni, and Bruno Lepri. 2014. Automatic personality and interaction style recognition from Facebook profile pictures. In Proceedings of the 22nd ACM International Conference on Multimedia (MM’14). ACM, New York, NY, 1101--1104.
[29]
S. Chen, J. L. Moore, D. Turnbull, and T. Joachims. 2012. Playlist prediction via metric embedding. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’12).
[30]
Xiaojie Chen, Pengpeng Zhao, Jiajie Xu, Zhixu Li, Lei Zhao, Yanchi Liu, Victor S. Sheng, and Zhiming Cui. 2018. Exploiting visual contents in posters and still frames for movie recommendation. IEEE Access 6 (2018), 68874--68881.
[31]
Heng-Yu Chi, Chun-Chieh Chen, Wen-Huang Cheng, and Ming-Syan Chen. 2016. UbiShop: Commercial item recommendation using visual part-based object representation. Multimedia Tools Applic. 75, 23 (2016), 16093--16115.
[32]
Wei-Ta Chu and Ya-Lun Tsai. 2017. A hybrid recommendation system considering visual information for predicting favorite restaurants. World Wide Web 20, 6 (2017), 1313--1331.
[33]
Kyung-Yong Chung. 2014. Effect of facial makeup style recommendation on visual sensibility. Multimedia Tools Applic. 71, 2 (2014), 843--853.
[34]
Paolo Cremonesi, Mehdi Elahi, and Franca Garzotto. 2017. User interface patterns in recommendation-empowered content intensive multimedia applications. Multimedia Tools Applic. 76, 4 (2017), 5275--5309.
[35]
Paolo Cremonesi, Franca Garzotto, and Roberto Turrin. 2013. User-centric vs. system-centric evaluation of recommender systems. In Proceedings of the IFIP Conference on Human-Computer Interaction. Springer, 334--351.
[36]
Guillem Cucurull, Pau Rodríguez, V. Oguz Yazici, Josep M. Gonfaus, F. Xavier Roca, and Jordi Gonzàlez. 2018. Deep inference of personality traits by integrating image and word use in social networks. arXiv preprint arXiv:1802.06757 (2018).
[37]
Bin Cui, Anthony K. H. Tung, Ce Zhang, and Zhe Zhao. 2010. Multiple feature fusion for social media applications. In Proceedings of the ACM SIGMOD International Conference on Management of Data. ACM, 435--446.
[38]
Peng Cui, Zhiyu Wang, and Zhou Su. 2014. What videos are similar with you?: Learning a common attributed representation for video recommendation. In Proceedings of the 22nd ACM International Conference on Multimedia. ACM, 597--606.
[39]
Maurizio Ferrari Dacrema, Paolo Cremonesi, and Dietmar Jannach. 2019. Are we really making much progress? A worrying analysis of recent neural recommendation approaches. In Proceedings of the 13th ACM Conference on Recommender Systems (RecSys’19). 101--109.
[40]
Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), Vol. 1. IEEE, 886--893.
[41]
Ritendra Datta, Dhiraj Joshi, Jia Li, and James Z. Wang. 2006. Studying aesthetics in photographic images using a computational approach. In Proceedings of the European Conference on Computer Vision. Springer, 288--301.
[42]
Marco de Gemmis, Pasquale Lops, Cataldo Musto, Fedelucio Narducci, and Giovanni Semeraro. 2015. Semantics-aware content-based recommender systems. In Recommender Systems Handbook. Springer, 119--159.
[43]
N. Dehak, P. J. Kenny, R. Dehak, P. Dumouchel, and P. Ouellet. 2011. Front-end factor analysis for speaker verification. IEEE Trans. Aud. Speech, Lang. Proc. 19, 4 (May 2011), 788--798.
[44]
Yashar Deldjoo, Vito Walter Anelli, Hamed Zamani, Alejandro Bellogin, and Tommaso Di Noia. 2020a. A flexible framework for evaluating user and item fairness in recommender systems. User Modeling and User-Adapted Interaction (2020).
[45]
Yashar Deldjoo, Mihai Gabriel Constantin, Hamid Eghbal-Zadeh, Bogdan Ionescu, Markus Schedl, and Paolo Cremonesi. 2018a. Audio-visual encoding of multimedia content for enhancing movie recommendations. In Proceedings of the 12th ACM Conference on Recommender Systems. ACM, 455--459.
[46]
Yashar Deldjoo, Mihai Gabriel Constantin, Bogdan Ionescu, Markus Schedl, and Paolo Cremonesi. 2018b. MMTF-14k: A multifaceted movie trailer feature dataset for recommendation and retrieval. In Proceedings of the 9th ACM Multimedia Systems Conference. ACM, 450--455.
[47]
Yashar Deldjoo, Maurizio Ferrari Dacrema, Mihai Gabriel Constantin, Hamid Eghbal-Zadeh, Stefano Cereda, Markus Schedl, Bogdan Ionescu, and Paolo Cremonesi. 2019. Movie genome: Alleviating new item cold start in movie recommendation. User Model. User-Adapt. Interact. 29, 2 (2019), 291--343.
[48]
Yashar Deldjoo, Tommaso Di Noia, and Felice Antonio Merra. 2020b. How dataset characteristics affect the robustness of collaborative recommendation models. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval.
[49]
Yashar Deldjoo, Mehdi Elahi, and Paolo Cremonesi. 2016a. Using visual features and latent factors for movie recommendation. In Proceedings of the 3rd Workshop on New Trends in Content-Based Recommender Systems co-located with ACM Conference on Recommender Systems (RecSys'16), Boston, MA, USA, September 16, 2016, Vol. 1673. CEUR-WS.org, 15--18. Retrieved from http://ceur-ws.org/Vol-1673/paper3.pdf.
[50]
Yashar Deldjoo, Mehdi Elahi, Paolo Cremonesi, Franca Garzotto, Pietro Piazzolla, and Massimo Quadrana. 2016b. Content-based video recommendation system based on stylistic visual features. J. Data Seman. 5, 2 (2016), 99--113.
[51]
Yashar Deldjoo, Mehdi Elahi, Massimo Quadrana, and Paolo Cremonesi. 2018c. Using visual features based on MPEG-7 and deep learning for movie recommendation. Int. J. Multim. Inf. Retr. 7, 4 (2018), 207--219.
[52]
Yashar Deldjoo, Tommaso Di Noia, and Felice Antonio Merra. 2020c. Adversarial machine learning in recommender systems: State of the art and challenges. CoRR abs/2005.10322 (2020).
[53]
Yashar Deldjoo, Markus Schedl, Paolo Cremonesi, and Gabriella Pasi. 2018d. Content-based multimedia recommendation systems: Definition and application domains. In Proceedings of the 9th Italian Information Retrieval Workshop, Rome, Italy, May, 28-30, 2018, Vol. 2140. CEUR-WS.org. Retrieved from http://ceur-ws.org/Vol-2140/paper15.pdf.
[54]
Zhengyu Deng, Jitao Sang, and Changsheng Xu. 2013. Personalized video recommendation based on cross-platform user modeling. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME’13). IEEE, 1--6.
[55]
Tommaso Di Noia, Daniele Malitesta, and Felice Antonio Merra. 2020. TAaMR: Targeted adversarial attack against multimedia recommender systems. In Proceedings of the 3rd International Workshop on Dependable and Secure Machine Learning (DSML’20). IEEE.
[56]
Xingzhong Du, Hongzhi Yin, Ling Chen, Yang Wang, Yi Yang, and Xiaofang Zhou. 2020. Personalized video recommendation using rich contents from videos. IEEE Trans. Knowl. Data Eng. 32, 3 (2020), 492--505.
[57]
Michael D. Ekstrand, John T. Riedl, Joseph A. Konstan et al. 2011. Collaborative filtering recommender systems. Found. Trends® Hum.-comput. Interact. 4, 2 (2011), 81--173.
[58]
David Elsweiler, Christoph Trattner, and Morgan Harvey. 2017. Exploiting food choice biases for healthier recipe recommendation. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 575--584.
[59]
Aleksandr Farseev, Liqiang Nie, Mohammad Akbari, and Tat-Seng Chua. 2015. Harvesting multiple sources for user profile learning: A big data study. In Proceedings of the 5th ACM International Conference on Multimedia Retrieval. ACM, 235--242.
[60]
Aleksandr Farseev, Ivan Samborskii, Andrey Filchenkov, and Tat-Seng Chua. 2017. Cross-domain recommendation via clustering on multi-layer graphs. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 195--204.
[61]
Bruce Ferwerda and Marko Tkalcic. 2018. Predicting users’ personality from Instagram pictures: Using visual and/or content features? In Proceedings of the 26th Conference on User Modeling, Adaptation and Personalization (UMAP’18). ACM, New York, NY, 157--161.
[62]
Bruce Ferwerda, Marko Tkalcic, and Markus Schedl. 2017. Personality traits and music genres: What do people prefer to listen to? In Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization (UMAP’17). ACM, New York, NY, 285--288.
[63]
Zeno Gantner, Lucas Drumond, Christoph Freudenthaler, Steffen Rendle, and Lars Schmidt-Thieme. 2010. Learning attribute-to-feature mappings for cold-start recommendations. In Proceedings of the IEEE 10th International Conference on Data Mining (ICDM’10). IEEE, 176--185.
[64]
Mouzhi Ge and Fabio Persia. 2018. Evaluation in multimedia recommender systems: A practical guide. In Proceedings of the 12th IEEE International Conference on Semantic Computing (ICSC’18). 294--297.
[65]
Xue Geng, Hanwang Zhang, Jingwen Bian, and Tat-Seng Chua. 2015. Learning image and user features for recommendation in social networks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’15). 4274--4282.
[66]
Jennifer Golbeck and Eric Norris. 2013. Personality, movie preferences, and recommendations. In Proceedings of the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining. ACM, New York, NY, 1414--1415.
[67]
Samuel D. Gosling, Peter J. Rentfrow, and William B. Swann Jr. 2003. A very brief measure of the big-five personality domains. J. Res. Personal. 37, 6 (2003), 504--528.
[68]
Xiaoling Gu, Lidan Shou, Pai Peng, Ke Chen, Sai Wu, and Gang Chen. 2016. iGlasses: A novel recommendation system for best-fit glasses. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 1109--1112.
[69]
Sharath Chandra Guntuku, Lin Qiu, Sujoy Roy, Weisi Lin, and Vinit Jakhetiya. 2015a. Do others perceive you as you want them to?: Modeling personality based on selfies. In Proceedings of the 1st International Workshop on Affect 8 Sentiment in Multimedia. ACM, 21--26.
[70]
Sharath Chandra Guntuku, Sujoy Roy, and Lin Weisi. 2015b. Personality modeling based image recommendation. In Proceedings of the International Conference on Multimedia Modeling. Springer, 171--182.
[71]
Sharath Chandra Guntuku, Michael James Scott, Gheorghita Ghinea, and Weisi Lin. 2016. Personality, culture, and system factors-impact on affective response to multimedia. arXiv preprint arXiv:1606.06873 (2016).
[72]
Mario Haim, Andreas Graefe, and Hans-Bernd Brosius. 2018. Burst of the filter bubble? Effects of personalization on the diversity of Google News. Dig. J. 6, 3 (2018), 330--343.
[73]
Ruining He and Julian McAuley. 2016a. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the WWW Conference. 507--517.
[74]
Ruining He and Julian McAuley. 2016b. VBPR: Visual Bayesian personalized ranking from implicit feedback. In Proceedings of the 30th AAAI Conference on Artificial Intelligence. 144--150.
[75]
Luis Herranz, Weiqing Min, and Shuqiang Jiang. 2018. Food recognition and recipe analysis: Integrating visual content, context, and external knowledge. arXiv preprint arXiv:1801.07239 (2018).
[76]
Prajakta A. Holey and S. S. Prabhune. 2014. Review of content-based recommendation system. Int. J. Sci. Eng. Technol. Res. 3, 4 (2014).
[77]
Min Hou, Le Wu, Enhong Chen, Zhi Li, Vincent W. Zheng, and Qi Liu. 2019. Explainable fashion recommendation: A semantic attribute region guided approach. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI’19). 4681--4688.
[78]
Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative filtering for implicit feedback datasets. In Proceedings of the 8th IEEE International Conference on Data Mining (ICDM’08). 263--272.
[79]
Mark J. Huiskes and Michael S. Lew. 2008. The MIR flickr retrieval evaluation. In Proceedings of the 1st ACM SIGMM International Conference on Multimedia Information Retrieval (MIR’08). 39--43.
[80]
Oliver John and Sanjay Srivastava. 1999. The big five trait taxonomy: History, measurement, and theoretical perspectives. In Handbook of Personality: Theory and Research (2nd ed.), Lawrence A. Pervin and Oliver P. John (Eds.). Guilford Press, New York, 102--138.
[81]
Oliver P. John, Eileen M. Donahue, and Robert L. Kentle. 1991. The big five inventory. Journal of Personality and Social Psychology (1991).
[82]
Marius Kaminskas and Derek Bridge. 2017. Diversity, serendipity, novelty, and coverage: A survey and empirical analysis of beyond-accuracy objectives in recommender systems. Trans. Internet Inf. Syst. 7, 1 (2017), 2:1--2:42.
[83]
Marius Kaminskas, Francesco Ricci, and Markus Schedl. 2013. Location-aware music recommendation using auto-tagging and hybrid matching. In Proceedings of the ACM Conference on Recommender Systems. ACM, 17--24.
[84]
Peter Knees and Markus Schedl. 2013. A survey of music similarity and recommendation from music context data. ACM Trans. Multimedia Comput. Commun. Applic. 10, 1 (2013).
[85]
Yehuda Koren and Robert Bell. 2015. Advances in collaborative filtering. In Recommender Systems Handbook. Springer, 77--118.
[86]
Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (Aug. 2009), 30--37.
[87]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 1097--1105.
[88]
Xingchen Li, Xiang Wang, Xiangnan He, Long Chen, Jun Xiao, and Tat-Seng Chua. 2020. Hierarchical fashion graph network for personalized outfit recommendation. CoRR abs/2005.12566 (2020).
[89]
Dawen Liang, Minshu Zhan, and Daniel P. W. Ellis. 2015. Content-aware collaborative music recommendation using pre-trained neural networks. In Proceedings of the 16th International Society for Music Information Retrieval Conference (ISMIR’15). 295--301.
[90]
Jimmy Lin. 2019. The neural hype and comparisons against weak baselines. SIGIR Forum 52, 2 (Jan. 2019), 40--51.
[91]
Q. Lin, Y. Niu, Y. Zhu, H. Lu, K. Z. Mushonga, and Z. Niu. 2018. Heterogeneous knowledge-based attentive neural networks for short-term music recommendations. IEEE Access 6 (2018), 58990--59000.
[92]
Yusan Lin, Maryam Moosaei, and Hao Yang. 2020. OutfitNet: Fashion outfit recommendation with attention-based multiple instance learning. In Proceedings of the WWW Conference (WWW’20). ACM/IW3C2, 77--87.
[93]
Zijia Lin, Guiguang Ding, and Jianmin Wang. 2011. Image annotation based on recommendation model. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 1097--1098.
[94]
Jing Liu, Zechao Li, Jinhui Tang, Yu Jiang, and Hanqing Lu. 2014. Personalized geo-specific tag recommendation for photos on social websites. IEEE Trans. Multimedia 16, 3 (2014), 588--600.
[95]
Beth Logan. 2000. Mel frequency cepstral coefficients for music modeling. In Proceedings of the International Symposium on Music Information Retrieval (ISMIR’00).
[96]
David G. Lowe. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 2 (2004), 91--110.
[97]
Hangzai Luo, Jianping Fan, Daniel A. Keim, and Shin’ichi Satoh. 2009. Personalized news video recommendation. In Proceedings of the International Conference on Multimedia Modeling. Springer, 459--471.
[98]
Jingwei Ma, Guang Li, Mingyang Zhong, Xin Zhao, Lei Zhu, and Xue Li. 2018. LGA: Latent genre aware micro-video recommendation on social media. Multimedia Tools Applic. 77, 3 (2018), 2991--3008.
[99]
Anand Mahadevan, Jason Freeman, Brian Magerko, and Juan Carlos Martinez. 2015. EarSketch: Teaching computational music remixing in an online web audio--based learning environment. In Proceedings of the Web Audio Conference.
[100]
Bangalore S. Manjunath and Wei-Ying Ma. 1996. Texture features for browsing and retrieval of image data. IEEE Trans. Pattern Anal. Mach. Intell. 18, 8 (1996), 837--842.
[101]
Richard E. Mayer. 2005. The Cambridge Handbook of Multimedia Learning, 1st Edition. Cambridge University Press. Retrieved from http://www.worldcat.org/oclc/57526976.
[102]
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015. Image-based recommendations on styles and substitutes. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 43--52.
[103]
Brian McFee, Luke Barrington, and Gert Lanckriet. 2012. Learning content similarity for music recommendation. IEEE Trans. Aud. Speech, Lang. Proc. 20, 8 (2012), 2207--2218.
[104]
Brian McFee and Gert Lanckriet. 2011. The natural language of playlists. In Proceedings of the 12th International Society for Music Information Retrieval Conference (ISMIR’11).
[105]
Tao Mei, Bo Yang, Xian-Sheng Hua, and Shipeng Li. 2011. Contextual video recommendation by multimodal relevance and user feedback. ACM Trans. Inf. Syst. 29, 2 (2011), 10.
[106]
Tao Mei, Bo Yang, Xian-Sheng Hua, Linjun Yang, Shi-Qiang Yang, and Shipeng Li. 2007. VideoReach: An online video recommendation system. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 767--768.
[107]
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. In Proceedings of the 1st International Conference on Learning Representations (ICLR’13).
[108]
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013b. Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS’13), Vol. 2. Curran Associates Inc., 3111--3119.
[109]
Naila Murray, Luca Marchesotti, and Florent Perronnin. 2012. AVA: A large-scale database for aesthetic visual analysis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). IEEE, 2408--2415.
[110]
Radford M. Neal. 2000. Markov chain sampling methods for Dirichlet process mixture models. J. Comput. Graph. Stat. 9, 2 (2000), 249--265.
[111]
Robert Neumayer and Andreas Rauber. 2007. Integration of text and audio features for genre classification in music information retrieval. In Proceedings of the European Conference on Information Retrieval. Springer, 724--727.
[112]
Vinh-Tiep Nguyen, Khanh-Duy Le, Minh-Triet Tran, and Morten Fjeld. 2016. NowAndThen: A social network-based photo recommendation tool supporting reminiscence. In Proceedings of the 15th International Conference on Mobile and Ubiquitous Multimedia. ACM, 159--168.
[113]
Wei Niu, James Caverlee, and Haokai Lu. 2018. Neural personalized ranking for image recommendation. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining. ACM, 423--431.
[114]
Timo Ojala, Matti Pietikainen, and Topi Maenpaa. 2002. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24, 7 (2002), 971--987.
[115]
Sergio Oramas, Oriol Nieto, Mohamed Sordo, and Xavier Serra. 2017. A deep multimodal approach for cold-start music recommendation. In Proceedings of the 2nd Workshop on Deep Learning for Recommender Systems (DLRS’17). ACM, New York, NY, 32--37.
[116]
Sergio Oramas, Vito Claudio Ostuni, Tommaso Di Noia, Xavier Serra, and Eugenio Di Sciascio. 2016. Sound and music recommendation with knowledge graphs. ACM Trans. Intell. Syst. Technol. 8, 2 (Oct. 2016).
[117]
Claudia Orellana-Rodriguez, Ernesto Diaz-Aviles, and Wolfgang Nejdl. 2015. Mining affective context in short films for emotion-aware recommendation. In Proceedings of the 26th ACM Conference on Hypertext 8 Social Media (HT’15). ACM, New York, NY, 185--194.
[118]
Vito Claudio Ostuni, Tommaso Di Noia, Eugenio Di Sciascio, Sergio Oramas, and Xavier Serra. 2015. A semantic hybrid approach for sound recommendation. In Proceedings of the WWW Conference. 85--86.
[119]
Deuk Hee Park, Hyea Kyeong Kim, Il Young Choi, and Jae Kyeong Kim. 2012. A literature review and classification of recommender systems research. Exp. Syst. Applic. 39, 11 (2012), 10059--10072.
[120]
Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14). 1532--1543.
[121]
Florent Perronnin and Christopher Dance. 2007. Fisher kernels on visual vocabularies for image categorization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’07). IEEE, 1--8.
[122]
Ladislav Peska and Hana Trojanova. 2017. Towards recommender systems for police photo lineup. arXiv preprint arXiv:1707.01389 (2017).
[123]
Gabriele Prato, Federico Sallemi, Paolo Cremonesi, Mario Scriminaci, Stefan Gudmundsson, and Silvio Palumbo. 2020. Outfit completion and clothes recommendation. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI’20). ACM, 1--7.
[124]
Pearl Pu and Li Chen. 2007. Trust-inspiring explanation interfaces for recommender systems. Knowl.-based Syst. 20, 6 (2007), 542--556.
[125]
Xueming Qian, He Feng, Guoshuai Zhao, and Tao Mei. 2014. Personalized recommendation combining user interest and social circle. IEEE Trans. Knowl. Data Eng. 26, 7 (2014), 1763--1777.
[126]
Massimo Quadrana, Paolo Cremonesi, and Dietmar Jannach. 2018. Sequence-aware recommender systems. Comput. Surv. 51, 4 (July 2018).
[127]
Amir Hossein Nabizadeh Rafsanjani, Naomie Salim, Atae Rezaei Aghdam, and Karamollah Bagheri Fard. 2013. Recommendation systems: A review. Int. J. Comput. Eng. Res. 3, 5 (2013), 47--52.
[128]
Yogesh Singh Rawat and Mohan S. Kankanhalli. 2017. ClickSmart: A context-aware viewpoint recommendation system for mobile photography. IEEE Trans. Circ. Syst. Video Technol. 27, 1 (2017), 149--158.
[129]
Peter Rentfrow, Lewis R. Goldberg, and Ran Zilca. 2011. Listening, watching, and reading: The structure and correlates of entertainment preferences. J. Personal. 79 (Apr. 2011), 223--258.
[130]
Sujoy Roy and Sharat Chandra Guntuku. 2016. Latent factor representations for cold-start video recommendation. In Proceedings of the ACM Conference on Recommender Systems. ACM, 99--106.
[131]
Noveen Sachdeva, Kartik Gupta, and Vikram Pudi. 2018. Attentive neural architecture incorporating song features for music recommendation. In Proceedings of the ACM Conference on Recommender Systems (RecSys’18). ACM, New York, NY, 417--421.
[132]
Markus Schedl. 2019. Deep learning in music recommender systems. Frontiers in Applied Mathematics and Statistics 5 (2019), 44 pages.
[133]
Markus Schedl, Arthur Flexer, and Julián Urbano. 2013. The neglected user in music information retrieval research. J. Intell. Inf. Syst. 41, 3 (Dec. 2013), 523--539.
[134]
Markus Schedl, Hamed Zamani, Ching-Wei Chen, Yashar Deldjoo, and Mehdi Elahi. 2018. Current challenges and visions in music recommender systems research. Int. J. Multimedia Inf. Retr. 7, 2 (2018), 95--116.
[135]
Jan Schlüter. 2016. Learning to pinpoint singing voice from weakly labeled examples. In Proceedings of the 17th International Society for Music Information Retrieval Conference (ISMIR’16).
[136]
Klaus Seyerlehner, Gerhard Widmer, and Tim Pohle. 2010. Fusing block-level features for music similarity estimation. In Proceedings of the 13th International Conference on Digital Audio Effects (DAFx’10).
[137]
Dandan Sha, Daling Wang, Xiangmin Zhou, Shi Feng, Yifei Zhang, and Ge Yu. 2016. An approach for clothing recommendation based on multiple image attributes. In Proceedings of the International Conference on Web-age Information Management. Springer, 272--285.
[138]
Rajiv Ratn Shah, Yi Yu, and Roger Zimmermann. 2014. ADVISOR: Personalized video soundtrack recommendation by late fusion with heuristic rankings. In Proceedings of the ACM International Conference on Multimedia (MM’14). 607--616.
[139]
Guy Shani and Asela Gunawardana. 2011. Evaluating recommendation systems. Recommender Systems Handbook. Springer, 257--297.
[140]
Bo Shao, Dingding Wang, Tao Li, and Mitsunori Ogihara. 2009. Music recommendation based on acoustic features and user access patterns. IEEE Trans. Aud. Speech, Lang. Proc. 17, 8 (2009), 1602--1611.
[141]
Yue Shi, Alexandros Karatzoglou, Linas Baltrunas, Martha Larson, Nuria Oliver, and Alan Hanjalic. 2012. CLiMF: Learning to maximize reciprocal rank with collaborative less-is-more filtering. In Proceedings of the ACM Conference on Recommender Systems. ACM, 139--146.
[142]
Yue Shi, Martha Larson, and Alan Hanjalic. 2014. Collaborative filtering beyond the user-item matrix: A survey of the state of the art and future challenges. ACM Comput. Surv. 47, 1 (2014), 3.
[143]
Marcin Skowron, Bruce Ferwerda, Marko Tkalčič, and Markus Schedl. 2016. Fusing social media cues: Personality prediction from Twitter and Instagram. In Proceedings of the WWW Conference.
[144]
Jason Smith, Dillon Weeks, Mikhail Jacob, Jason Freeman, and Brian Magerko. 2019. Towards a hybrid recommendation system for a sound library. In Joint Proceedings of the ACM IUI Workshops co-located with the 24th ACM Conference on Intelligent User Interfaces (ACM-IUI’19).
[145]
Adrian Stanciulescu. 2008. A Methodology for Developing Multimodal User Interfaces of Information Systems. Ph.D. Dissertation. Catholic University of Louvain, Louvain-la-Neuve, Belgium. Retrieved from http://hdl.handle.net/2078.1/12738.
[146]
Guang-Lu Sun, Zhi-Qi Cheng, Xiao Wu, and Qiang Peng. 2018. Personalized clothing recommendation combining user social circle and fashion style consistency. Multimedia Tools Applic. 77, 14 (2018), 17731--17754.
[147]
Jinhui Tang, Xiaoyu Du, Xiangnan He, Fajie Yuan, Qi Tian, and Tat-Seng Chua. 2020. Adversarial training towards robust multimedia recommender system. IEEE Trans. Knowl. Data Eng. 32, 5 (2020), 855--867.
[148]
Marko Tkalcic and Li Chen. 2015. Personality and Recommender Systems. Springer US, Boston, MA, 715--739.
[149]
Christoph Trattner, Dominik Moesslang, and David Elsweiler. 2018. On the predictability of the popularity of online recipes. EPJ Data Sci. 7, 1 (2018), 20.
[150]
Aäron van den Oord, Sander Dieleman, and Benjamin Schrauwen. 2013. Deep content-based music recommendation. In Proceedings of the International Conference on Advances in Neural Information Processing Systems (NIPS’13), Christopher Burges, Léon Bottou, Max Welling, Zoubin Ghahramani, and Kilian Weinberger (Eds.). Curran Associates, Inc., 2643--2651.
[151]
Andreu Vall, Matthias Dorfer, Hamid Eghbal-zadeh, Markus Schedl, Keki Burjorjee, and Gerhard Widmer. 2019. Feature-combination hybrid recommender systems for automated music playlist continuation. User Model. User-Adapt. Interact. 29, 2 (2019), 527--572.
[152]
Shangfei Wang and Qiang Ji. 2015. Video affective content analysis: A survey of state-of-the-art methods. IEEE Trans. Affect. Comput. 6, 4 (2015), 410--430.
[153]
Suhang Wang, Yilin Wang, Jiliang Tang, Kai Shu, Suhas Ranganath, and Huan Liu. 2017. What your images reveal: Exploiting visual contents for point-of-interest recommendation. In Proceedings of the WWW Conference. 391--400.
[154]
Xinxi Wang and Ye Wang. 2014. Improving content-based and hybrid music recommendation using deep learning. In Proceedings of the 22nd ACM International Conference on Multimedia. ACM, 627--636.
[155]
Zhangyang Wang, Shiyu Chang, Florin Dolcos, Diane Beck, Ding Liu, and Thomas S. Huang. 2016. Brain-inspired deep networks for image aesthetics assessment. arXiv preprint arXiv:1601.04155 (2016).
[156]
Kangning Wei, Jinghua Huang, and Shaohong Fu. 2007. A survey of e-commerce recommender systems. In Proceedings of the International Conference on Service Systems and Service Management. IEEE, 1--5.
[157]
Jiqing Wen, James She, Xiaopeng Li, and Hui Mao. 2018. Visual background recommendation for dance performances using deep matrix factorization. ACM Trans. Multimedia Comput. Commun. Applic. 14, 1 (2018), 11:1--11:19.
[158]
Stina Westman and Pirkko Oittinen. 2006. Image retrieval by end-users and intermediaries in a journalistic work context. In Proceedings of the 1st International Conference on Information Interaction in Context. ACM, 102--110.
[159]
Chun-Che Wu, Tao Mei, Winston H. Hsu, and Yong Rui. 2014. Learning to personalize trending image search suggestion. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 727--736.
[160]
Feng Xia, Nana Yaw Asabere, Ahmedin Mohammed Ahmed, Jing Li, and Xiangjie Kong. 2013. Mobile multimedia recommendation in smart communities: A survey. IEEE Access 1 (2013), 606--624.
[161]
Zhou Xing, Marzieh Parandehgheibi, Fei Xiao, Nilesh Kulkarni, and Chris Pouliot. 2016. Content-based recommendation for podcast audio-items using natural language processing techniques. In Proceedings of the IEEE International Conference on Big Data (Big Data’16). IEEE, 2378--2383.
[162]
Bo Yang, Tao Mei, Xian-Sheng Hua, Linjun Yang, Shi-Qiang Yang, and Mingjing Li. 2007. Online video recommendation based on multimodal fusion and relevance feedback. In Proceedings of the 6th ACM International Conference on Image and Video Retrieval. ACM, 73--80.
[163]
Longqi Yang, Cheng-Kang Hsieh, Hongjian Yang, John P. Pollak, Nicola Dell, Serge Belongie, Curtis Cole, and Deborah Estrin. 2017. Yum-me: A personalized nutrient-based meal recommender system. ACM Trans. Inf. Syst. 36, 1 (2017), 7.
[164]
Longqi Yang, Michael Sobolev, Christina Tsangouri, and Deborah Estrin. 2018. Understanding user interactions with podcast recommendations delivered via voice. In Proceedings of the ACM Conference on Recommender Systems. ACM, 190--194.
[165]
Kazuyoshi Yoshii, Masataka Goto, Kazunori Komatani, Tetsuya Ogata, and Hiroshi G. Okuno. 2007. Improving efficiency and scalability of model-based music recommender system based on incremental training. In Proceedings of the International Conference on Music Information Retrieval (ISMIR’07).
[166]
Kazuyoshi Yoshii, Masataka Goto, Kazuhiro Komatani, Tetsuya Ogata, and Hiroshi G. Okuno. 2008. An efficient hybrid music recommender system using an incrementally trainable probabilistic generative model. IEEE Trans. Aud. Speech Lang. Proc. 16, 2 (2008), 435--447.
[167]
Dongfei Yu, Xinmei Tian, Tao Mei, and Yong Rui. 2015. On the selection of trending image from the web. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME’15). IEEE, 1--6.
[168]
Wenhui Yu, Huidi Zhang, Xiangnan He, Xu Chen, Li Xiong, and Zheng Qin. 2018. Aesthetic-based clothing recommendation. In Proceedings of the WWW Conference. 649--658.
[169]
Jianbo Yuan, Walid Shalaby, Mohammed Korayem, David Lin, Khalifeh AlJadda, and Jiebo Luo. 2016. Solving cold-start problem in large-scale recommendation engines: A deep learning approach. In Proceedings of the IEEE International Conference on Big Data (Big Data’16). IEEE, 1901--1910.
[170]
Hamed Zamani, Markus Schedl, Paul Lamere, and Ching-Wei Chen. 2018. An analysis of approaches taken in the ACM recsys challenge 2018 for automatic music playlist continuation. CoRR arXiv:1810.01520 (2018).
[171]
Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019. Deep learning based recommender system: A survey and new perspectives. ACM Comput. Surv. 52, 1 (2019), 1--38.
[172]
Yongfeng Zhang and Xu Chen. 2020. Explainable recommendation: A survey and new perspectives. Found. Trends Inf. Retr. 14, 1 (2020), 1--101. Retrieved from
[173]
Xiaojian Zhao, Huanbo Luan, Junjie Cai, Jin Yuan, Xiaoming Chen, and Zhoujun Li. 2012a. Personalized video recommendation based on viewing history with the study on YouTube. In Proceedings of the 4th International Conference on Internet Multimedia Computing and Service. ACM, 161--165.
[174]
Xiaojian Zhao, Jin Yuan, Richang Hong, Meng Wang, Zhoujun Li, and Tat-Seng Chua. 2012b. On video recommendation over social network. In Proceedings of the International Conference on Multimedia Modeling. Springer, 149--160.
[175]
Tao Zhou, Zoltán Kuscsik, Jian-Guo Liu, Matúš Medo, Joseph Rushton Wakeling, and Yi-Cheng Zhang. 2010. Solving the apparent diversity-accuracy dilemma of recommender systems. Proc. Nat. Acad. Sci. 107, 10 (2010), 4511--4515.
[176]
Qiusha Zhu, Mei-Ling Shyu, and Haohong Wang. 2013. Videotopic: Content-based video recommendation using a topic model. In Proceedings of the IEEE International Symposium on Multimedia (ISM’13). IEEE, 219--222.

Cited By

View all
  • (2024)Recommender System: A Comprehensive Overview of Technical Challenges and Social ImplicationsIECE Transactions on Sensing, Communication, and Control10.62762/TSCC.2024.8985031:1(30-51)Online publication date: 15-Oct-2024
  • (2024)U-Net-based Recommender Systems for Political Election System using Collaborative Filtering AlgorithmsJournal of information and communication convergence engineering10.56977/jicce.2024.22.1.722:1(7-13)Online publication date: 31-Mar-2024
  • (2024)Narrative Threads and Cinematic Connections Using Intelligent Systems to Enhance Movie Recommendations with Market Basket Analysis and Advanced AlgorithmsData-Driven Business Intelligence Systems for Socio-Technical Organizations10.4018/979-8-3693-1210-0.ch013(319-364)Online publication date: 23-Feb-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Computing Surveys
ACM Computing Surveys  Volume 53, Issue 5
September 2021
782 pages
ISSN:0360-0300
EISSN:1557-7341
DOI:10.1145/3426973
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 September 2020
Accepted: 01 June 2020
Revised: 01 June 2020
Received: 01 September 2019
Published in CSUR Volume 53, Issue 5

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Content-based recommender systems
  2. audio
  3. deep learning
  4. e-commerce
  5. fashion
  6. food
  7. image
  8. machine learning
  9. multimedia
  10. music
  11. signal processing
  12. social media
  13. tourism
  14. video

Qualifiers

  • Survey
  • Research
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)866
  • Downloads (Last 6 weeks)85
Reflects downloads up to 18 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Recommender System: A Comprehensive Overview of Technical Challenges and Social ImplicationsIECE Transactions on Sensing, Communication, and Control10.62762/TSCC.2024.8985031:1(30-51)Online publication date: 15-Oct-2024
  • (2024)U-Net-based Recommender Systems for Political Election System using Collaborative Filtering AlgorithmsJournal of information and communication convergence engineering10.56977/jicce.2024.22.1.722:1(7-13)Online publication date: 31-Mar-2024
  • (2024)Narrative Threads and Cinematic Connections Using Intelligent Systems to Enhance Movie Recommendations with Market Basket Analysis and Advanced AlgorithmsData-Driven Business Intelligence Systems for Socio-Technical Organizations10.4018/979-8-3693-1210-0.ch013(319-364)Online publication date: 23-Feb-2024
  • (2024)Perspectives of Digital Marketing for the Restaurant IndustryAdvancements in Socialized and Digital Media Communications10.4018/979-8-3693-0855-4.ch009(118-134)Online publication date: 26-Jan-2024
  • (2024)Accurate and efficient floor localization with scalable spiking graph neural networksSatellite Navigation10.1186/s43020-024-00127-85:1Online publication date: 11-Mar-2024
  • (2024)Enhanced content-based fashion recommendation system through deep ensemble classifier with transfer learningFashion and Textiles10.1186/s40691-024-00382-y11:1Online publication date: 1-Jul-2024
  • (2024)Multimodal Recommender Systems: A SurveyACM Computing Surveys10.1145/3695461Online publication date: 10-Sep-2024
  • (2024)Multimodal-aware Multi-intention Learning for RecommendationProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681412(5663-5672)Online publication date: 28-Oct-2024
  • (2024)A Survey on Variational Autoencoders in Recommender SystemsACM Computing Surveys10.1145/366336456:10(1-40)Online publication date: 24-Jun-2024
  • (2024)DHyper: A Recurrent Dual Hypergraph Neural Network for Event Prediction in Temporal Knowledge GraphsACM Transactions on Information Systems10.1145/365301542:5(1-23)Online publication date: 29-Apr-2024
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media