research-article

An emotion-aware music recommender system: bridging the user’s interaction and music recommendation

Authors:

Saba Yousefian Jazi,

Afsaneh FatemiAuthors Info & Claims

Multimedia Tools and Applications, Volume 80, Issue 9

Pages 13559 - 13574

https://doi.org/10.1007/s11042-020-10386-7

Published: 01 April 2021 Publication History

Abstract

In emotion-aware music recommender systems, the user’s current emotion is identified and considered in recommending music to him. We have two motivations to extend the existing systems: (1) to the best of our knowledge, the current systems first estimate the user’s emotions and then suggest music based on it. Therefore, the emotion estimation error affects the recommendation accuracy. (2) Studies show that the pattern of users’ interactions with input devices can reflect their emotions. However, these patterns have not been used yet in emotion-aware music recommender systems. In this study, a music recommender system is proposed to suggest music based on users’ keystrokes and mouse clicks patterns. Unlike the previous ones, the proposed system maps these patterns directly to the user’s favorite music, without labeling its current emotion. The results show that even though this system does not use any additional device, it is highly accurate compared to previous methods.

References

[1]

Abdul A, Chen J, Liao H-Y, and Chang S-H An emotion-aware personalized music recommendation system using a convolutional neural networks approach Appl Sci 2018 8 1103

[2]

Aljanaki A, Wiering F, and Veltkamp RC Studying emotion induced by music through a crowdsourcing game Inform Process Manag 2016 52 1 115-128

[3]

Aracena C, Basterrech S, Snáel V, Velásquez J (2015) Neural networks for emotion recognition based on eye tracking data. Paper presented at the 2015 IEEE International Conference on Systems, Man, and Cybernetics

[4]

Ayata D, Yaslan Y, and Kamasak ME Emotion based music recommendation system using wearable physiological sensors IEEE Trans Consumer Electron 2018 64 2 196-203

[5]

Burns H, Burns S (2015) Moving Averages 101: Incredible Signals That Will Make You Money in the Stock Market: CreateSpace Independent Publishing Platform

[6]

Burns S, Burns H (2017) 5 Moving Average Signals That Beat Buy and Hold: Backtested Stock Market Signals: CreateSpace Independent Publishing Platform

[7]

Deng S, Wang D, Li X, and Xu G Exploring user emotion in microblogs for music recommendation Expert Syst Appl 2015 42 23 9284-9293

[8]

Estrada J, Buhia J, Guevarra A, Forcado MR (2017) Keyboard and mouse: tools in identifying emotions during computer activities. Paper presented at the International Conference on Big Data Technologies and Applications

[9]

Gavrilescu M (2015) Recognizing emotions from videos by studying facial expressions, body postures and hand gestures. Paper presented at the 2015 23rd Telecommunications Forum Telfor (TELFOR)

[10]

Geng B, Li L, Jiao L, Gong M, Cai Q, and Wu Y NNIA-RS: A multi-objective optimization based recommender system Physica A: Statistic Mech Appl 2015 424 383-397

[11]

Goldberg D, Nichols D, Oki BM, Terry D (1992) Using collaborative filtering to weave an information tapestry. Commun ACM 35:61–70

[12]

Gu YH, Yoo SJ, Piao Z, No J, Jiang Z, Yin H (2016) A smart-device news recommendation technology based on the user click behavior. Paper presented at the Sixth International Conference on Emerging Databases: Technologies, Applications, and Theory

[13]

Hamedani EM, Kaedi M (2019) Recommending the long tail items through personalized diversification. Knowledge-Based Syst 164:348–357

[14]

Hu X, Bai K, Cheng J, Deng J-q, Guo Y, Hu B, . . . Wang F (2017) MeDJ: multidimensional emotion-aware music delivery for adolescent. Paper presented at the Proceedings of the 26th International Conference on World Wide Web Companion

[15]

Jannach D, Zanker M, Felfernig A, Friedrich G (2011) Recommender systems, an introduction. Hardback (November 2010)

[16]

Kabani H, Khan S, Khan O, and Tadvi S Emotion based music player Int J Eng Res Gen Sci 2015 3 1 2091-2730

[17]

Kawakami A, Furukawa K, Katahira K, Kamiyama K, and Okanoya K Relations between musical structures and perceived and felt emotions Music Perception: Interdisciplin J 2012 30 4 407-417

[18]

Kawakami A, Furukawa K, Katahira K, and Okanoya K Sad music induces pleasant emotion Frontiers Psychol 2013 4 311

[19]

Khan IA, Brinkman WP, and Hierons R Towards estimating computer users’ mood from interaction behaviour with keyboard and mouse Front Comput Sci 2013 7 6 943-954

[20]

Khanna P and Sasikumar M Recognising emotions from keyboard stroke pattern Int J Comput Appl 2010 11 9 1-5

[21]

KM AK, Kiran B, Shreyas B, and Victor SJ A multimodal approach to detect user's emotion Procedia Comput Sci 2015 70 296-303

[22]

Lukose S, Upadhya SS (2017) Music player based on emotion recognition of voice signals. Paper presented at the 2017 International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT)

[23]

Nahin ANH, Alam JM, Mahmud H, and Hasan K Identifying emotion by keystroke dynamics and text pattern analysis Behav Information Technol 2014 33 9 987-996

[24]

Pentel A (2017) Emotions and user interactions with Keyboard and Mouse. Paper presented at the 2017 8th International Conference on Information, Intelligence, Systems & Applications (IISA)

[25]

Pichl M (2018) Multi-Context-Aware Recommender Systems: A Study on Music Rfecommendation. University of Innsbruck

[26]

Pichl M, Zangerle E (2018) Latent feature combination for multi-context music recommendation. Paper presented at the 2018 International Conference on Content-Based Multimedia Indexing (CBMI)

[27]

Pichl M, Zangerle E (2020) User models for multi-context-aware music recommendation. Multimed Tools Appl.

[28]

Poirson E, Da Cunha C (2018) A recommender approach based on customer emotions. Expert Syst Appl

[29]

Robinson J and Hatten RS Emotions in music Music Theory Spectrum 2012 34 2 71-106

[30]

Roy S, Biswas M, and De D iMusic: A session-sensitive clustered classical music recommender system using contextual representation learning Multimed Tools Appl 2020 79 24119-24155

[31]

Salmeron-Majadas S, Santos OC, and Boticario JG An evaluation of mouse and keyboard interaction indicators towards non-intrusive and low cost affective modeling in an educational context Procedia Comput Sci 2014 35 691-700

[32]

Sarwar BM, Karypis G, Konstan JA, and Riedl J Item-based collaborative filtering recommendation algorithms Www 2001 1 285-295

[33]

Schedl M, Zamani H, Chen C-W, Deldjoo Y, and Elahi M Current challenges and visions in music recommender systems research Int J Multimed Information Retriev 2018 7 2 95-116

[34]

Shakirova E (2017) Collaborative filtering for music recommender system. Paper presented at the 2017 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus)

[35]

Shikder R, Rahaman S, Afroze F, Al Islam AA (2017) Keystroke/mouse usage based emotion detection and user identification. Paper presented at the 2017 International Conference on Networking, Systems and Security (NSysS)

[36]

Song K-T, Cervantes C (2017) Music playing system and music playing method based on speech emotion recognition. In: Google Patents

[37]

Sulikowski P, Zdziebko T, Turzyński D, and Kańtoch E Human-website interaction monitoring in recommender systems Procedia Comput Sci 2018 126 1587-1596

[38]

Sunitha M, Adilakshmi T (2018) Music recommendation system with user-based and item-based collaborative filtering technique. In Networking Communication and Data Knowledge Engineering (pp 267–278): Springer

[39]

Tso K, Schmidt-Thieme L (2006) Attribute-aware collaborative filtering. In From data and information analysis to knowledge engineering (pp 614–621): Springer

[40]

Xing B, Zhang K, Sun S, Zhang L, Gao Z, Wang J, and Chen S Emotion-driven Chinese folk music-image retrieval based on DE-SVM Neurocomput 2015 148 619-627

[41]

Yusefi Hafshejani Z, Kaedi M, Fatemi A (2018) Improving sparsity and new user problems in collaborative filtering by clustering the personality factors. Electron Commerce Res 18(4):813–836

[42]

Zakamulin V (2017) Market Timing with Moving Averages: The Anatomy and Performance of Trading Rules: Springer

[43]

Zentner M, Grandjean D, and Scherer KR Emotions evoked by the sound of music: characterization, classification, and measurement Emotion 2008 8 4 494

[44]

Zhang R, Liu Q-d, Chun-Gui, Wei J-X, and Huiyi-Ma Collaborative filtering for recommender systems 2014 Huangshan Paper presented at the Second International Conference on Advanced Cloud and Big Data (CBD)

[45]

Zhou C, Jin Y, Zhang K, Li JYS, and Wang X MusicRoBot: Towards Conversational Context-Aware Music Recommender System Paper presented at the International Conference on Database Systems for Advanced Applications 2018 Australia Gold Coast

Cited By

Tanwar AVishwakarma D(2023)A deep neural network-based hybrid recommender system with user-user networksMultimedia Tools and Applications10.1007/s11042-022-13936-382:10(15613-15633)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s11042-022-13936-3
Xu JGan MZhang X(2023)MMusic: a hierarchical multi-information fusion method for deep music recommendationJournal of Intelligent Information Systems10.1007/s10844-023-00786-061:3(795-818)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/s10844-023-00786-0
Yun HDeng XDu Y(2022)News Recommendation with Multi-views Emotion AnalysisProceedings of the 4th International Conference on Advanced Information Science and System10.1145/3573834.3574478(1-5)Online publication date: 25-Nov-2022
https://dl.acm.org/doi/10.1145/3573834.3574478

Index Terms

An emotion-aware music recommender system: bridging the user’s interaction and music recommendation

Index terms have been assigned to the content through auto-classification.

Recommendations

A probabilistic music recommender considering user opinions and audio features
Special issue: AIRS2005: Information retrieval research in Asia

A recommender system has an obvious appeal in an environment where the amount of on-line information vastly outstrips any individual's capability to survey. Music recommendation is considered a popular application area. In order to make personalized ...
A novel music recommender by discovering preferable perceptual-patterns from music pieces
SAC '10: Proceedings of the 2010 ACM Symposium on Applied Computing

Nowadays, advanced information and communication technologies ease the access of music pieces. However, it is still hard for the users to find what she/he prefers from a huge amount of music works. To solve this problem, most music recommenders based on ...
A music recommender based on audio features
SIGIR '04: Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval

Many collaborative music recommender systems (CMRS) have succeeded in capturing the similarity among users or items based on ratings, however they have rarely considered about the available information from the multimedia such as genres, let alone audio ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Multimedia Tools and Applications

Multimedia Tools and Applications Volume 80, Issue 9

Apr 2021

1581 pages

ISSN:1380-7501

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC part of Springer Nature 2021.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 April 2021

Accepted: 22 December 2020

Revision received: 20 October 2020

Received: 26 July 2019

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tanwar AVishwakarma D(2023)A deep neural network-based hybrid recommender system with user-user networksMultimedia Tools and Applications10.1007/s11042-022-13936-382:10(15613-15633)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s11042-022-13936-3
Xu JGan MZhang X(2023)MMusic: a hierarchical multi-information fusion method for deep music recommendationJournal of Intelligent Information Systems10.1007/s10844-023-00786-061:3(795-818)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/s10844-023-00786-0
Yun HDeng XDu Y(2022)News Recommendation with Multi-views Emotion AnalysisProceedings of the 4th International Conference on Advanced Information Science and System10.1145/3573834.3574478(1-5)Online publication date: 25-Nov-2022
https://dl.acm.org/doi/10.1145/3573834.3574478

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents