short-paper

Evaluating Tag Recommender Algorithms in Real-World Folksonomies: A Comparative Study

Authors:

Dominik Kowald,

Elisabeth LexAuthors Info & Claims

RecSys '15: Proceedings of the 9th ACM Conference on Recommender Systems

Pages 265 - 268

https://doi.org/10.1145/2792838.2799664

Published: 16 September 2015 Publication History

Abstract

To date, the evaluation of tag recommender algorithms has mostly been conducted in limited ways, including p-core pruned datasets, a small set of compared algorithms and solely based on recommender accuracy. In this study, we use an open-source evaluation framework to compare a rich set of state-of-the-art algorithms in six unfiltered, open datasets via various metrics, measuring not only accuracy but also the diversity, novelty and computational costs of the approaches. We therefore provide a transparent and reproducible tag recommender evaluation in real-world folksonomies. Our results suggest that the efficacy of an algorithm highly depends on the given needs and thus, they should be of interest to both researchers and developers in the field of tag-based recommender systems.

References

[1]

F. Belém, E. Martins, J. Almeida, and M. Gonçalves. Exploiting novelty and diversity in tag recommendation. In Advances in Information Retrieval, pages 380--391. Springer, 2013.

Digital Library

[2]

S. Doerfel and R. Jaschke. An analysis of tag-recommender evaluation procedures. In Proc. of RecSys'13, pages 343--346. ACM, 2013.

Digital Library

[3]

J. Gemmell, T. Schimoler, M. Ramezani, L. Christiansen, and B. Mobasher. Improving folkrank with item-based collaborative filtering. Recommender Systems & the Social Web, 2009.

[4]

D. Helic, C. Körner, M. Granitzer, M. Strohmaier, and C. Trattner. Navigational efficiency of broad vs. narrow folksonomies. In Proc. of HT'12, pages 63--72. ACM, 2012.

Digital Library

[5]

R. Jaschke, L. Marinho, A. Hotho, L. Schmidt-Thieme, and G. Stumme. Tag recommendations in folksonomies. In Knowledge Discovery in Databases: PKDD 2007, pages 506--514. Springer, 2007.

Digital Library

[6]

R. Jaschke, L. Marinho, A. Hotho, L. Schmidt-Thieme, and G. Stumme. Tag recommendations in social bookmarking systems. Ai Communications, 21(4):231--247, 2008.

Digital Library

[7]

D. Kowald, S. Kopeinik, P. Seitlinger, T. Ley, D. Albert, and C. Trattner. Refining frequency-based tag reuse predictions by means of time and semantic context. In Mining, Modeling, and Recommending 'Things' in Social Media, pages 55--74. Springer, 2015.

[8]

D. Kowald, E. Lacic, and C. Trattner. Tagrec: towards a standardized tag recommender benchmarking framework. In Proc. of HT'14, pages 305--307. ACM, 2014.

Digital Library

[9]

D. Kowald, P. Seitlinger, S. Kopeinik, T. Ley, and C. Trattner. Forgetting the words but remembering the meaning: Modeling forgetting in a verbal and semantic tag recommender. In Mining, Modeling, and Recommending 'Things' in Social Media, pages 75--95. Springer, 2015.

[10]

R. Krestel, P. Fankhauser, and W. Nejdl. Latent dirichlet allocation for tag recommendation. In Proc. of RecSys'09, pages 61--68. ACM, 2009.

Digital Library

[11]

M. Lipczak. Hybrid tag recommendation in collaborative tagging systems. PhD thesis, Dalhousie University Halifax, 2012.

[12]

L. B. Marinho and L. Schmidt-Thieme. Collaborative tag recommendations. In Data Analysis, Machine Learning and Applications, pages 533--540. Springer, 2008.

[13]

S. Rendle, L. Balby Marinho, A. Nanopoulos, and L. Schmidt-Thieme. Learning optimal ranking with tensor factorization for tag recommendation. In Proc. of KDD'09, pages 727--736. ACM, 2009.

Digital Library

[14]

S. Rendle and L. Schmidt-Thieme. Pairwise interaction tensor factorization for personalized tag recommendation. In Proc. of WSDM'10, pages 81--90. ACM, 2010.

Digital Library

[15]

L. Zhang, J. Tang, and M. Zhang. Integrating temporal usage pattern into personalized tag prediction. In Web Technologies and Applications. Springer, 2012.

Digital Library

Cited By

Kowald DReiter-Haas MKopeinik SSchedl MLex E(2024)Transparent Music Preference Modeling and Recommendation with a Model of Human Memory TheoryA Human-Centered Perspective of Intelligent Personalized Environments and Systems10.1007/978-3-031-55109-3_4(113-136)Online publication date: 1-May-2024
https://doi.org/10.1007/978-3-031-55109-3_4
Brandstetter CSuda FPapariello LDopler FGöschlberger B(2021)Exploring Learning Resource Recommendation Approaches for Secondary EducationThe 23rd International Conference on Information Integration and Web Intelligence10.1145/3487664.3487774(28-33)Online publication date: 29-Nov-2021
https://dl.acm.org/doi/10.1145/3487664.3487774
Paudel BKocev DEftimov T(2019)Mix and Rank: A Framework for Benchmarking Recommender Systems2019 IEEE International Conference on Big Data (Big Data)10.1109/BigData47090.2019.9006199(3717-3726)Online publication date: Dec-2019
https://doi.org/10.1109/BigData47090.2019.9006199
Show More Cited By

Index Terms

Evaluating Tag Recommender Algorithms in Real-World Folksonomies: A Comparative Study
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Document filtering
      2. Information extraction
  2. Information systems applications
    1. Data mining

Recommendations

A Novel Framework for Improving Recommender Diversity
International Workshop on Behavior and Social Informatics on Behavior and Social Computing - Volume 8178

Recommender systems are being used to assist users in finding relevant items from a large set of alternatives in many online applications. However, while most research up to this point has focused on improving the accuracy of recommender systems, other ...
Investigating serendipity in recommender systems based on real user feedback
SAC '18: Proceedings of the 33rd Annual ACM Symposium on Applied Computing

Over the past several years, research in recommender systems has emphasized the importance of serendipity, but there is still no consensus on the definition of this concept and whether serendipitous items should be recommended is still not a well-...
User Similarity Adjustment for Improved Recommendations
MIKE 2015: Proceedings of the Third International Conference on Mining Intelligence and Knowledge Exploration - Volume 9468

Recommender systems are becoming more and more attractive in both research and commercial communities due to Information overload problem and the popularity of the Internet applications. Collaborative Filtering, a popular branch of recommendation ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

RecSys '15: Proceedings of the 9th ACM Conference on Recommender Systems

September 2015

414 pages

ISBN:9781450336925

DOI:10.1145/2792838

General Chairs:
Hannes Werthner
TU Wien, Austria
,
Markus Zanker
Alpen-Adria-Universität Klagenfurt, Austria
,
Program Chairs:
Jennifer Golbeck
University of Maryland, USA
,
Giovanni Semeraro
University of Bari Aldo Moro, Italy

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 September 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

RecSys '15

Sponsor:

SIGCHI

RecSys '15: Ninth ACM Conference on Recommender Systems

September 16 - 20, 2015

Vienna, Austria

Acceptance Rates

RecSys '15 Paper Acceptance Rate 28 of 131 submissions, 21%;

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
310
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kowald DReiter-Haas MKopeinik SSchedl MLex E(2024)Transparent Music Preference Modeling and Recommendation with a Model of Human Memory TheoryA Human-Centered Perspective of Intelligent Personalized Environments and Systems10.1007/978-3-031-55109-3_4(113-136)Online publication date: 1-May-2024
https://doi.org/10.1007/978-3-031-55109-3_4
Brandstetter CSuda FPapariello LDopler FGöschlberger B(2021)Exploring Learning Resource Recommendation Approaches for Secondary EducationThe 23rd International Conference on Information Integration and Web Intelligence10.1145/3487664.3487774(28-33)Online publication date: 29-Nov-2021
https://dl.acm.org/doi/10.1145/3487664.3487774
Paudel BKocev DEftimov T(2019)Mix and Rank: A Framework for Benchmarking Recommender Systems2019 IEEE International Conference on Big Data (Big Data)10.1109/BigData47090.2019.9006199(3717-3726)Online publication date: Dec-2019
https://doi.org/10.1109/BigData47090.2019.9006199
Yuan JJin YLiu WWang X(2019)Attention-Based Neural Tag RecommendationDatabase Systems for Advanced Applications10.1007/978-3-030-18579-4_21(350-365)Online publication date: 24-Apr-2019
https://doi.org/10.1007/978-3-030-18579-4_21
Kowald DSeitlinger PLey TLex EChampin PGandon FMédini LLalmas MIpeirotis P(2018)The Impact of Semantic Context Cues on the User Acceptance of Tag RecommendationsCompanion Proceedings of the The Web Conference 201810.1145/3184558.3186899(1-2)Online publication date: 23-Apr-2018
https://dl.acm.org/doi/10.1145/3184558.3186899
d'Aquin MKowald DFessl ALex EThalmann SChampin PGandon FMédini LLalmas MIpeirotis P(2018)AFEL - Analytics for Everyday LearningCompanion Proceedings of the The Web Conference 201810.1145/3184558.3186206(439-440)Online publication date: 23-Apr-2018
https://dl.acm.org/doi/10.1145/3184558.3186206
Alslaity A(2018)A Unified Evaluation Framework for RecommendersAdvances in Artificial Intelligence10.1007/978-3-319-89656-4_39(351-354)Online publication date: 6-Apr-2018
https://doi.org/10.1007/978-3-319-89656-4_39
Alslaity ATran T(2018)Towards a Comprehensive Evaluation of Recommenders: A Cognition-Based ApproachAdvances in Artificial Intelligence10.1007/978-3-319-89656-4_32(310-315)Online publication date: 6-Apr-2018
https://doi.org/10.1007/978-3-319-89656-4_32
Kowald DKopeinik SLex EBielikova MHerder EDesmarais MCena F(2017)The TagRec Framework as a Toolkit for the Development of Tag-Based Recommender SystemsAdjunct Publication of the 25th Conference on User Modeling, Adaptation and Personalization10.1145/3099023.3099069(23-28)Online publication date: 9-Jul-2017
https://dl.acm.org/doi/10.1145/3099023.3099069
Kowald DPujari SLex EBarrett RCummings RAgichtein EGabrilovich E(2017)Temporal Effects on Hashtag Reuse in TwitterProceedings of the 26th International Conference on World Wide Web10.1145/3038912.3052605(1401-1410)Online publication date: 3-Apr-2017
https://dl.acm.org/doi/10.1145/3038912.3052605
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten