research-article

"We Need a Woman in Music": Exploring Wikipedia's Values on Article Priority

Authors:

Soumya Khandelwal,

Aviral Bhatnagar,

Loren TerveenAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 6, Issue CSCW2

Article No.: 266, Pages 1 - 28

https://doi.org/10.1145/3555156

Published: 11 November 2022 Publication History

Abstract

Wikipedia---like most peer production communities---suffers from a basic problem: the amount of work that needs to be done (articles to be created and improved) exceeds the available resources (editor effort). Recommender systems have been deployed to address this problem, but they have tended to recommend work tasks that match individuals' personal interests, ignoring more global community values. In English Wikipedia, discussion about Vital articles constitutes a proxy for community values about the types of articles that are most important, and should therefore be prioritized for improvement. We first analyzed these discussions, finding that an article's priority is considered a function of 1) its inherent importance and 2) its effects on Wikipedia's global composition. One important example of the second consideration is balance, including along the dimensions of gender and geography. We then conducted a quantitative analysis evaluating how four different article prioritization methods---two from prior research---would affect Wikipedia's overall balance on these two dimensions; we found significant differences among the methods. We discuss the implications of our results, including particularly how they can guide the design of recommender systems that take into account community values, not just individuals' interests.

References

[1]

[n.d.]. Definition of IMPORTANT. https://www.merriam-webster.com/dictionary/important

[2]

[n.d.]. Wikimedia Downloads. https://dumps.wikimedia.org/backup-index.html

[3]

[n.d.]. Wikimedia Statistics. https://stats.wikimedia.org

[4]

2011. Editor Survey 2011/Location & Language - Meta. https://meta.wikimedia.org/wiki/Editor_Survey_2011/Location_%26_Language

[5]

2014. Wikipedia's gender imbalance. BBC News (Aug. 2014). https://www.bbc.com/news/av/business-28701772

[6]

2017. Research:Studies of Importance - Meta. https://meta.wikimedia.org/w/index.php?title=Research:Studies_of_Importance&oldid=17022987

[7]

2020. Wikipedia talk:Vital articles/Frequently Asked Questions. https://en.wikipedia.org/w/index.php?title=Wikipedia_talk:Vital_articles/Frequently_Asked_Questions&oldid=951607894 Page Version ID: 951607894.

[8]

2020. Wikipedia:Featured articles. https://en.wikipedia.org/w/index.php?title=Wikipedia:Featured_articles&oldid=970548014 Page Version ID: 970548014.

[9]

2021. apache/spark. https://github.com/apache/spark original-date: 2014-02--25T08:00:08Z.

[10]

2021. Criticism of Wikipedia. https://en.wikipedia.org/w/index.php?title=Criticism_of_Wikipedia&oldid=1015572594 Page Version ID: 1015572594.

[11]

2021. List of countries by regional classification - Meta. https://meta.wikimedia.org/wiki/List_of_countries_by_regional_classification

[12]

2021. North--South divide in the World. https://en.wikipedia.org/w/index.php?title=North%E2%80%93South_divide_in_the_World&oldid=1015889006 Page Version ID: 1015889006.

[13]

2021. Wikipedia:About. https://en.wikipedia.org/w/index.php?title=Wikipedia:About&oldid=1013393289 Page Version ID: 1013393289.

[14]

2021. Wikipedia:Featured article criteria. https://en.wikipedia.org/w/index.php?title=Wikipedia:Featured_article_criteria&oldid=1015433168 Page Version ID: 1015433168.

[15]

2021. Wikipedia:Pageview statistics. https://en.wikipedia.org/w/index.php?title=Wikipedia:Pageview_statistics&oldid=1005194727 Page Version ID: 1005194727.

[16]

2021. Wikipedia:Please do not bite the newcomers. https://en.wikipedia.org/w/index.php?title=Wikipedia:Please_do_not_bite_the_newcomers&oldid=1016589894 Page Version ID: 1016589894.

[17]

2021. Wikipedia:Statistics. https://en.wikipedia.org/w/index.php?title=Wikipedia:Statistics&oldid=1013622731 Page Version ID: 1013622731.

[18]

2021. Wikipedia:Today's featured article. https://en.wikipedia.org/w/index.php?title=Wikipedia:Today%27s_featured_article&oldid=1008479339 Page Version ID: 1008479339.

[19]

Himan Abdollahpouri, Gediminas Adomavicius, Robin Burke, Ido Guy, Dietmar Jannach, Toshihiro Kamishima, Jan Krasnodebski, and Luiz Pizzato. 2020. Multistakeholder recommendation: Survey and research directions. User Modeling and User-Adapted Interaction 30, 1 (March 2020), 127--158. https://doi.org/10.1007/s11257-019-09256--1

[20]

Judd Antin, Raymond Yee, Coye Cheshire, and Oded Nov. 2011. Gender differences in Wikipedia editing. In Proceedings of the 7th International Symposium on Wikis and Open Collaboration (WikiSym '11). Association for Computing Machinery, New York, NY, USA, 11--14. https://doi.org/10.1145/2038558.2038561

Digital Library

[21]

Jacob Assa and Radhika Desai. 2015. Gross Domestic Power: Geopolitical Economy and the History of National Accounts (first edition. ed.). Theoretical Engagements in Geopolitical Economy, Vol. 30. Emerald Group Publishing? Bingley :. 175--203 pages.

[22]

Samuel Baltz. 2021. Wikipedia's political science coverage is biased. I tried to fix it. Washington Post (Feb. 2021). https://www.washingtonpost.com/politics/2021/02/24/wikipedias-political-science-coverage-is-biased-i-tried-fix-it/

[23]

Amber Berson, Monika Sengul-Jones, and Melissa Tamani. 2021. Reliable Sources and Marginalized Communities in French, English and Spanish Wikipedias. (June 2021), 49.

[24]

Pablo Beytía. 2020. The Positioning Matters: Estimating Geographical Bias in the Multilingual Record of Biographies on Wikipedia. In Companion Proceedings of the Web Conference 2020 (WWW '20). Association for Computing Machinery, New York, NY, USA, 806--810. https://doi.org/10.1145/3366424.3383569

Digital Library

[25]

Carwil Bjork-James. 2021. New maps for an inclusive Wikipedia: decolonial scholarship and strategies to counter systemic bias. New Review of Hypermedia and Multimedia 0, 0 (Jan. 2021), 1--22. https://doi.org/10.1080/13614568.2020.1865463 Publisher: Taylor & Francis _eprint: https://doi.org/10.1080/13614568.2020.1865463.

[26]

Susan L. Bryant, Andrea Forte, and Amy Bruckman. 2005. Becoming Wikipedian: transformation of participation in a collaborative online encyclopedia. In Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work (GROUP '05). Association for Computing Machinery, New York, NY, USA, 1--10. https://doi.org/10.1145/1099203.1099205

Digital Library

[27]

Noam Cohen. 2011. Define Gender Gap? Look Up Wikipedia's Contributor List. The New York Times (Jan. 2011). https://www.nytimes.com/2011/01/31/business/media/31link.html

[28]

Dan Cosley, Dan Frankowski, Loren Terveen, and John Riedl. 2007. SuggestBot: using intelligent task routing to help people find work in wikipedia. In Proceedings of the 12th international conference on Intelligent user interfaces (IUI '07). Association for Computing Machinery, New York, NY, USA, 32--41. https://doi.org/10.1145/1216295.1216309

Digital Library

[29]

Fernando Diaz, Bhaskar Mitra, Michael D. Ekstrand, Asia J. Biega, and Ben Carterette. 2020. Evaluating Stochastic Rankings with Expected Exposure. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (CIKM '20). Association for Computing Machinery, New York, NY, USA, 275--284. https://doi.org/10.1145/3340531.3411962

Digital Library

[30]

Young-Ho Eom, Pablo Aragón, David Laniado, Andreas Kaltenbrunner, Sebastiano Vigna, and Dima L. Shepelyansky. 2015. Interactions of Cultures and Top People of Wikipedia from Ranking of 24 Language Editions. PLOS ONE 10, 3 (March 2015), e0114825. https://doi.org/10.1371/journal.pone.0114825 Publisher: Public Library of Science.

[31]

Young-Ho Eom, Klaus M. Frahm, András Benczúr, and Dima L. Shepelyansky. 2013. Time evolution of Wikipedia network ranking. The European Physical Journal B 86, 12 (Dec. 2013), 492. https://doi.org/10.1140/epjb/e2013--40432--5

[32]

Young-Ho Eom and Dima L. Shepelyansky. 2013. Highlighting Entanglement of Cultures via Ranking of Multilingual Wikipedia Articles. PLOS ONE 8, 10 (Oct. 2013), e74554. https://doi.org/10.1371/journal.pone.0074554 Publisher: Public Library of Science.

[33]

Cynthia Fuchs Epstein. 2007. Great Divides: The Cultural, Cognitive, and Social Bases of the Global Subordination of Women. American Sociological Review 72, 1 (Feb. 2007), 1--22. https://doi.org/10.1177/000312240707200101 Publisher: SAGE Publications Inc.

[34]

James Gleick. 2013. Wikipedia's Women Problem. https://www.nybooks.com/daily/2013/04/29/wikipedia-women-problem/

[35]

Andreea D. Gorbatai. 2011. Exploring underproduction in Wikipedia. In Proceedings of the 7th International Symposium on Wikis and Open Collaboration (WikiSym '11). Association for Computing Machinery, New York, NY, USA, 205--206. https://doi.org/10.1145/2038558.2038595

Digital Library

[36]

Aaron Halfaker, Aniket Kittur, and John Riedl. 2011. Don't bite the newbies: how reverts affect the quantity and quality of Wikipedia work. In Proceedings of the 7th International Symposium on Wikis and Open Collaboration (WikiSym '11). Association for Computing Machinery, New York, NY, USA, 163--172. https://doi.org/10.1145/2038558.2038585

Digital Library

[37]

Raíza Hanada, Marco Cristo, and Maria da Graça Campos Pimentel. 2013. How do metrics of link analysis correlate to quality, relevance and popularity in wikipedia?. In Proceedings of the 19th Brazilian symposium on Multimedia and the web (WebMedia '13). Association for Computing Machinery, New York, NY, USA, 105--112. https://doi.org/10.1145/2526188.2526198

Digital Library

[38]

Eszter Hargittai and Aaron Shaw. 2015. Mind the skills gap: the role of Internet know-how and gender in differentiated contributions to Wikipedia. Information, Communication & Society 18, 4 (April 2015), 424--442. https://doi.org/10.1080/1369118X.2014.957711 Publisher: Routledge _eprint: https://doi.org/10.1080/1369118X.2014.957711.

[39]

Brent Hecht and Darren Gergle. 2009. Measuring self-focus bias in community-maintained knowledge repositories. In Proceedings of the fourth international conference on Communities and technologies - C&T '09. ACM Press, University Park, PA, USA, 11. https://doi.org/10.1145/1556460.1556463

Digital Library

[40]

Jonathan L. Herlocker, Joseph A. Konstan, Loren G. Terveen, and John T. Riedl. 2004. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems 22, 1 (Jan. 2004), 5--53. https://doi.org/10.1145/963770.963772

Digital Library

[41]

Benjamin Mako Hill and Aaron Shaw. 2013. The Wikipedia Gender Gap Revisited: Characterizing Survey Response Bias with Propensity Score Estimation. PLOS ONE 8, 6 (June 2013), e65782. https://doi.org/10.1371/journal.pone.0065782 Publisher: Public Library of Science.

[42]

Michael D. Intriligator. 2002. Economizing, and the Economy. In Mathematical Optimization and Economic Theory. Society for Industrial and Applied Mathematics, 2--6. https://doi.org/10.1137/1.9780898719215.ch1

[43]

Isaac Johnson, Florian Lemmerich, Diego Sáez-Trumper, Robert West, Markus Strohmaier, and Leila Zia. 2020. Global gender differences in Wikipedia readership. arXiv preprint arXiv:2007.10403 (2020).

[44]

Isaac L. Johnson, Yilun Lin, Toby Jia-Jun Li, Andrew Hall, Aaron Halfaker, Johannes Schöning, and Brent Hecht. 2016. Not at Home on the Range: Peer Production and the Urban/Rural Divide. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). Association for Computing Machinery, New York, NY, USA, 13--25. https://doi.org/10.1145/2858036.2858123

Digital Library

[45]

José Lages, Antoine Patt, and Dima L. Shepelyansky. 2016. Wikipedia ranking of world universities. The European Physical Journal B 89, 3 (March 2016), 69. https://doi.org/10.1140/epjb/e2016--60922-0

[46]

Michael Lieberman and Jimmy Lin. 2009. You Are Where You Edit: Locating Wikipedia Contributors through Edit Histories. Proceedings of the International AAAI Conference on Web and Social Media 3, 1 (March 2009), 106--113. https://ojs.aaai.org/index.php/ICWSM/article/view/13952 Number: 1.

[47]

Michael Mandiberg. 2020. Mapping Wikipedia. https://www.theatlantic.com/technology/archive/2020/02/where-wikipedias-editors-are-where-they-arent-and-why/605023/ Section: Technology.

[48]

Masoud Mansoury, Himan Abdollahpouri, Mykola Pechenizkiy, Bamshad Mobasher, and Robin Burke. 2020. FairMatch: A Graph-based Approach for Improving Aggregate Diversity in Recommender Systems. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization (UMAP '20). Association for Computing Machinery, New York, NY, USA, 154--162. https://doi.org/10.1145/3340631.3394860

Digital Library

[49]

Connor McMahon, Isaac Johnson, and Brent Hecht. 2017. The Substantial Interdependence of Wikipedia and Google: A Case Study on the Relationship Between Peer Production Communities and Information Technologies. Proceedings of the International AAAI Conference on Web and Social Media 11, 1 (May 2017). https://ojs.aaai.org/index.php/ICWSM/article/view/14883 Number: 1.

[50]

Amanda Menking and Jon Rosenberg. 2021. WP:NOT, WP:NPOV, and Other Stories Wikipedia Tells Us: A Feminist Critique of Wikipedia's Epistemology. Science, Technology, & Human Values 46, 3 (May 2021), 455--479. https://doi.org/10.1177/0162243920924783 Publisher: SAGE Publications Inc.

[51]

Brian Resnick. 2018. The 2018 Nobel Prize reminds us that women scientists too often go unrecognized. https://www.vox.com/science-and-health/2018/10/2/17929366/nobel-prize-physics-donna-strickland

[52]

Corinne Purtill Schlanger, Zoë. 2018. Wikipedia rejected an entry on a Nobel Prize winner because she wasn't famous enough. https://qz.com/1410909/wikipedia-had-rejected-nobel-prize-winner-donna-strickland-because-she-wasnt-famous-enough/

[53]

Xin Shuai, Zhuoren Jiang, Xiaozhong Liu, and Johan Bollen. 2013. A comparative study of academic and Wikipedia ranking. In Proceedings of the 13th ACM/IEEE-CS joint conference on Digital libraries (JCDL '13). Association for Computing Machinery, New York, NY, USA, 25--28. https://doi.org/10.1145/2467696.2467746

Digital Library

[54]

Philipp Singer, Florian Lemmerich, Robert West, Leila Zia, Ellery Wulczyn, Markus Strohmaier, and Jure Leskovec. 2017. Why We Read Wikipedia. In Proceedings of the 26th International Conference on World Wide Web (WWW '17). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1591--1600. https://doi.org/10.1145/3038912.3052716

Digital Library

[55]

Özge Sürer, Robin Burke, and Edward C. Malthouse. 2018. Multistakeholder recommendation with provider constraints. In Proceedings of the 12th ACM Conference on Recommender Systems (RecSys '18). Association for Computing Machinery, New York, NY, USA, 54--62. https://doi.org/10.1145/3240323.3240350

Digital Library

[56]

Andreas Thalhammer and Achim Rettinger. 2016. PageRank on Wikipedia: towards general importance scores for entities. In European Semantic Web Conference. Springer, 227--240.

[57]

Nicole Torres. 2016. Why Do So Few Women Edit Wikipedia? Harvard Business Review (June 2016). https://hbr.org/2016/06/why-do-so-few-women-edit-wikipedia Section: Gender.

[58]

Morten Warncke-Wang, Vivek Ranjan, Loren Terveen, and Brent Hecht. 2015. Misalignment Between Supply and Demand of Quality Content in Peer Production Communities. Proceedings of the International AAAI Conference on Web and Social Media 9, 1 (April 2015). https://ojs.aaai.org/index.php/ICWSM/article/view/14631 Number: 1.

[59]

Martin Wattenberg, Fernanda B. Viégas, and Katherine Hollenbach. 2007. Visualizing Activity on Wikipedia with Chromograms. In Human-Computer Interaction -- INTERACT 2007 (Lecture Notes in Computer Science), Cécilia Baranauskas, Philippe Palanque, Julio Abascal, and Simone Diniz Junqueira Barbosa (Eds.). Springer, Berlin, Heidelberg, 272--287. https://doi.org/10.1007/978--3--540--74800--7_23

[60]

Ellery Wulczyn, Robert West, Leila Zia, and Jure Leskovec. 2016. Growing Wikipedia Across Languages via Recommendation. arXiv:1604.03235 [cs] (April 2016). http://arxiv.org/abs/1604.03235 arXiv: 1604.03235.

[61]

Haiyi Zhu, Bowen Yu, Aaron Halfaker, and Loren Terveen. 2018. Value-Sensitive Algorithm Design: Method, Case Study, and Lessons. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (Nov. 2018), 194:1--194:23. https://doi.org/10.1145/3274463

Digital Library

Cited By

Champion KHill B(2024)Countering underproduction of peer produced goodsNew Media & Society10.1177/14614448241248139Online publication date: 16-May-2024
https://doi.org/10.1177/14614448241248139
Ajmani LForiest JTaylor JPittman KGilbert SDeVito M(2024)Whose Knowledge is Valued? Epistemic Injustice in CSCW ApplicationsProceedings of the ACM on Human-Computer Interaction10.1145/36870628:CSCW2(1-28)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3687062
Ajmani LVincent NChancellor S(2023)Peer Produced Friction: How Page Protection on Wikipedia Affects Editor Engagement and ConcentrationProceedings of the ACM on Human-Computer Interaction10.1145/36101987:CSCW2(1-33)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610198
Show More Cited By

Index Terms

"We Need a Woman in Music": Exploring Wikipedia's Values on Article Priority
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing systems and tools
      1. Wikis
  2. Human computer interaction (HCI)
    1. Empirical studies in HCI

Recommendations

Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

On Wikipedia, sophisticated algorithmic tools are used to assess the quality of edits and take corrective actions. However, algorithms can fail to solve the problems they were designed for if they conflict with the values of communities who use them. In ...
Examining Wikipedia With a Broader Lens: Quantifying the Value of Wikipedia's Relationships with Other Large-Scale Online Communities
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

The extensive Wikipedia literature has largely considered Wikipedia in isolation, outside of the context of its broader Internet ecosystem. Very recent research has demonstrated the significance of this limitation, identifying critical relationships ...
Fast Linking of Mathematical Wikidata Entities in Wikipedia Articles Using Annotation Recommendation
WWW '21: Companion Proceedings of the Web Conference 2021

Mathematical information retrieval (MathIR) applications such as semantic formula search and question answering systems rely on knowledge-bases that link mathematical expressions to their natural language names. For database population, mathematical ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 6, Issue CSCW2

CSCW

November 2022

8205 pages

EISSN:2573-0142

DOI:10.1145/3571154

Editor:
Jeff Nichols
Google

Issue’s Table of Contents

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 November 2022

Published in PACMHCI Volume 6, Issue CSCW2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
146
Total Downloads

Downloads (Last 12 months)58
Downloads (Last 6 weeks)11

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Champion KHill B(2024)Countering underproduction of peer produced goodsNew Media & Society10.1177/14614448241248139Online publication date: 16-May-2024
https://doi.org/10.1177/14614448241248139
Ajmani LForiest JTaylor JPittman KGilbert SDeVito M(2024)Whose Knowledge is Valued? Epistemic Injustice in CSCW ApplicationsProceedings of the ACM on Human-Computer Interaction10.1145/36870628:CSCW2(1-28)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3687062
Ajmani LVincent NChancellor S(2023)Peer Produced Friction: How Page Protection on Wikipedia Affects Editor Engagement and ConcentrationProceedings of the ACM on Human-Computer Interaction10.1145/36101987:CSCW2(1-33)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610198
Yu YMcDonald D(2023)"Why do you need 400 photographs of 400 different Lockheed Constellation?": Value Expressions by Contributors and Users of Wikimedia CommonsProceedings of the ACM on Human-Computer Interaction10.1145/36100947:CSCW2(1-34)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610094
Ajmani LHoutti MForiest JDevito MVincent NJohnson I(2023)Epistemic Injustice in Online Communities: Unpacking the Values of Knowledge Creation and Curation within CSCW ApplicationsCompanion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing10.1145/3584931.3611280(527-531)Online publication date: 14-Oct-2023
https://dl.acm.org/doi/10.1145/3584931.3611280

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents