Cited By
View all- V. JS. A(2024)A multi-aspect framework for explainable sentiment analysisPattern Recognition Letters10.1016/j.patrec.2024.01.001178:C(122-129)Online publication date: 1-Feb-2024
“Do we like this, or do we like like this?”: Reflections on a Human-Centered Machine Learning Approach to Sentiment Analysis
Authors: 
Sourojit Ghosh, Murtaza Ali, Anna Batra, Cheng Guo, Mohit Jain, Joseph Kang, Julia Kharchenko, Varun Suravajhela, Vincent Zhou, 
Cecilia AragonAuthors Info & Claims
Pages 63 - 82
Abstract
Machine Learning is a powerful tool, but it also has a great potential to cause harm if not approached carefully. Designers must be reflexive and aware of their algorithms’ impacts, and one such way of reflection is known as human-centered machine learning. In this paper, we approach a classical problem that has been approached through ML - sentiment analysis - through a Human-Centered Machine Learning lens. Through a case study of trying to differentiate between degrees of positive emotions in reviews of online fanfiction, we offer a set of recommendations for future designers of ML-driven sentiment analysis algorithms.
References
[1]
Ahmad M, Aftab S, and Ali I Sentiment analysis of tweets using svm Int. J. Comput. Appl. 2017 177 5 25-29
[2]
Al Amrani Y, Lazaar M, and El Kadiri KE Random forest and support vector machine based hybrid approach to sentiment analysis Procedia Comput. Sci. 2018 127 511-520
[3]
Altman M, Wood A, and Vayena E A harm-reduction framework for algorithmic fairness IEEE Secur. Privacy 2018 16 3 34-45
[4]
Andalibi, N., Buss, J.: The human in emotion recognition on social media: attitudes, outcomes, risks. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–16 (2020)
[5]
Annett M and Kondrak G Bergler S A comparison of sentiment analysis techniques: polarizing movie blogs Advances in Artificial Intelligence 2008 Heidelberg Springer 25-35
[6]
Aragon, C., Guha, S., Kogan, M., Muller, M., Neff, G.: Human-Centered Data Science: An Introduction. MIT Press (2022)
[7]
Black RW Language, culture, and identity in online fanfiction E-learning Digital Media 2006 3 2 170-184
[8]
Bond, F., Foster, R.: Linking and extending an open multilingual wordnet. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1352–1362 (2013)
[9]
Boyd D and Crawford K Critical questions for big data: provocations for a cultural, technological, and scholarly phenomenon Inf. Commun. Soc. 2012 15 5 662-679
[10]
Burnap P, Rana OF, Avis N, Williams M, Housley W, Edwards A, Morgan J, and Sloan L Detecting tension in online communities with computational twitter analysis Technol. Forecast. Soc. Chang. 2015 95 96-108
[11]
Cambo, S.A., Gergle, D.: Model positionality and computational reflexivity: promoting reflexivity in data science. In: CHI Conference on Human Factors in Computing Systems, pp. 1–19 (2022)
[12]
Campbell, J., Aragon, C., Davis, K., Evans, S., Evans, A., Randall, D.: Thousands of positive reviews: distributed mentoring in online fan communities. In: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, pp. 691–704 (2016)
[13]
Chancellor, S.: Towards practices for human-centered machine learning. arXiv preprint arXiv:2203.00432 (2022)
[14]
Chancellor, S., Baumer, E.P., De Choudhury, M.: Who is the “human” in human-centered machine learning: the case of predicting mental health from social media. In: Proceedings of the ACM on Human-Computer Interaction 3(CSCW), pp. 1–32 (2019)
[15]
Chen NC, Drouhard M, Kocielnik R, Suh J, and Aragon CR Using machine learning to support qualitative coding in social science: shifting the focus to ambiguity ACM Trans. Interact. Intell. Syst. (TiiS) 2018 8 2 1-20
[16]
Costanza-Chock, S.: Design justice: community-led practices to build the worlds we need. The MIT Press (2020)
[17]
Daeli NOF and Adiwijaya A Sentiment analysis on movie reviews using information gain and k-nearest neighbor J. Data Sci. Appl. 2020 3 1 1-7
[18]
Díaz, M., Johnson, I., Lazar, A., Piper, A.M., Gergle, D.: Addressing age-related bias in sentiment analysis. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2018)
[19]
Draude, C., Klumbyte, G., Lücking, P., Treusch, P.: Situated algorithms: a sociotechnical systemic approach to bias. Online Information Review (2019)
[20]
Dym, B., Brubaker, J.R., Fiesler, C., Semaan, B.: “Coming out okay” community narratives for LGBTQ identity recovery work. In: Proceedings of the ACM on Human-Computer Interaction 3(CSCW), pp. 1–28 (2019). https://dl.acm.org/doi/10.1145/3359256
[21]
Ekman, P.: All emotions are basic. The nature of emotion: Fundamental questions, pp. 15–19 (1994)
[22]
Evans, S., Davis, K., Evans, A., Campbell, J.A., Randall, D.P., Yin, K., Aragon, C.: More than peer production: Fanfiction communities as sites of distributed mentoring. In: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, pp. 259–272 (2017)
[23]
Fiesler, C., Proferes, N.: “Participant” perceptions of twitter research ethics. Social Media+ Society 4(1), 2056305118763366 (2018)
[24]
Figueroa, A., Ghosh, S., Aragon, C.: Generalized cohen’s kappa: a novel inter-rater reliability metric for non-mutually exclusive categories. In: Proceedings of the Human Interface and the Management of Information Thematic Area in the context of the 25th International Conference on Human-Computer Interaction (HCI International). Springer (2023)
[25]
Ghosh, S., Figueroa, A.: Establishing tiktok as a platform for informal learning: Evidence from mixed-methods analysis of creators and viewers. In: Proceedings of the 56th Hawaii International Conference on System Sciences, pp. 2431–2440 (2023)
[26]
Ghosh, S., Froelich, N., Aragon, C.: “i love you, my dear friend”: analyzing the role of emotions in the building of friendships in online fanfiction communities. In: Proceedings of the 15th International Conference on Social Computing and Social Media in the context of the 25th International Conference on Human-Computer Interaction (HCI International). Springer (2023)
[27]
Goel, A., Gautam, J., Kumar, S.: Real time sentiment analysis of tweets using naive bayes. In: 2016 2nd International Conference on Next Generation Computing Technologies (NGCT), pp. 257–261. IEEE (2016)
[28]
Gui, X., Chen, Y., Kou, Y., Pine, K., Chen, Y.: Investigating support seeking from peers for pregnancy in online health communities. In: Proceedings of the ACM on Human-Computer Interaction 1(CSCW), pp. 1–19 (2017)
[29]
Gupta, A., Lamba, H., Kumaraguru, P., Joshi, A.: Faking sandy: characterizing and identifying fake images on twitter during hurricane sandy. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 729–736 (2013)
[30]
Hegde, Y., Padma, S.: Sentiment analysis using random forest ensemble for mobile product reviews in kannada. In: 2017 IEEE 7th International Advance Computing Conference (IACC), pp. 777–782. IEEE (2017)
[31]
Hicks, A., Rutherford, M., Fellbaum, C., Bian, J.: An analysis of wordnet’s coverage of gender identity using twitter and the national transgender discrimination survey. In: Proceedings of the 8th Global WordNet Conference (GWC), pp. 123–130 (2016)
[32]
Hong, L., Doumith, A.S., Davison, B.D.: Co-factorization machines: modeling user interests and predicting individual decisions in twitter. In: Proceedings of the sixth ACM International Conference on Web Search and Data Mining, pp. 557–566 (2013)
[33]
Hu, X., Tang, J., Gao, H., Liu, H.: Unsupervised sentiment analysis with emotional signals. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 607–618 (2013)
[34]
Huq, M.R., Ahmad, A., Rahman, A.: Sentiment analysis on twitter data using KNN and svm. Int. J. Adv. Comput. Sci. Appl. 8(6) (2017)
[35]
Kassens-Noor E Twitter as a teaching practice to enhance active and informal learning in higher education: the case of sustainable tweets Act. Learn. High. Educ. 2012 13 1 9-21
[36]
Kaya, M., Fidan, G., Toroslu, I.H.: Sentiment analysis of Turkish political news. In: 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology, vol. 1, pp. 174–180. IEEE (2012)
[37]
Kivran-Swaine, F., Brody, S., Diakopoulos, N., Naaman, M.: Of joy and gender: emotional expression in online social networks. In: The ACM Conference on Computer Supported Cooperative Work Companion, pp. 139–142 (2012)
[38]
Lazer D, Pentland A, Adamic L, Aral S, Barabási AL, Brewer D, Christakis N, Contractor N, Fowler J, Gutmann M, et al. Computational social science Science 2009 323 5915 721-723
[39]
Levonian, Z., Dow, M., Erikson, D., Ghosh, S., Miller Hillberg, H., Narayanan, S., Terveen, L., Yarosh, S.: Patterns of patient and caregiver mutual support connections in an online health community. In: Proceedings of the ACM on Human-Computer Interaction 4(CSCW3), pp. 1–46 (2021)
[40]
López-Chau A, Valle-Cruz D, and Sandoval-Almazán R Ramachandran M and Mahmood Z Sentiment analysis of twitter data through machine learning techniques Software Engineering in the Era of Cloud Computing 2020 Cham Springer 185-209
[41]
Lulu: The slow dance of the infinite stars (2013)
[42]
Lulu: Archive of our own: 2020 statistics, November 2020
[43]
Lulu: Archive of our own: Overall gender and sexuality of ao3 users, November 2020
[44]
Maynard, D.G., Greenwood, M.A.: Who cares about sarcastic tweets? investigating the impact of sarcasm on sentiment analysis. In: Lrec 2014 proceedings. ELRA (2014)
[45]
Miller GA Wordnet: a lexical database for English Commun. ACM 1995 38 11 39-41
[46]
Mohammad SM Ethics sheet for automatic emotion recognition and sentiment analysis Comput. Linguist. 2022 48 2 239-278
[47]
Nasukawa, T., Yi, J.: Sentiment analysis: capturing favorability using natural language processing. In: Proceedings of the 2nd International Conference on Knowledge Capture, pp. 70–77 (2003)
[48]
Neviarouskaya A, Prendinger H, and Ishizuka M Sentiful: a lexicon for sentiment analysis IEEE Trans. Affect. Comput. 2011 2 1 22-36
[49]
Noble, S.U.: Algorithms of oppression. In: Algorithms of Oppression. New York University Press (2018)
[50]
O’neil, C.: Weapons of math destruction: How big data increases inequality and threatens democracy. Broadway books (2016)
[51]
Ortigosa-Hernández J, Rodríguez JD, Alzate L, Lucania M, Inza I, and Lozano JA Approaching sentiment analysis by using semi-supervised learning of multi-dimensional classifiers Neurocomputing 2012 92 98-115
[52]
Pang, B., Lee, L., et al.: Opinion mining and sentiment analysis. Found. Trends Inf. Retrieval 2(1–2), 1–135 (2008)
[53]
Poria, S., Gelbukh, A., Cambria, E., Yang, P., Hussain, A., Durrani, T.: Merging senticnet and wordnet-affect emotion lists for sentiment analysis. In: 2012 IEEE 11th International Conference on Signal Processing, vol. 2, pp. 1251–1255. IEEE (2012)
[54]
Rana, S., Singh, A.: Comparative analysis of sentiment orientation using SVM and naive bayes techniques. In: 2016 2nd International Conference on Next Generation Computing Technologies (NGCT), pp. 106–111. IEEE (2016)
[55]
Rennie, J.D., Shih, L., Teevan, J., Karger, D.R.: Tackling the poor assumptions of naive bayes text classifiers. In: Proceedings of the 20th International Conference on Machine Learning (ICML-03), pp. 616–623 (2003)
[56]
Roback, A., Hemphill, L.: “i’d have to vote against you” issue campaigning via Twitter. In: Proceedings of the 2013 Conference on Computer Supported Cooperative Work Companion, pp. 259–262 (2013)
[57]
Rudnicka, E., Bond, F., Grabowski, Ł., Piasecki, M., Piotrowski, T.: Lexical perspective on wordnet to wordnet mapping. In: Proceedings of the 9th Global Wordnet Conference, pp. 209–218 (2018)
[58]
Saif H, He Y, and Alani H Cudré-Mauroux P, Heflin J, Sirin E, Tudorache T, Euzenat J, Hauswirth M, Parreira JX, Hendler J, Schreiber G, Bernstein A, and Blomqvist E Semantic Sentiment analysis of Twitter The Semantic Web – ISWC 2012 2012 Heidelberg Springer 508-524
[59]
Scheuerman MK, Wade K, Lustig C, and Brubaker JR How we’ve taught algorithms to see identity: Constructing race and gender in image databases for facial analysis Proceedings of the ACM on Human-computer Interaction 2020 4 CSCW1 1-35
[60]
Shen, J.H., Fratamico, L., Rahwan, I., Rush, A.M.: Darling or babygirl? investigating stylistic bias in sentiment analysis. In: Proc. of FATML (2018)
[61]
Singh, A.K., Shashi, M.: Vectorization of text documents for identifying unifiable news articles. Int. J. Adv. Comput. Sci. Appl. 10(7) (2019)
[62]
Stanoevska-Slabeva, K., Schmid, B.F.: A typology of online communities and community supporting platforms. In: Proceedings of the 34th Annual Hawaii International Conference on System Sciences, pp. 10-pp. IEEE (2001)
[63]
Sterling, S., Marton, H.: Design justice: an exhibit of emerging design practices, vol. 2. The Allied Media Conference (2016)
[64]
Suresh, H., Guttag, J.: A framework for understanding sources of harm throughout the machine learning life cycle. In: Equity and Access in Algorithms, Mechanisms, and Optimization, pp. 1–9 (2021)
[65]
Thelwall, M.: Gender bias in sentiment analysis. Online Information Review (2018)
[66]
Tosenberger, C.: “Oh my god, the fanfiction!”: Dumbledore’s outing and the online harry potter fandom. Children’s Literature Association Quarterly 33(2), 200–206 (2008)
[67]
Venigalla, A.S.M., Chimalakonda, S., Vagavolu, D.: Mood of india during covid-19-an interactive web portal based on emotion analysis of twitter data. In: Conference Companion Publication of the 2020 on Computer Supported Cooperative Work and Social Computing, pp. 65–68 (2020)
[68]
Wang, S.I., Manning, C.D.: Baselines and bigrams: Simple, good sentiment and topic classification. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 90–94 (2012)
[69]
Wiens J, Saria S, Sendak M, Ghassemi M, Liu VX, Doshi-Velez F, Jung K, Heller K, Kale D, Saeed M, et al. Do no harm: a roadmap for responsible machine learning for health care Nat. Med. 2019 25 9 1337-1340
[70]
Yang, X., Steck, H., Liu, Y.: Circle-based recommendation in online social networks. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1267–1275 (2012)
[71]
Yin, K., Aragon, C., Evans, S., Davis, K.: Where no one has gone before: a meta-dataset of the world’s largest fanfiction repository. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 6106–6110 (2017)
[72]
Zadeh, A., Chen, M., Poria, S., Cambria, E., Morency, L.P.: Tensor fusion network for multimodal sentiment analysis. Empirical Methods in Natural Language Processing (2017)
Recommendations
Human-Centred Machine Learning
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing SystemsMachine learning is one of the most important and successful techniques in contemporary computer science. It involves the statistical inference of models (such as classifiers) from data. It is often conceived in a very impersonal way, with algorithms ...
Sentiment-Specific Representation Learning for Document-Level Sentiment Analysis
WSDM '15: Proceedings of the Eighth ACM International Conference on Web Search and Data MiningIn this paper, we propose a representation learning research framework for document-level sentiment analysis. Given a document as the input, document-level sentiment analysis aims to automatically classify its sentiment/opinion (such as thumbs up or ...
Sentiment Analysis of Twitter Data Using Machine Learning Techniques and Scikit-learn
ACAI '18: Proceedings of the 2018 International Conference on Algorithms, Computing and Artificial IntelligenceSentiment analysis of Twitter data is an area that has experienced significant growth in recent years. The ability to identify sentiment from tweets using machine learning techniques has attracted researchers because of the simple efficiency of machine ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Jul 2023
682 pages
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 23 July 2023
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 18 Feb 2025
Other Metrics
Citations
Cited By
View all- V. JS. A(2024)A multi-aspect framework for explainable sentiment analysisPattern Recognition Letters10.1016/j.patrec.2024.01.001178:C(122-129)Online publication date: 1-Feb-2024