research-article

The challenges of sentiment detection in the social programmer ecosystem

Authors:

Nicole Novielli,

Fabio Calefato,

Filippo LanubileAuthors Info & Claims

SSE 2015: Proceedings of the 7th International Workshop on Social Software Engineering

Pages 33 - 40

https://doi.org/10.1145/2804381.2804387

Published: 01 September 2015 Publication History

Abstract

A recent research trend has emerged to study the role of affect in in the social programmer ecosystem, by applying sentiment analysis to the content available in sites such as GitHub and Stack Overflow. In this paper, we aim at assessing the suitability of a state-of-the-art sentiment analysis tool, already applied in social computing, for detecting affective expressions in Stack Overflow. We also aim at verifying the construct validity of choosing sentiment polarity and strength as an appropriate way to operationalize affective states in empirical studies on Stack Overflow. Finally, we underline the need to overcome the limitations induced by domain-dependent use of lexicon that may produce unreliable results.

References

[1]

Althoff, T., Danescu-Niculescu-Mizil, C., and Jurafsky, D. 2014. How to Ask for a Favor: A Case Study on the Success of Altruistic Requests. In Proceedings of the 8th International AAAI Conference on Weblogs and Social Media (ICWSM 2014).

[2]

Anderson, A., Huttenlocher, D., Kleinberg, J. and Leskovec. J. 2012. Discovering value from community activity on focused question answering sites: a case study of stack overflow. Proc. of KDD '12. ACM, 850-858.

Digital Library

[3]

Asaduzzaman, M., Mashiyat, A.S., Roy, C.K., Schneider, K.A. 2013. Answering questions about unanswered questions of Stack Overflow, Proc. of MSR 2013, 97-100.B.

Digital Library

[4]

Akkaya, C., Wiebe, J., Conrad, A., and Mihalcea, R. 2011. Improving the Impact of Subjectivity Word Sense Disambiguation on Contextual Opinion Analysis. Proc. Of CoNNL ’11, 87-96.

Digital Library

[5]

Austin, J 1962. How to do Things with Words. Oxford University Press, New York.

[6]

Basile, P. and Novielli, N. 2015. UNIBA: Sentiment Analysis of English Tweets Combining Micro-blogging, Lexicon and Semantic Features. Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), ACL, 595—600.

[7]

Batliner, A., Fisher, K., Huber, R., Spilker, J., No ̈th, E. 2003. How to find trouble in communication. Speech Communication 40, 117–143.

Digital Library

[8]

Brooks, M., Kuksenok, K., Torkildson, M.K., Perry, D., Robinson, J.J., Scott, T.J., Anicello, O., Zukowski, A., Harris, P., and Aragon, C.R. 2013. Statistical affect detection in collaborative chat. In Proceedings of the 2013 conference on Computer supported cooperative work (CSCW '13). ACM, New York, NY, USA, 317-328.

Digital Library

[9]

Calefato, F., Lanubile, F., Marasciulo, M.C., Novielli, N. 2015. MSR Challenge: “Mining Successful Answers in Stack Overflow.” In Proc. 12th IEEE Working Conf. on Mining Software Repositories (MSR ‘15), Florence, Italy, May 16- 17, 2015.

Digital Library

[10]

Clore, Gerald L.; Ortony, Andrew; Foss, Mark A. 1987. The psychological foundations of the affective lexicon. Journal of Personality and Social Psychology, Vol 53(4), Oct 1987, 751-766.

[11]

Cowie, R. 2006. Emotional life, terminological and conceptual clarifications. Available at the HUMAINE Portal: http://emotion-research.net/deliverables/D3i final.pdf

[12]

Ekman, P. 1999. Basic Emotions. Handbook of Cognition and Emotion, John Wiley & Sons Ltd.

[13]

Esuli, A. and Sebastiani, F. 2006. Sentiwordnet: A publicly available lexical resource for opinion mining. Proceedings of LREC, volume 6, 417–422.

[14]

Fellbaum C. WordNet: An Electronic Lexical Database. Language, Speech, and Communication. MIT Press, 1998.

[15]

Gamon, M., Aue, A., Corston-Oliver, S. and Ringger, E. 2005. Pulse: Mining customer opinions from free text. LNCS 3646, 121-132.

Digital Library

[16]

Guzman, E. and Bruegge, B. 2013. Towards emotional awareness in software development teams. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2013). ACM, New York, NY, USA, 671-674.

Digital Library

[17]

Guzman, E., Azócar, D., and Li, Y. 2014. Sentiment analysis of commit comments in GitHub: an empirical study. In Proc. of the 11th Working Conf. on Mining Software Repositories (MSR 2014). ACM, New York, NY, USA, 352-355.

Digital Library

[18]

Hu, M. and Liu, B. 2004. Mining and Summarizing Customer Reviews. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 168–177.

Digital Library

[19]

Kucuktunc, O., Cambazoglu, B.B., Weber, I., and Ferhatosmanoglu, H. 2012. A large-scale sentiment analysis for Yahoo! answers. In Proceedings of the fifth ACM international conference on Web search and data mining (WSDM '12). ACM, New York, NY, USA, 633-642.

Digital Library

[20]

Lazarus R S (1991) Emotion and adaptation. New York: Oxford University Press.

[21]

Le Tallec, M., Antoine, J.-Y., Villaneau, J. and Duhaut, D. 2011. Affective interaction with a companion robot for hospitalized children: a linguistically based model for emotion detection. Proc. of the 5th Language and Technology Conference (LTC2011).

[22]

Litman, D. Forbes-Riley, K., Silliman, S. 2003. Towards emotion prediction in spoken tutoring dialogues. In: Companion Proceedings of the Human Language Technology Conference: 3rd Meeting of the North American Chapter of the Association for Computational Linguistics (HLT/NAACL), Kluwer, Amsterdam, pp. 52–54.

Digital Library

[23]

Liu, H., Lieberman, H. and Selker, T. 2003. A model of textual affect sensing using real-world knowledge. Proceedings of the 8th international conference on Intelligent user interfaces, ser. IUI

Digital Library

[24]

Saif M. Mohammad and Peter D. Turney. 2010. Emotions Evoked by Common Words and Phrases: Using Mechanical Turk to Create an Emotion Lexicon. In Proc. of the NAACL HLT 2010 Workshop CAAGET ’10, pages 26–34.

Digital Library

[25]

Saif Mohammad, Svetlana Kiritchenko, and Xiaodan Zhu. 2013. Nrc-canada: Building the state-of-the art in sentiment analysis of tweets. Proc. SemEval 2013, 321–327.

[26]

Murgia, A., Tourani, P., Adams, B., and Ortu, M. 2014. Do developers feel emotions? An exploratory analysis of emotions in software artifacts. In Proceedings of the 11th Working Conference on Mining Software Repositories (MSR 2014). ACM, New York, NY, USA, 262-271.

Digital Library

[27]

Neviarouskaya, A., Prendinger, H. and Ishizuka, M. 2011. Affect analysis model: Novel rule-based approach to affect sensing from text. Nat. Lang. Eng., vol. 17(1), 95–135.

Digital Library

[28]

Nielsen, F.A. 2011. A New ANEW: Evaluation of a Word List for Sentiment Analysis in Microblogs. MSM 2011: 93- 98.

[29]

Novielli, N., Calefato, F., Lanubile, F. 2014. Towards discovering the role of emotions in stack overflow. In Proc. of 6th Int’l Workshop on Social Software Engineering (SSE ‘14). ACM, New York, NY, USA, 33-36

Digital Library

[30]

Novielli, N., de Rosis, F. and Mazzotta, I. 2010. User Attitude Towards an Embodied Conversational Agent: Effects of the Interaction Mode. Journal of Pragmatics, 42(9), Elsevier Science, 2385-2397.

[31]

Novielli N. and Strapparava, C. 2013. The Role of Affect Analysis in Dialogue Act Identification. IEEE Transactions on Affective Computing, 4:439–451.

[32]

O’Connor, B, Balasubramanyan, R., Routledge, B., and Smith, N. 2010. From tweets to polls: Linking text sentiment to public opinion time series. In Intl AAAI Conf. on Weblogs and Social Media (ICWSM), volume 11, pages 122–129.

[33]

Pang, B. and Lee, L. 2008. Opinion Mining and Sentiment Analysis. Foundations and trends in information retrieval, 2(1-2):1–135.

Digital Library

[34]

Pennebaker, J. and Francis, M. 2001. Linguistic Inquiry and Word Count: LIWC. Erlbaum Publishers.

[35]

Picard, R. W. 2000. Affective Computing. MIT Press.

[36]

Plutchick, R. 1984. Emotions: A general psychoevolutionary theory. In K.R. Scherer & P. Ekman (Eds) Approaches to emotion. Hillsdale, NJ; Lawrence Ealrbaum Associates.

[37]

Polanyi, L. and Zaenen, A. 2006. Contextual Valence Shifters. In Computing Attitude and Affect in Text: Theory and Applications, The Information Retrieval Series, 1-10

[38]

Riloff, E., Patwardhan, S., & Wiebe, J. 2006. Feature subsumption for opinion analysis. Proceedings of the Conference on Empirical Methods in Natural Language Processing, 440-448.

Digital Library

[39]

Rosenthal, R., Nakov, P., Kiritchenko, S., Mohammad, S.M., Ritter, A., and Stoyanov, V. 2015. SemEval-2015 Task 10: Sentiment Analysis in Twitter. In Proceedings of SemEval 2015.

[40]

Russell, J A (2003) Core affect and the psychological construction of emotion.

[41]

Storey, M-A. 2012. The evolution of the social programmer. In Proc of the 9th IEEE Working Conf on Mining Software Repositories (MSR '12). IEEE Press, Piscataway, NJ, USA, 140-140.

Digital Library

[42]

Leif Singer, Fernando Figueira Filho, Brendan Cleary, Christoph Treude, Margaret-Anne Storey, and Kurt Schneider. 2013. Mutual assessment in the social programmer ecosystem: an empirical investigation of developer profile aggregators. In Proceedings of the 2013 conference on Computer supported cooperative work (CSCW '13). ACM, New York, NY, USA, 103-116.

Digital Library

[43]

Storey, M-A., Singer, L., Cleary, B., Figueira Filho, F. and Zagalsky, A. 2014. The (R) Evolution of social media in software engineering. In Proceedings of the on Future of Software Engineering (FOSE ‘14). ACM, New York, NY, USA, 100-116. DOI=10.1145/2593882.2593887

Digital Library

[44]

Strapparava, C., and Valitutti, A. 2004. WordNet-affect: an affective extension of WordNet. In Proceedings of the 4th International Conference on Language Resources and Evaluation, 1083-1086.

[45]

Strapparava, C., Valitutti, A., & Stock, O. 2006. The affective weight of lexicon. Proceedings of the Fifth International Conference on Language Resources and Evaluation, LREC 2006.

[46]

Taboada, M., Brooke, J., Tofiloski, M., Voll, K.and Stede, M. 2011. Lexicon-based methods for sentiment analysis. Comput. Linguist. 37, 2, 267-307.

Digital Library

[47]

Thelwall, M. Buckley, K. and Paltoglou, G. 2012. Sentiment Strength Detection for the Social Web. Journal of the American Society for Information Science and Technology, 63(1):163-173.

Digital Library

[48]

Vapnik, V. 1995. The Nature of Statistical Learning Theory. Springer-Verlag.

Digital Library

[49]

Wilson, T., Wiebe, J. and Hoffman, P. 2005. Recognizing Contextual Polarity in Phrase-level Sentiment Analysis. Proc. of HLT ’05, 347–354.

Digital Library

[50]

Wittgenstein, L. 1965. Philosophical Investigations. The Macmillan Company, New York, NY, USA.

Cited By

Anwar ZAfzal HKadry SCheng X(2024)Semantic Web Approaches in Stack OverflowInternational Journal on Semantic Web and Information Systems10.4018/IJSWIS.35861720:1(1-61)Online publication date: 9-Nov-2024
https://doi.org/10.4018/IJSWIS.358617
Grassi D(2024)Supporting Developers' Emotional Awareness: from Self-reported Emotions to BiometricsProceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering10.1145/3661167.3661209(500-504)Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3661167.3661209
Anwar ZAfzal H(2024)Mining crowd sourcing repositories for open innovation in software engineeringAutomated Software Engineering10.1007/s10515-023-00410-z31:1Online publication date: 8-Jan-2024
https://doi.org/10.1007/s10515-023-00410-z
Show More Cited By

Index Terms

The challenges of sentiment detection in the social programmer ecosystem
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Towards discovering the role of emotions in stack overflow
SSE 2014: Proceedings of the 6th International Workshop on Social Software Engineering

Today, people increasingly try to solve domain-specific problems through interaction on online Question and Answer (Q&A) sites, such as Stack Overflow. The growing success of the Stack Overflow community largely depends on the will of their members to ...
Social sentiment sensor: a visualization system for topic detection and topic sentiment analysis on microblog

As a new form of social media, microblogging provides platform sharing, wherein users can share their feelings and ideas on certain topics. Bursty topics from microblogs are the results of the emerging issues that instantly attract more followers and ...
Sentiment Polarity Detection for Software Development
Abstract
The role of sentiment analysis is increasingly emerging to study software developers’ emotions by mining crowd-generated content within social software engineering tools. However, off-the-shelf sentiment analysis tools have been trained on non-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SSE 2015: Proceedings of the 7th International Workshop on Social Software Engineering

September 2015

52 pages

ISBN:9781450338189

DOI:10.1145/2804381

Program Chairs:
Imed Hammouda,
Alberto Sillitti

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 September 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ESEC/FSE'15

Sponsor:

SIGSOFT

ESEC/FSE'15: Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering

September 1, 2015

Bergamo, Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

85
Total Citations
View Citations
539
Total Downloads

Downloads (Last 12 months)32
Downloads (Last 6 weeks)1

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Anwar ZAfzal HKadry SCheng X(2024)Semantic Web Approaches in Stack OverflowInternational Journal on Semantic Web and Information Systems10.4018/IJSWIS.35861720:1(1-61)Online publication date: 9-Nov-2024
https://doi.org/10.4018/IJSWIS.358617
Grassi D(2024)Supporting Developers' Emotional Awareness: from Self-reported Emotions to BiometricsProceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering10.1145/3661167.3661209(500-504)Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3661167.3661209
Anwar ZAfzal H(2024)Mining crowd sourcing repositories for open innovation in software engineeringAutomated Software Engineering10.1007/s10515-023-00410-z31:1Online publication date: 8-Jan-2024
https://doi.org/10.1007/s10515-023-00410-z
Kuriu MMendonça IAritsugi M(2023)Automatic Rule Definition for Pattern-Based Text Mining2023 IEEE International Conference on Big Data and Smart Computing (BigComp)10.1109/BigComp57234.2023.00039(187-194)Online publication date: Feb-2023
https://doi.org/10.1109/BigComp57234.2023.00039
Mondal SUddin GRoy C(2023)Automatic prediction of rejected edits in Stack OverflowEmpirical Software Engineering10.1007/s10664-022-10242-228:1Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1007/s10664-022-10242-2
Novielli NSerebrenik A(2023)Emotion Analysis in Software EcosystemsSoftware Ecosystems10.1007/978-3-031-36060-2_5(105-127)Online publication date: 26-May-2023
https://doi.org/10.1007/978-3-031-36060-2_5
Mens TRoover C(2023)An Introduction to Software EcosystemsSoftware Ecosystems10.1007/978-3-031-36060-2_1(1-29)Online publication date: 26-May-2023
https://doi.org/10.1007/978-3-031-36060-2_1
Alrababah SGan KTan TAl-Kabi M(2022)Product Aspect Ranking Using Multi-Criteria Decision MakingHandbook of Research on Consumer Behavior Change and Data Analytics in the Socio-Digital Era10.4018/978-1-6684-4168-8.ch004(74-94)Online publication date: 24-Jun-2022
https://doi.org/10.4018/978-1-6684-4168-8.ch004
Li ZYu YWang TLi SWang H(2022)Opportunities and Challenges in Repeated Revisions to Pull-Requests: An Empirical StudyProceedings of the ACM on Human-Computer Interaction10.1145/35552086:CSCW2(1-35)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555208
Sun KShi XGao HKuang HMa XRong GShao DZhao ZZhang H(2022)Incorporating Pre-trained Transformer Models into TextCNN for Sentiment Analysis on Software Engineering TextsProceedings of the 13th Asia-Pacific Symposium on Internetware10.1145/3545258.3545273(127-136)Online publication date: 11-Jun-2022
https://dl.acm.org/doi/10.1145/3545258.3545273
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents