Abstract
This study focuses on the detection of bias in news articles from a British research-intensive university, given the substantial significance of higher education institutions as information sources and their considerable influence in shaping public opinion. While prior research has underscored the existence of bias in news content, there has been limited exploration of bias detection in the specific realm of higher education news articles. To address this gap, we adopt a similar approach to Raza et al. [9] by utilising DistilBert to classify news articles published on a university website. Our primary objective is to evaluate the performance of the model in detecting bias within this domain. Furthermore, we utilise the capabilities of GPT-3.5-turbo for annotation tasks, supported by recent studies showcasing its effectiveness. In addition, we conduct performance comparisons by testing Google’s Bard for annotation tasks alongside GPT-3.5-turbo. To assess the quality of DistilBert, GPT-3.5-turbo, and Google’s Bard annotations, we acquired ground truth labels through Amazon Mturk to annotate a subset of our data. The experimental findings demonstrate that the DistilBert model, trained on the MBIC dataset, shows moderate performance in detecting biased language in university news articles. Moreover, the assessment of GPT-3.5-turbo and Google Bard annotations against human-annotated data reveals a low level of accuracy, highlighting their unreliable annotations in this domain.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ahmed, I., Kajol, M., Hasan, U., Datta, P.P., Roy, A., Reza, M.R.: ChatGPT vs. Bard: a comparative study. UMBC Student Collection. (2023)
Baly, R., Karadzhov, G., Alexandrov, D., Glass, J., Nakov, P.: Predicting factuality of reporting and bias of news media sources (2018)
ElSherief, M., et al.: Latent hatred: A benchmark for understanding implicit hate speech (2021)
Gilardi, F., Alizadeh, M., Kubli, M.: ChatGPT outperforms crowd-workers for text-annotation tasks (2023)
Huang, F., Kwak, H., An, J.: Is chatGPT better than human annotators? Potential and limitations of chatGPT in explaining implicit hate speech (2023)
Kuzman, T., Ljubešić, N., Mozetič, I.: ChatGPT: beginning of an end of manual annotation? Use case of automatic genre identification (2023)
Raimondi, R., Tzoumas, N., Salisbury, T., Di Simplicio, S., Romano, M.R.: Comparative analysis of large language models in the Royal College of Ophthalmologists fellowship exams (2023)
Raza, S.: A news recommender system considering temporal dynamics and diversity (2021)
Raza, S., Reji, D.J., Liu, D.D., Bashir, S.R., Naseem, U.: An approach to ensure fairness in news articles (2022)
Spinde, T., Rudnitckaia, L., Sinha, K., Hamborg, F., Gipp, B., Donnay, K.: MBIC--a media bias annotation dataset including annotator characteristics (2021)
Gaucher, D., Friesen, J., Kay, A.C.: Evidence that gendered wording in job advertisements exists and sustains gender inequality. J. Pers. Soc. Psychol. 101(1), 109 (2011)
Matfield, K.: Gender decoder: find subtle bias in job ads. http://gender-decoder.katmatfield.com/. Accessed 09 June 2023
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bin Shiha, R., Atwell, E., Abbas, N. (2023). Detecting Bias in University News Articles: A Comparative Study Using BERT, GPT-3.5 and Google Bard Annotations. In: Bramer, M., Stahl, F. (eds) Artificial Intelligence XL. SGAI 2023. Lecture Notes in Computer Science(), vol 14381. Springer, Cham. https://doi.org/10.1007/978-3-031-47994-6_42
Download citation
DOI: https://doi.org/10.1007/978-3-031-47994-6_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47993-9
Online ISBN: 978-3-031-47994-6
eBook Packages: Computer ScienceComputer Science (R0)