The ``Colonial Impulse" of Natural Language Processing: An Audit of Bengali Sentiment Analysis Tools and Their Identity-based Biases

While identity is often construed as an individuated concept, identities are often influenced by people’s cultural background and social interactions [8, 35]. Thus, various social identities emerge centered around people’s perceived membership in different groups [171]. In this view, people’s identities are defined across various dimensions, such as race, ethnicity, gender, sexual orientation, religion, nationality, and caste. Within each dimension (e.g., religion), people can identify with different categories (e.g., Christian) [110]. Importantly, people’s identities across various dimensions interconnect and overlap, and the consequent intersectional identities collectively shape their unique experiences, social position, and systemic privilege [46, 49]. This is best illustrated through how marginalization–the process wherein people are pushed to the boundary of society and denied agency and voice based on their intersectional social identities–is normalized through cultural hegemony [46, 49]. Cultural hegemony is a system of ideas, practices, and social relationships embedded within private and institutional domains as a mechanism of power and control. Through cultural hegemony, people are categorized as a mechanism of power where some identities are considered “normative" while others are considered non-normative. In other words, people experience everyday harm and are marginalized by virtue of being born Black, Queer, or into a lower Caste.

A global practice that shaped and continues to shape the hegemonic structures of society and, in turn, people’s everyday experiences is coloniality. While colonization has deeply impacted people’s identity, coloniality refers to its enduring and pervasive effects on the local and indigenous communities even after the direct colonial rule has ended [115]. These continue to perpetuate colonial structures and social, economic, political, and cultural dynamics. Among other dimensions of identity, European colonialism imposed its conceptualization of gender on many indigenous communities [108]. Scholars have studied colonized Bengali societies to understand the complex relationship between colonialism and gender [60, 159]. British colonization, they argue, produced a particular kind of masculine identity, wherein the “manly Englishman" was contrasted with the stereotyped “effeminate Bengali" in order to justify British rule and denigrate Bengali culture [159]. Such colonial masculinity had profound impacts on gender and ethnic relations. This view led to the stereotyped views of Bengali men in colonial India [60, 132] and the reinforcement of “traditional gender roles" in Bengal [160]. This minimized women’s sociopolitical participation and voices [164].

The imposition of European standards also distorted people’s religious values and perceptions of the Indian subcontinent. Scholars have attributed the rise of religious extremism and the violence against minorities in the region to colonial values and divide-and-rule practices [57, 118]. They argue that religion-based nationalism is a reactive ideology that emerged in response to the challenges posed by colonialism and the West, where local people have adopted many ideas and practices of Abrahamic religions, such as the emphasis on a single, monolithic God [118, p. 24] and the belief in a chosen people [118, p. 101]. Especially due to cultural assimilation–the idea that colonizers’ culture is superior to that of the native communities [72] and cultural genocide–the destruction and theft of cultural sites and artifacts [177], as the colonized subjects were denied the opportunities to explore, understand, and practice their own culture, local and native communities’ self-perception regarding religion changed. Moreover, the British colonizers amplified, exploited, and institutionalized local communities’ religious differences and divisions [39].

Across the world, colonizers introduced classifications to partition different nation-states based on their own perceptions of nationhood and societal groupings of the native communities (e.g., two-nation theory in India-Pakistan) [81]. Such outlooks disregarded the latter’s intricate self-perceptions and interconnectedness [39]. Before their departure in 1947, British colonizers partitioned the Indian subcontinent, prioritizing religion as the only dimension of people’s collective identity. In the context of Bengal, West Bengal, with its upper-caste Hindu majority, was annexed to India, while East Bengal, characterized by a Muslim and underprivileged-caste Hindu majority, became a part of Pakistan [155]. This displaced millions of Bengalis as refugees across the India-Pakistan border [125] and marginalized the Bengali people under Pakistani subjugation [6] as the long geographic distance and myriad cultural differences between Pakistan and East Bengal were overlooked in this colonially imposed idea of nationality. Eventually, in 1971, East Bengal gained independence from Pakistan and formed Bangladesh based on people’s ethnolinguistic identity.

Overall, among myriad dimensions of marginalization, colonization crucially impacted the expression of social identities in the context of Bengali communities by impacting their perception of gender roles of men and women, the religious division of Hindus and Muslims, and the socio-economic structures and political consciousness culminating in Bengali communities assuming different nationalities (e.g., Bangladeshi and Indian).

This coloniality has continued to shape people’s everyday experiences and, on a deeper level, mediate how they express their social identities. One can express one’s social identity both explicitly and implicitly. Explicit expressions of identity refer to deliberate and direct ways individuals communicate and assert their affiliations, characteristics, and beliefs. For example, mentioning one’s nationality and political views or openly discussing one’s religious beliefs are examples of explicit expressions of identity [171]. Meanwhile, implicit expressions of identity include subtle and indirect ways in which identity is communicated or inferred from a person’s actions, behaviors, choices, and interactions [175] and are bound up with cultural norms, societal expectations, and institutionalized practices [35, 88]. For example, how one speaks, the words they use, or their hobbies can implicitly give insights about one’s identity. While people’s social identities can be communicated implicitly through different speech acts and non-verbal acts, this paper focuses on linguistic expressions of various identity categories through writing. Particularly, we considered how different gender, religion, and nationality-based identities are expressed explicitly and implicitly in Bengali texts.

Cultural-linguistic scholars have detailed how languages are often standardized differently in different countries (e.g., English in England vs. the United States; German in Germany vs. Austria) [32]. These geo-cultural variations, often referred to as dialects, operate as important signs and implicit expressions of cultural identity [70, 86]. In Bengali, the two main dialects are Bangal and Ghoti, which are spoken in East Bengal (Bangladesh) and West Bengal (in India), respectively  [53]. These variations of the Bengali language manifest both phonologically and textually [100, 126] and use different colloquial vocabularies in written texts for the same everyday objects. For example, Bangladeshi and Indian Bengalis respectively use the words "জল" (/zɔl/) and "পানি" (/ˈpɑːniː/) to mean “water." Consistently using vocabulary from either the Bangal or Ghoti dialects can implicitly express a Bengali person’s national identity without any explicit mention. Similarly, Bengali textual communication often implies the gender and religious identities of the people it describes. While in Bengali, unlike many other Indo-European languages, gender does not change the choice of pronouns (as in English) and verbs (as in Hindi and Urdu) [25], culturally, most names and kinship terms are gender-specific with some exceptions [59]. Moreover, commonly used kinship terms, names, and commonly used vocabularies often implicitly indicate one’s membership or being born into either Hindu or Muslim communities [53, 59]. For example, while Bengali Hindus often draw inspiration from Demigods’ names and characters in legends for their personal names and commonly tend to use Bengali words derived from Sanskrit, in Bengali Muslim communities being named after Prophets, Caliphs, and Mughal emperors and the vernacular use of Perso-Arabic words are widely popular [59]. Thus, written Bengali communication can lead to the inference of one’s gender, religion, and nationality-based identities.

As the colonizers invented categorization and classifications by viewing and interpreting cultures, societies, and people from non-Western locations in a stereotyped and exoticized manner [139], hierarchies among these artificial categories have been established and embedded within colonized societies [55, 72]. Broadly, these experiences included everything from colonially shaped racism (a belief in certain racial groups’ inherent superiority or inferiority) to colorism (favoring lighter skin tones over darker ones within a single racial group). With respect to how people express their social identities through written language, the influence and affluence of West Bengal’s upper-caste Hindu landlords and elites, who predominantly spoke the Ghoti dialect, led to the establishment of their dialect as the institutional and “normative" standard for the Bengali language during the introduction of printing presses in the region [39]. In contrast, the Bangal dialect became associated with East Bengal’s agrarian socioeconomic system and refugees due to mass migrations following the colonial partition and a means of Muslim and underprivileged caste Hindus’ social harassment [54, 78]. Through coloniality, these impacts on identity, such as sociolects (dialects of particular social classes [111]) and colonial ontologies and epistemologies–the ways of being and knowing–are embedded within the world structures at regional and global scales and continued across generations through various artifacts, media, and technology [5, 18].

This leads to critical and important questions: Are sociotechnical systems “mindful" of such sociocultural and historical complexities that shape people’s identities? How are identities translated into “something a microchip can understand" [137]?

To better interrogate these questions, we draw on postcolonial computing scholarship. Broadly construed, postcolonial and decolonial scholars have worked to highlight the “colonial impulse" of technology [62, 90]. Dourish and Mainwaring identified notions that undergird both colonial narratives and computing systems, such as belief in universality, reliance on reductive representation, and comparative evaluation of different sociocultural identities [62]. While prior critical HCI scholarship has studied the design and development of ubiquitous computing [62] and computer vision [149] from postcolonial and decolonial perspectives, in this paper, we seek to understand how BSA tools reanimate social biases based on identities in previously colonized communities.

Computing systems construct people’s algorithmic identities–how digital technologies and algorithms construct and represent individuals’ identities through data-driven processes [42]. These data can be from historical archives, near-real-time sources, or both. Since historical archives often reflect colonial ontologies and hierarchies [172], when used to inform computing systems like algorithms, they can inadvertently perpetuate these colonial values [34]. Moreover, their under-representation or misrepresentation of certain identities can reinforce the existing colonial power structures. Even near-real-time data being interpreted through colonial taxonomies assign people to hierarchized categories across race, gender, or nationality [42]. Moreover, power imbalances emerge among groups of users, big tech companies, and different countries due to the substantial financial resources required for developing, deploying, and maintaining large-scale technological infrastructures and the regulatory frameworks and capacity to influence policy decisions. This can create exclusionary digital spaces that prioritize certain identities over others, perpetuating historical injustices. Therefore, scholars have described sociotechnical systems’ approaches to conceptualizing people without considering social contexts as “colonial impulses" [62].

Sociotechnical systems, broadly construed, reanimate and reinforce existing societal power structures; they are likely to discriminate [21, 138]. Scholars have explored how systems like facial recognition, predictive policing, hiring algorithms, facial beauty apps, recommendation systems, and standardized tests exhibit biases [21, 31, 42]. More specific to AI, beyond the biases that originate from individuals having significant input into the design of an AI system, biases also manifest from social institutions, practices, and values [67]. Bias could also arise from technical constraints (e.g., while making qualitative human constructs quantitatively amenable to computers [62]) as well as based on the context of use (e.g., users having different values from the system or dataset developers [67, 156]). AI systems’ reductionist representations rely on codified stereotypes [21] and induce essentialization of certain identities [82], which Scheuerman et al. in the case of computer vision (CV) characterized as an “extended colonial project" [149]. Researchers in CHI and adjacent fields have recently been studying the biases and fairness of systems reliant on ML, NLP, and CV [27, 113, 151]. Many of them proposed and used “algorithmic audit" as a way to evaluate sociotechnical systems for fairness and detect their discrimination and biases [114].

Audits have become a popular approach to conducting randomized controlled experiments by probing a system by providing it with one or more inputs while changing some attributes of that input (e.g., race, gender) in environments different from the system’s development [114]. For example, Bertrand and Mullainathan’s classic audit study [22] tested for racial discrimination in hiring, specifically in reviewing resumes, created and submitted fictitious resumes with similar qualifications bearing white-sounding or Black-sounding names to job postings in many companies and industries and quantified the frequency at which those imaginary job seekers received interview callback responses. They found white-sounding names to receive 50% more callbacks than Black-sounding names, indicating widespread racial bias in the labor market. Algorithm audits particularly examine algorithmic systems and content [140].

While some studies have delved into codes of open-source algorithms to study structural biases [95], given that many algorithms we use are proprietary and like “black boxes", algorithmic audits seek to decipher algorithms by interpreting output while varying inputs [58, 114]. This differs from other tests popularly used in computing and HCI literature. For example, unlike other common experiments in HCI, such as A/B tests in which the subject of the study is the users, in algorithmic audit, the subject of study is the system itself [114]. Algorithm audits are also different from other types of system testing due to their broader scope, resulting in systematic evaluations rather than binary pass/fail conclusions for individual test cases. Moreover, audits are purposefully intended to be external evaluations based only on outputs, without insider knowledge of the system or algorithm being studied [114]. Traditionally, querying an algorithm with a wide range of inputs and statistically comparing the corresponding results has been one of the most effective ways for algorithmic audits [114, 169]. Seminal work by Sweeney [169, 170] queried the Google Search algorithm with Black-identifying and white-identifying names from two prior studies [22, 76]. She found that names associated with certain racial or ethnic groups can lead to differential and discriminatory ad delivery, and the difference in ads having negative sentiment for the Black and white name-bearing groups was statistically significant [169].

Using a similar approach to Sweeney’s, Kiritchenko and Mohammad examined gender and race biases in two hundred sentiment analysis systems based on common African American and European American female and male names and found racial biases to be more prevalent than gender biases [101]. Though the perturbation sensitivity analysis framework [129] detects such unintended biases related to names, it relies on associating social bias with proper names and does not provide guidelines in the case of collectives. Extending studies [101, 169, 170] that relied on common names in different demographic groups as implicit indications of identity, Diaz and colleagues studied both implicit and explicit biases based on age. They examined outputs of 15 popular sentiment analysis tools in case of explicit encodings of age by using sentences containing words like “young" and “old"  [58]. While these studies focused on biases between traditionally dominant and marginalized social groups, CHI scholars have also emphasized the importance of studying power dynamics and harms within a marginalized community [182].

Especially in NLP, while a huge disparity exists in available resources for different languages [96], being mindful of bias, stereotypes, and variations within a marginalized and low-resource language (e.g., Bengali) is important [86]. While recent scholarships in NLP have started proposing gender, regional, religion, and caste-based stereotypical biases in Indian languages more broadly [20, 23, 173], Das and Mukherjee highlighting the centrality of gender, religion, national origin, and politics, urged for future research into biases related to specific target communities within the Bengalis [56]. Useful for such exploration, Das and colleagues prepared a cultural bias evaluation dataset considering both explicit and implicit encodings of different identities within the Bengali communities based on common female and male names in different religion-based communities, colloquial vocabularies in different national dialects, and explicit mentions of various intra-community groups [53]. Moreover, our work builds on Das, Østerlund and Semaan’s work [54] who, through a trace ethnographic study, found that various downstream effects of language-based automation for content moderation were likely shaping people’s everyday user experiences on the online platform BnQuora². In highlighting BnQuora’s algorithmic coloniality, they were unable to determine the extent to which the tools used to inform content moderation, such as sentiment analysis tools, were complicit in this experience. As such, we build on this work through an algorithmic audit to more systematically and broadly understand the extent to which these tools are shaped by and through a colonial impulse.

Researchers have used algorithmic audits in various domains, such as housing [65], hiring [40], healthcare [122], sharing economy [41, 64], gig work [84], music platforms [69], information [97], and products [83], and so on, where their underlying components like recommendation systems [17], search algorithms [135], CV-based processes (e.g., generative art [165], image captioning [187], facial recognition [34]), and language technologies (e.g., sentiment analysis [101], hate-speech detector [141], machine translation [142], text generation [71]) are often scrutinized. The social identity and demographic dimensions that researchers have previously include gender [89], race [141], nationality [178], religion [24], caste [15], age [58], occupation [174], disability [180], and political affiliations [2]. Algorithmic audits have also been used to scrutinize categories produced by computational assessments (e.g., risk) [144, 146]. Often, NLP systems are used in producing such computational categories and concepts that are then used for decision-making (e.g., automated content moderation, public sector [146, 176]). In this paper, we are critiquing that process itself.

Like CHI, where an overwhelming 73% of research is based on Western participant samples representing less than 12% of the world’s population [106], critical algorithmic studies focus on predominantly Western contexts, communities, and languages [61]. Algorithmically auditing Bengali sentiment analysis tools (BSA) for identity-based biases, this paper contributes to HCI, NLP, and fairness, accountability, and transparency (FAccT) literature by bringing a large ethnolinguistic yet under-represented communities’ experience with language technologies forth from a fairness perspective. Moreover, we reflect on our findings while critically engaging with these communities’ sociohistoric and cultural contexts.

Prior HCI and social computing scholarship have highlighted how researchers’ positionality impacts researchers’ motivations and perspectives, especially while studying under-represented communities [13, 105, 153]. Recent work in computational linguistics has also echoed the importance of local communities’ agency in NLP research, especially for decolonizing language technologies [26, 53]. The first two authors were born and brought up in the Bangladeshi and Indian Bengali communities, respectively, while the third author is a White American, and the anchor author is an Iraqi-American who is a member of an Indigenous group from Iraq. All are cis-male researchers affiliated with North American universities. We come from interdisciplinary backgrounds, including computer science, economics, information science, psychology, and statistics. Our decision to examine identity-based biases in non-English language technology stems from our interests and concentration in critical HCI, marginalized groups, and ethnolinguistic communities. Our positionalities, backgrounds, and research experience put us in the capacity to prioritize the local communities’ perspectives in the paper on language technologies in Bengali.

We analyzed the available BSA tools in the Python programming language, which is widely used in data science and machine learning communities. Exploring multiple sentiment analysis tools can minimize the likelihood of reporting idiosyncratic findings from a single tool. However, because fewer sentiment analysis tools are available in Bengali than in English, we curated BSA tools from GitHub in addition to PyPI. We searched on these two platforms on November 3, 2022, using the phrases “Bengali sentiment analysis" and “Bangla sentiment analysis." We retrieved two tools from PyPI and 31 tools from GitHub. We also closely read the description and documentation of each package and repository. We included a tool/repository in our study if the tool was operational for basic sentiment analysis tasks (e.g., outputting a sentiment score or classification for a Bengali sentence) or if the repository contained an already trained model or sufficient documentation, code, and data to reproduce the tools. If a repository contained multiple independent tools (e.g., naïve Bayes or dictionary-based classification), we included the one that the developers found to have the highest accuracy in our study. Table 1 shows the BSA tools (n=13) included and examined in our study, how those were implemented, and the sources of data used to train the models. Since all of our examined BSA tools are based on various machine learning and deep learning models, we use the terms “tool" and “model" interchangeably. Studying these multiple BSA tools will allow us to compare common implementation techniques and data sources that may influence bias. We also collected metadata about these tools, including developers’ names, contact information, affiliations, and countries, by looking up their PyPI and GitHub profiles, README files, documentation websites, and published research papers. With approval from the institutional review board (IRB) at our university, we contacted the developers through email and LinkedIn. Seven tools’ developers self-identified their demographics, which we also mention in Table 1. To protect the privacy of these developers, we de-identified the tools by assigning an ID to each tool or repository instead of using its URL for identification. Inspired by ethics literature on using internet resources in research that provide methods for obfuscating people’s online identities to protect their anonymity [33, 73], we further obfuscated the tools by describing their implementation and data at a higher level (e.g., describing linear regression as a parametric ML model or generic references like “social media" instead of specific platform names as the sources of data). We did not wish to provide any information that would allow anyone to trace back to and identify these developers.

In this paper, to evaluate whether and how different BSA tools demonstrate biases based on Bengali identities across the three dimensions of gender, religion, and nationality, we used the Bengali Identity Bias Evaluation Dataset (BIBED) prepared by Das et al. [53]. To propose a method for developing datasets to evaluate cultural biases, they chose the context of the Bengali language and people due to their demographic distribution across major religions (e.g., Hinduism and Islam), nationalities (e.g., Bangladeshi and Indian), and diverse linguistic practices. Whereas Das and colleagues were solely focused on creating the dataset [53], in this paper, we use their dataset to audit available sentiment analysis tools in the Bengali language.

BIBED comprises a wide array of sentences collected from Wikipedia, Banglapedia⁴, Bengali classic literature, Bangladesh law documents, and the Human Rights Watch portal or constructed from template sentences that explicitly and implicitly express gender, religion, and nationality-based Bengali identities. Explicit expressions involve direct references to a particular nationality, religion, or gender in a sentence. Implicit expressions, on the other hand, rely on common names, kinship terms, or colloquial vocabularies predominantly used within specific communities to infer nationality, religion, or gender [53]. The dataset contains 25,396 pairs of sentences explicitly representing gender-based identities (female-male), 11,724 pairs explicitly representing religion-based identities (Hindu-Muslim), and 13,528 pairs explicitly representing nationality-based identities (Bangladeshi-Indian). In each sentence pair, two sentences are identical, other than the identities expressed by each sentence. This dataset also includes unpaired sentences implicitly representing gender and religious identities using common names and kinship terms, with 1,200 sentences for each category. Additionally, there are 8,834 pairs of sentences that implicitly represent Bangladeshi and Indian nationalities based on colloquial vocabularies of Bangladeshi Bengali and Indian Bengali dialects. We used all the sentences in BIBED to audit BSA tools’ biases across different dimensions.

We designed our experiment as an algorithmic audit [114, 140]. In our experiment, we queried the curated BSA tools, listed in Table 1, with sentences explicitly and implicitly representing different Bengali identity categories across gender, religion, and nationality dimensions. Different sentiment analysis tools process their outputs differently for a given input. Whereas some tools choose the most likely sentiment from a binary (positive-negative) or a trinary (positive-neutral-negative) classification, most tools often output a sentiment score. Again, while some tools use a scale of [0, 1], some tools follow a scale of [-1, +1] for this sentiment score. To standardize and facilitate the comparison of the outputs of all BSA tools, we normalized their output sentiment scores or polarities within a range between 0 and 1. A higher score indicates a more positive sentiment for a given input sentence. For tools that provided sentiment labels without specific scores, we made slight adjustments (e.g., returning a neural network-based classifier’s input to its final softmax layer as the sentiment score) within their codes to ensure that they also produced sentiment scores falling within the 0 to 1 range. Such conversion of categorical outputs into a probability-based metric associated with the positive class for quantifying bias is common in NLP literature [51]. This normalization process allowed us to effectively assess and compare results from various BSA tools. The null hypotheses for our RQs are as follows:

We conducted inferential statistical tests to determine whether we should reject or retain these null hypotheses. In the next section, we will explain our rationale for selecting the test directions (two-tailed, left-tailed, and right-tailed) and formulate the alternative hypotheses. Unlike prior work by Kiritchenko and Mohammad that used tests on the assumption of normality [101], for all research questions, we decided on either the parametric or the non-parametric alternative of a test upon checking the normality of the sentiment scores’ distributions using the Shapiro-Wilk test [157]. Following the recommendation from a previous study in computational linguistics [163], we opted to utilize a significance threshold, α = 0.0025. In addition to computing the test statistics and comparing p-values at the significance level α, we also evaluated the tests’ power–the likelihood of a significance test detecting an effect when there actually is one [45]. In doing so, we repeated each test ten times using one-tenth of the complete dataset per iteration and checked whether that test passed the recommended threshold of 0.8 [44]. Another important metric in statistical comparison is the effect size–a standardized measure indicating the magnitude of the relationship or difference between two variables, especially when they are measured in different units [44]. However, since we have already normalized the sentiment scores from all BSA tools to a common scale of 0 to 1, we can directly interpret the differences between the two columns without calculating effect size separately [50]. The experiment and statistical analyses were conducted using Python, with a fixed seed value, where applicable (e.g., sampling), for replicability and consistency of our results.

Scholars have emphasized the importance of responsible research in big data and adjacent fields (e.g., NLP) by urging researchers to consider the environmental impacts of their studies [48, 166, 188]. In this work, we used four pre-trained models (T1, T5, T7, and T11) and trained other models ourselves. We trained eight models (T2, T3, T4, T6, T9, T10, T12, and T13) on an M2 MacBook Air 2022 and one (T8) using NVIDIA Tesla-T4 on Google Colab. Considering these devices’ power consumption under high loads⁵, and the facts that Google’s typical data center’s carbon footprint is 0.08kgCO₂/kWh [127], global average carbon intensity for electricity is 0.475kgCO₂/kWh [1], and 38.2% of our local electricity comes from renewable energy [43], our study released approximately 0.57 kg of carbon into the environment for training AI models, which is negligible compared to the most resource-intensive models [166]. Almost half of our studied tools were statistical machine learning models, and even those utilizing deep learning relied on small networks and datasets, contributing to a minimal environmental impact. As a gesture to offset carbon pollution, we donated to the US Forest Service’s Plant-a-Tree program.

While using an existing dataset (BIBED) to evaluate different BSA tools, our study adopted its binary notion of Bengali gender, religion, and nationality-based identities and, consequently, overlooks various Bengali identities like non-heteronormative genders (e.g., হিজড়া: /ɦidʒɽa/ that loosely represents queer and transgender people), religious minorities (e.g., Buddhists, Christians), and diaspora nationalities. While adhering to this binary notion of identity streamlined our experiment setup, this limitation of our paper is indicative of the field’s limitations, in general–to be restricted to using artifacts produced in colonial ontologies as research materials. Since this study relies on quantitative methods, it is limited in its capacity, and in our future work, we will draw on interviews and ethnography to continue to critically study how BSA tools process the expressions of minority gender, religious, or national identities. Moreover, in this study, we examine BSA tools’ bias in relation to Bengali categorical identities within a single dimension, focusing on gender, religion, and nationality individually. Future work should examine how these tools show biases based on intersectional identities in Bengali communities. While in this work, we studied how different BSA tools calculate sentiment scores for different Bengali identities, inspired by prior works on the politics of datasets [148], in our future work, we will explore how BSA datasets impact the construction and performance of BSA tools with greater details and nuances. Future work should also explore how sociotechnical systems like sentiment analysis tools extend colonial influences in other identity dimensions (e.g., caste, sexuality) in Bengali communities. Lastly, it is important to highlight how, in many cases, it can be difficult to explore the nuances and fluidity of people’s gender and sexual expression as the tools and datasets often represent data in binary ways, or nuance can become lost when explored as aggregated data.

We scrutinized BSA tools’ assumption of universality, i.e., if tools generally agree on the subjectivity and sentiment of sentences, especially when conveying various identities. We also investigate how BSA tools relying on reductionist representations act with various ways of identity expression.

We examined if BSA tools reanimate colonial hierarchies among identities by privileging a gender, religion, or regional group over others. We also investigated how the politics of design reinforce such values (e.g., who develops BSA tools and how their backgrounds permeate these tools.)

Comparing average sentiment scores from different Bengali sentiment analysis (BSA) tools in RQ1.a, we found that for the same lexical content, sentence structure, and identity category, BSA tools’ outputs are significantly different from each other. While several BSA tools using the same dataset (e.g., YouTube Bengali drama reviews [147]) and similar models (e.g., logistic regression, RNN model), most BSA tool pairs resulting in different outputs for a particular identity category imply that various combinations of dataset and model architectures lead the tools to respond differently for identical sentences expressing a particular identity. With an assumption of universality–generalizing perception of sentiment across cultures and populations, sentiment analysis is used in various tasks, such as in gauging public sentiment toward political figures and issues [16, 183], social issues and contemporary events [74, 186], and gathering insights from textual data in customer service [80, 104], healthcare [80], and public sectors [145, 181], amongst other applications. Our finding from RQ1.a implies that the extracted insights about subjectivity and polarity from textual data can vary significantly depending on which BSA tools are used.

Reading through the documentation, README files and associated research articles of our examined BSA tools indicated that none of these included post-development user testing and checking for identity-related biases. This leaves room for inconsistencies and discrepancies among sentiment analysis tools to go unscrutinized and unattended. Moreover, the lack of participation of users from different demographic groups within Bengali communities leads to disparities in accessing and using Bengali language technologies. Returning to our discussions on cultural hegemony in section 2, such a digital divide among developers and users and invisible politics of code institutionalize a specific group’s power and control through technological artifacts and, consequently, their perceptions and beliefs shape technology used within a larger community. By convincing others that their values and interests align with the overall community’s perspectives and benefits, that specific group achieves technological hegemony. To resist certain groups systematically benefiting more from a sociotechnical system than other communities and systematically having influence over data-centric infrastructures, following prior scholarship [7, 10], we urge collaboration among stakeholders to ensure that their developed sentiment analysis tools’ responses to Bengali sentences are aligned with the perspectives of the community and that they are not prejudiced against any particular identity or group of people.

To answer RQ1.b, we examined how different BSA tools respond to different identity categories, expressed explicitly (e.g., through direct mention) and implicitly (e.g., through colloquial vocabularies, community norms around names and kinship). Similar to our examination of varied Bengali dialects in Bangladesh and India, other major languages have different dialects that are sociohistorically and culturally connected with particular groups within the broader linguistic communities (e.g., Southern and Coastal accents of American English, Quebec accent of French). For example, due to the refugee crises created by the postcolonial partition in Bengal, Bangladeshi (then East Bengal) dialects were associated with refugees in India, and speakers of this dialect are often subject to contempt both online and offline [36, 37, 54]. According to identity scholars, identity is constructed and learned through everyday speech acts and non-verbal activities in different social settings and are thus modeled after normative cultural and societal logics [35]. Though researchers have qualitatively studied how sociotechnical platforms marginalize people based on their performative identity [54, 117, 150], only a few works quantitatively studied how computing systems codify the performativity–the expression of identity through repetition of norms [35] (e.g., colloquial verbal and speech acts) of various communities and groups [58, 141].

As parochial and stereotypical representations influence the development of datasets and tools, sentiment analysis and NLP tools broadly can inflict representational harm by conflating particular identities into one (e.g., viewing all Indic languages as the same or limiting a linguistic identity by nation states⁷). While researchers found evidence of accent gaps and racial disparity in speech recognition and language identifiers (e.g., not recognizing Southern American English) [28, 85], our study highlighted how sentiment analysis tools codify different country-based communities’ preference of vocabularies as implicit expressions of identities and exhibit biases based on those. Prior CHI literature proposed using readily available sentiment analysis (e.g., VADER) to gather insights from textual data in algorithmic decision-making [136, 145]. Based on our finding that sentiment analysis tools codify the internal practices of different religion and nationality-based communities, we need to ask how these community practices and various societal biases and prejudices regarding those practices being embedded within sentiment analysis tools would impact algorithmic decision-making. We explore this issue further through the application of sentiment analysis tools in the context of content moderation in the following section.

In RQ2.a, we found that most sentiment analysis tools available in the Bengali language are biased toward a particular category in cases of identity dimensions of gender, religion, and nationality. For sentences with similar structure and word content, most BSA tools (77%) deemed Bangladeshi identity to be more positive than Indian identity, exhibiting a nationality-based bias. We found BSA tools exhibiting such favoritism toward female (38%), male (30%), Hindu (30%), Muslim (38%), and Indian (15%) identities. Such preference toward a particular religious or national community’s direct mention or linguistic practices resembles [54]’s finding of biases in content moderation. For some BSA tools, we could not find evidence of those consistently assigning significantly different sentiment scores to different identity categories under a single dimension (e.g., T1 for gender, T5 for religion, and T10 for nationality). While those tools did not show bias in a particular dimension, our analysis could not identify a BSA tool that maintains such impartiality across all three dimensions of gender, religion, and nationality. Using biased language technologies like a sentiment analysis tool can have downstream effects. For example, sentiment analysis is also a ubiquitously used component in automated content moderation systems [87, 161, 167, 176]. Scholarship in social computing and communications have studied the construction of automated content moderation [38, 92] and users’ perception of those systems [93, 154]. Though, due to algorithms’ complexities and common failure to understand the contexts of human languages, automated content moderation’s legitimacy is questioned [124], users perceive automated moderation to be more impartial with human oversight [123]. Related to user personality and social aspects [134], in some cases, researchers have found that “users trust AI as much as humans for flagging problematic content" [116, 168].

Given how the transnational and religiously diverse Bengali communities’ colonial past continues the distrust and division across religions and national borders and impacts their experience with platform governance and perception of biased content moderation, especially the anonymous human moderators [54], we ask if automated content moderation is used instead of human moderation, how would that impact user interaction and experience for diverse Bengali communities? This question stems from considering “automated" and “human" as two ends of a spectrum of moderation style [94]. If the sentiment analysis component within that automated moderation system is biased, as we found in our study, it can misinterpret non-normative opinions as negative and trigger automated content moderation systems to remove the content from the platforms. Thus, users, especially the ones from marginalized and minority communities, can fear being censored for expressing their perspectives. Rather than complementing human moderators’ efforts in managing large online communities, automated moderation can be employed as a pretext to justify the marginalization of diverse voices. Altogether, biased BSA tools being used in automated moderation can deter inclusive and in-depth discussions, prompt users to disengage or become inactive, and eventually shape a homogenized identity and reflect existing colonial divisions and structures in Bengali societies–much like the outcomes of biased human moderators [54].

Returning to RQ2.b, though we did not find any relationship between the BSA tools’ direction of biases and the demographics of those tools’ developers, we cannot overlook the homogeneity of developers’ identities. Since all the BSA tools we audited were developed by Bengali developers and not some Western entities, do we need to ask “who designs?" Does postcolonial computing’s concern about computing systems’ similarities with colonial practices apply here? Prior CHI research found that while transgenerational colonial values (e.g., collective identity posited on difference) shape Bengali users’ interaction with and through computing systems, collaborative discourses resist such views [55]. However, earlier in the paper, in Table 1, we saw that most BSA tools on PyPI and GitHub are developed by solo developers or teams of a few coders with little diversity–most tools being developed by individuals who identify as male, Muslim, and Bangladeshi. Similar to colonial Bengal, where certain exclusionary social identities (e.g., babu: educated Bengali men often based in Kolkata, West Bengal) emerged as accepted changes in Bengali identity and subjectivity [63], despite the Bengali language being spoken natively by diverse religious and national communities, we found certain isolation and lack of collaboration to exist among developers of diverse backgrounds. For example, though BSA tool T4 had both female and male developers, similar collaboration did not occur across various religion and nationality-based identities in any BSA tool.

Does the colonial past of the subcontinent and the Bengali people have anything to do with today’s lack of collaboration in the developing sociotechnical systems in the Bengali context? Prior work has highlighted that colonial rule fragmented the Bengali people’s imagination of communities, deepened the communal distrust among Hindus and Muslims, and increased the communication gaps among Bengalis in Bangladesh and India [39, 54]. For example, whereas Indian Bengalis’ nationality is shaped by linguistically diverse Indian identity [158], Bangladesh defines its people’s concept of nationalism as being derived from Bengali language and culture [19]. Therefore, language’s role in shaping Bengali people’s cultural identity and imagination of communities varies in Bangladesh and India. This difference translates to Bengali researchers’ participation in computational linguistics research in their local language. For example, developers of all but one BSA tool self-identified as being from Bangladesh. Beyond our study, most leading Bengali NLP research endeavors, such as learning and research groups⁸ and workshops⁹, are supported and advanced by Bangladeshi communities and government. Though Indian researchers also regularly contribute to Bengali NLP, it is often done through the framing of NLP for Indic languages [12] and lacks the concentrated attention that the Bangladeshi NLP community puts in the Bengali language. As NLP tools in Bengali are predominantly developed by Bangladeshi Bengalis, those technologies, reflecting Bangladeshi values, norms, and prejudices, can become biased. Actively collaborating among individuals from different religions within the Bengali communities and institutions across geographic boundaries can contribute to mitigating such digital divisions.