Political Polarization On Twitter: M. D. Conover, J. Ratkiewicz, M. Francisco, B. Gonc Alves, A. Flammini, F. Menczer
Political Polarization On Twitter: M. D. Conover, J. Ratkiewicz, M. Francisco, B. Gonc Alves, A. Flammini, F. Menczer
Political Polarization On Twitter: M. D. Conover, J. Ratkiewicz, M. Francisco, B. Gonc Alves, A. Flammini, F. Menczer
101
tive Twitter users. This structural difference is of particular Clusters A
Cluster B
importance with respect to political communication, as we Different clusters
P(cos(a, b))
now have statistical evidence to suggest that mentions and Retweet Mention 100
replies may serve as a conduit through which users are ex- A↔A 0.31 0.31
B↔B 0.20 0.22
posed to information and opinions they might not choose in 10-1
A↔B 0.13 0.26
advance. Despite this promising finding, the work of Yardi
and boyd (2010) suggests that cross-ideological interactions 0 0.2 0.4 0.6 0.8 1
cos(a, b)
may reinforce pre-existing in-group/out-group identities, ex-
acerbating the problem of political polarization. Figure 2: Cosine similarities among user profiles. The table
on the left shows the average similarities in the retweet and
3.2 Content Homogeneity mention networks for pairs of users both in cluster A, both in
The clustering described above was based only on the net- cluster B, and for users in different clusters. All differences
work properties of the retweet and mention graphs. An inter- are significant at the 95% confidence level. The plot on the
esting question, therefore, is whether it has any significance right displays the actual distributions of cosine similarities
in terms of the actual content of the discussions involved. for the retweet network.
To address this issue we associate each user with a profile
vector containing all the hashtags in her tweets, weighted by
their frequencies. We can then compute the cosine similari- 3.3 Political Polarization
ties between each pair of user profiles, separately for users Given the communities of the retweet network identified in
in the same cluster and users in different clusters. Figure 2 § 3.1, their content homogeneity uncovered in § 3.2, and
shows that in the mention network, users placed in the same the findings of previous studies, it is natural to investigate
cluster are not likely to be much more similar to each other whether the clusters in the retweet network correspond to
than users in different clusters. On the other hand, in the groups of users of similar political alignment.
retweet network, users in cluster A are more likely to have To accomplish this in a systematic, reproducible way we
very similar profiles than users in cluster B, and users in dif- used a set of techniques from the social sciences known
ferent clusters are the least similar to each other. As a result as qualitative content analysis (Krippendorff 2004; Kolbe
the average similarity within retweet clusters is higher than 1991). Similar to assigning class labels to training data in su-
across clusters. Further, we note that in both mention and pervised machine learning, content analysis defines a set of
retweet networks, one of the clusters is more cohesive than practices that enable social scientists to define reproducible
the other — meaning the tag usage within one community is categories for qualitative features of text. Next we outline
more homogeneous. our annotation categories, and then explain the procedures
used to establish the rigor of these category definitions.
Our coding goals were simple: for a given user we wanted Table 4: Partisan composition and size of network clusters
to identify whether his tweets express a ‘left’ or ‘right’ as determined by manual inspection of 1,000 random user
political identity, or if his identity is ‘undecidable.’ The profiles.
groups primarily associated with a ‘left’ political identity are Network Clust. Left Right Undec. Nodes
democrats and progressives; those primarily associated with A 1.19% 93.4% 5.36% 7,115
a ‘right’ political identity are republicans, conservatives, lib- Retweet
B 80.1% 8.71% 11.1% 11,355
ertarians and the Tea Party. A user coded as ‘undecidable’ A 39.5% 52.2% 8.18% 7,021
may be taking part in a political dialogue, but from the con- Mention
B 9.52% 85.7% 4.76% 154
tent of her tweets it is difficult to make a clear determination
about political alignment. Irrelevant non-English and spam
accounts constitute less than 3% of the total corpus and were
excluded from this analysis. We experimented with more de- if we assign a political identity only to users for whom both
tailed categorization rubrics but the simple definitions de- annotators agree, we report unambiguous political valences
scribed above yielded the highest inter-annotator agreement for more than 75% of users.
in early trials of the coding process. Using these annotations we can infer the expected politi-
Using this coding scheme one author first annotated 1,000 cal makeup of the network communities identified in § 3.1.
random users who appeared in both the retweet and mention As shown in Table 4, the network of political retweets ex-
networks. Annotations were determined solely on the basis hibits a highly partisan community structure with two ho-
of the tweets present in the six week sample. In line with mogenous clusters of users who tend to share the same po-
the standards of the field, we had a non-author judge with a litical identity. Surprisingly, the mention network does not
broad knowledge of politics annotate 200 random users from exhibit a clear partisan community structure. Instead we find
the set of 1,000 to establish the reproducibility of this anno- that it is dominated by a politically heterogeneous cluster ac-
tation scheme. The judge was provided a brief overview of counting for more than 97% of the users, suggests that po-
the study and introduced to the coding guidelines described litically active Twitter users may be exposed to views with
above, but did not have any other interaction with the authors which they do not agree in the form of cross-ideological
during the coding process. mentions.
The statistic typically used in the social sciences to mea- Increasing the number of target communities in the men-
sure the extent to which a coders’ annotations agree with an tion network does not reveal a more fine-grained ideological
objective judge is Cohen’s Kappa, defined as structure, but instead results in smaller yet politically hetero-
geneous clusters. Similarly, the retweet network communi-
P (α) − P () ties are maximally homogenous in the case of two clusters.
κ= (2)
1 − P ()
where P (α) is the observed rate of agreement between an- 4 Interaction Analysis
notators, and P () is the expected rate of random agreement The strong segregation evident in the retweet network and
given the relative frequency of each class label (Krippen- the fact that the two clusters correspond to political ideolo-
dorff 2004; Kolbe 1991). For agreement between the ‘left’ gies suggest that, when engaging in political discourse, users
and ‘right’ categories we report κ = 0.80 and κ = 0.82 often retweet just other users with whom they agree politi-
respectively, both of which fall in the “nearly perfect agree- cally. The dominance of the mention network by a single
ment” range (Landis and Koch 1977). For the undecidable heterogeneous cluster of users, however, suggests that indi-
category we found “fair to moderate” agreement (κ = 0.42), viduals of different political alignments may interact with
indicating that there are users for whom a political iden- one another much more frequently using mentions. Let us
tity might be discernible in the context of specific domain test these conjectures, and propose an explanation based on
knowledge. To address this issue of context-sensitive ambi- selective hashtag use by politically motivated individuals.
guity we had a second author also annotate the entire set of
1,000 users. This allowed us to assign a label to a user when
either author was able to determine a political alignment, re- 4.1 Cross-Ideological Interactions
solving ambiguity in 15.4% of users. To investigate cross-ideological mentions, we compare the
For completeness we also report binomial p-values for ob- observed number of links between manually-annotated
served agreement, treating annotation pairs as observations users with the value we would expect in a graph where users
from a series of Bernoulli trials. Similar to the Kappa statis- connect to one another without any knowledge of political
tic results, inter-annotator agreement for the ‘left’ and ‘right’ alignment. The intuition for the expected number of links is
categories is very high (p < 10−12 ). Agreement on the ‘un- as follows: for a set of users with k directed edges among
decidable’ category is again lower (p = 0.18). them, we preserve the source of each edge and assign the
Based on this analysis it is clear that a majority of polit- target vertex to a random user in the graph, simulating a sce-
ically active users on Twitter express a political identity in nario in which users connected irrespective of political ide-
their tweets. Both annotators were unable to determine a po- ology. For example, if there are a total of kR links originat-
litical identity in only 8% of users. A more conservative ap- ing from right-leaning users, and the numbers of left-leaning
proach to label assignment does not change this story much; and right-leaning users are UL and UR respectively, then the
Table 5: Ratios between observed and expected number of Table 6: The ten most popular hashtags produced by left- and
links between users of different political alignments in the right-leaning users in the manually annotated set of users,
mention and retweet networks. including frequency of use in the two retweet communities
Mention Retweet and ideological valence.
→ Left → Right → Left → Right Rank Hashtag Left Right Valence
Left 1.23 0.68 1.70 0.05 1 #tcot 2,949 13,574 0.384
Right 0.77 1.31 0.03 2.32 2 #p2 6,269 3,153 -0.605
3 #teaparty 1,261 5,368 0.350
4 #tlot 725 2,156 0.184
expected number of edges going from right-leaning to left- 5 #gop 736 1,951 0.128
leaning users is given by: 6 #sgp 226 2,563 0.694
7 #ocra 434 1,649 0.323
UL
E[R → L] = kR · . (3) 8 #dems 953 194 -0.818
UL + UR 9 #twisters 41 990 0.843
We compute the other expected numbers of edges (R → R, 10 #palin 200 838 0.343
L → R, L → L) in the same way. Total 26,341 53,880
In Table 5 we report the ratio between the observed and
expected numbers of links between users of each political
alignment. We see that for both means of communication,
These tweets were selected from the first page of the re-
users are more likely to engage people with whom they
altime search results for the #tcot (“Top Conservatives on
agree. This effect, however, is far less pronounced in the
Twitter”) and #p2 hashtags, respectively, and messages in
mention network, where we observe significant amounts of
this style make up a substantial portion of the results.
cross-ideological interaction.
This behavior does not go unnoticed by users, as under-
4.2 Content Injection scored by the emergence of the left-leaning hashtag #p21.
According to a crowdsourced hashtag definition site (www.
Any Twitter user can select arbitrary hashtags to annotate tagdef.com), #p21 is a hashtag for “Progressives sans
his or her tweets. We observe that users frequently produce RWNJs” and “Political progressives w/o all the RWNJ spam
tweets containing hashtags that target multiple politically that #p2 has,” where RWNJ is an acronym for “Right Wing
opposed audiences, and we propose that this phenomenon NutJob.” This tag appears to have emerged in response to
may be responsible in part for the network structures de- the efforts by right-leaning users to inject messages into the
scribed in this study. high-profile #p2 content stream, and ostensibly serves as a
As a thought experiment, consider an individual who place where progressives can once again be exposed only to
prefers to read tweets produced by users from the political content aligned with their views.
left. This user would frequently see the popular hashtag #p2 We propose that when a user is exposed to ideologically
(“Progressives 2.0”) in the body of tweets produced by other opposed content in this way, she will be unlikely to rebroad-
left-leaning users, as shown in Table 6. However, if this user cast it, but may choose to respond directly to the origina-
clicked on the #p2 hashtag hyperlink in one of these tweets, tor in the form of a mention. Consequently, the network of
or searched for it explicitly, she would be exposed to content retweets would exhibit ideologically segregated community
from users on both sides of the political spectrum. In fact, structure, while the network of mentions would not.
because of the disproportionate number of tweets produced
by left- and right-leaning users, nearly 30% of the tweets 4.3 Political Valence
in the #p2 search feed would originate from right-leaning
users. To explore the content injection phenomenon in more detail
A natural question is why a user would annotate tweets let us introduce the notion of political valence, a measure
with hashtags strongly associated with ideologically that encodes the relative prominence of a tag among left- and
opposed users. One explanation might be that he seeks right-leaning users. Let N (t, L) and N (t, R) be the numbers
to expose those users to information that reinforces his of occurrences of tag t in tweets produced by left- and right-
political views. Consider the following tweets: leaning users, respectively. Then define the valence of t as
N (t, R)/N (R)
User A: Please follow @Username for V (t) = 2 − 1 (4)
[N (t, L)/N (L)] + [N (t, R)/N (R)]
an outstanding progressive voice! #p2 P
#dems #prog #democrats #tcot where N (R) = t N (t, R) is the total number of occur-
rences of all tags in tweets by right-leaning users and N (L)
User B: Couple Aborts Twin Boys For is defined analogously for left-leaning users. The translation
Being Wrong Gender..http://bit.ly/xyz and scaling constants serve to bound the measure between
#tcot #hhrs #christian #tlot #teaparty −1 for a tag only used by the left, and +1 for a tag only used
#sgp #p2 #prolife by the right. Table 7 illustrates the usefulness of this measure
by listing hashtags sampled from valence quintiles ranging
Table 7: Hashtags in tweets by users across the political spectrum, grouped by valence quintiles.
Far Left Moderate Left Center Moderate Right Far Right
#healthcare #aarp #women #democrats #social #rangel #waste #912project #twisters
#judaism #hollywood #citizensunited #seniors #dnc #saveamerica #gop2112 #israel
#2010elections #democratic #budget #political #american #gold #foxnews #mediabias
#capitalism #recession #banksters #energy #goproud #christian #repeal #mexico #constitution
#security #dreamact #sarahpalin #media #nobel #terrorism #gopleader #patriots #rednov
#publicoption #progressives #palin12 #abortion
#topprogs #stopbeck #iraq