Exploring Media Bias and Toxicity in South Asian Political Discourse

Adnan Qayyum, Zafar Gilani, Siddique Latif, Junaid QadirarXiv 2018 · arXiv

TL;DR

This paper presents the first comparative study of political discourse on Twitter in Pakistan and India, quantifying media bias, sentiment, and toxicity. By analyzing official accounts of major political parties and news outlets, the researchers reveal how ruling parties dominate the conversation but also attract the highest levels of toxic criticism, while media houses exhibit clear partisan leanings.

What Problem Does This Paper Address?

While social media’s influence on politics is a hot topic in the West, most research focuses on US-centric "liberal vs. conservative" binaries. This doesn't translate well to the complex, multi-party systems of South Asian nations like Pakistan and India. In these developing economies, where the democratic apparatus is often fragile and the public is highly emotionally charged, social media can disproportionately shape voting behavior and promote intolerance.

The authors set out to fill this gap by answering three questions:

  1. Does news reporting on social media show bias toward specific political entities?
  2. Is toxic or inflammatory speech a common practice in these political spheres?
  3. How similar are the Twitter usage patterns of politicians across these two neighboring rivals?

Methods

The researchers developed a multi-stage pipeline to quantify the "nature" of political discourse:

  • Bias Measurement: They distinguished between coverage bias (the frequency of mentions a party receives) and statement bias (the sentiment—positive, negative, or neutral—associated with those mentions).
  • Sentiment & Toxicity Analysis: They used TextBlob for sentiment scoring and Google’s Perspective API to quantify toxicity (rudeness, disrespect, or hate speech).
  • Topic Modeling: They employed Latent Dirichlet Allocation (LDA) to discover latent themes in the tweets, visualizing them through word clouds.
  • Similarity Analysis: To compare politicians, they extracted 12 profile features (follower counts, tweet frequency, hashtag usage) and used Principal Component Analysis (PCA) to map them into a two-dimensional space.

Datasets

The study analyzed a custom dataset of tweets collected over six months (June to November 2017) using the Stweeler framework.

  • Entities: Four major political parties and four major news channels from both Pakistan and India.
  • Accounts: Official verified accounts of parties, their chairmen, secretaries, and prominent members of parliament.
  • Multilingual Processing: Since many tweets were in Urdu or Hindi, the authors used Google’s language detection and TextBlob’s translation libraries to standardize the corpus into English for analysis.

Key Insights and Ideas

  • The "Ruling Party" Paradox: In both countries, the ruling parties (PML-N in Pakistan and BJP in India) received the most media coverage. However, they also received the most negative and toxic mentions from rival parties and certain media outlets.
  • Media Leanings are Quantifiable: The data confirmed popular perceptions of media bias. For example, in Pakistan, GEO News showed a pro-PML-N/anti-PTI leaning, while ARY News leaned pro-PTI and anti-PML-N.
  • Personality over Policy: Topic modeling revealed that political discourse in South Asia is heavily focused on leadership, party members, and attacking rivals. National policy issues were rarely the primary focus of the conversation.
  • Cross-Border Similarities: Despite the geopolitical rivalry, politicians like Imran Khan (PTI) and Rahul Gandhi (INC) exhibit very similar Twitter usage patterns and profile characteristics, as shown in the PCA clustering.

Key Results

  • Activity Levels: The BJP (India) and PTI (Pakistan) were the most active and influential parties on Twitter within their respective countries.
  • Bias in Coverage: Figure 1 illustrates that almost every Pakistani news channel gave the most positive coverage to the ruling PML-N, while in India, every channel gave the most coverage to the BJP.
Fig. 1: Coverage and sentiments given by news channels to political parties. Where Figure 1(a) shows the coverage and sentiments given by various Pakistani news channels to their national political parties and 1(b) present the coverage and sentiments given by Indian news channels to their national political parties.
  • Toxicity Levels: The study found that Pakistani channels like GEO News spread relatively more toxic content about the opposition (PTI), while Indian channels like Aaj Tak and Zee News disseminated higher toxic content regarding the BJP and INC, respectively.
  • Clustering Politicians: The PCA analysis (Figure 3) showed that most popular politicians cluster together, meaning they use Twitter in similar ways. Outliers like Narendra Modi and Ali Raza Abidi were distinguished primarily by their massive follower bases or extreme tweeting frequencies.
Fig. 3: Visualisation of similarity among various politicians of Pakistan and India using dimensionality reduction of different features using PCA.

Limitations and Future Work

The authors acknowledge several hurdles:

  • Tool Bias: Sentiment and toxicity tools (like Perspective API) are primarily designed for US English. Even with translation, cultural nuances and "bad sentence structure" can lead to errors in toxicity quantification.
  • Selection Bias: Twitter users are not a perfectly representative sample of the general population in Pakistan or India.
  • Scope: The study focused on official, prominent accounts. Future work aims to include "less prominent but perhaps more extreme" accounts to capture a broader spectrum of the political landscape.

Summary updated

Abstract

Media outlets and political campaigners recognise social media as a means for widely disseminating news and opinions. In particular, Twitter is used by political groups all over the world to spread political messages, engage their supporters, drive election campaigns, and challenge their critics. Further, news agencies, many of which aim to give an impression of balance, are often of a particular political persuasion which is reflected in the content they produce. Driven by the potential for political and media organisations to influence public opinion, our aim is to quantify the nature of political discourse by these organisations through their use of social media. In this study, we analyse the sentiments, toxicity, and bias exhibited by the most prominent Pakistani and Indian political parties and media houses, and the pattern by which these political parties utilise Twitter. We found that media bias and toxicity exist in the political discourse of these two developing nations.
BibTeX
@misc{qayyum2018exploring,
  author = {Adnan Qayyum and Zafar Gilani and Siddique Latif and Junaid Qadir},
  title = {{Exploring Media Bias and Toxicity in South Asian Political Discourse}},
  year = {2018},
  eprint = {1811.06693},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1811.06693},
}
Figure