TL;DR
This paper explores using sentiment analysis to objectively detect political biases in Danish newspapers. By analyzing 360 articles from two politically opposed newspapers (Berlingske and Information) about two distinct political parties (Alternativet and Liberal Alliance), the study found significant differences in how these parties were portrayed, particularly by Information. This suggests sentiment analysis can be a valuable, quantitative tool for understanding media bias beyond traditional subjective methods.
What Problem Does This Paper Address?
Traditionally, identifying political biases in newspapers has relied on subjective methods like reader surveys and analysis of voting habits. These approaches can be prone to inaccuracies due to limited reader engagement with all media or reliance on hearsay. The authors aim to address this by investigating whether automated sentiment analysis, a computational method for extracting emotional tone from text, can provide a more objective and quantitative measure of political bias in news coverage. They specifically question if sentiment analysis can reveal a newspaper's bias by detecting a more positive or negative sentiment towards a particular political party.
Methods
The core methodology involves applying sentiment analysis to a corpus of Danish newspaper articles. The authors utilized the AFINN dictionary, a lexicon of Danish words rated on a sentiment scale from -5 (very negative) to +5 (very positive). For each article, a Mean Sentiment Score (MSS) was calculated by averaging the sentiment scores of the words within it. This MSS was then analyzed using a two-way factorial ANOVA to identify interaction effects between the newspaper and the political party being discussed. Pairwise Bonferroni corrected post-hoc tests were employed to pinpoint specific differences.
The sentiment analysis process involved:
- Tokenization: Breaking down articles into individual words.
- Normalization: Converting all characters to lowercase.
- Dictionary Lookup: Assigning sentiment scores to words present in the AFINN dictionary. Words not found in the dictionary were assigned a score of 0.
- MSS Calculation: Averaging the scores of all words in an article.
Datasets
The study used a custom-compiled dataset of 360 articles from two Danish newspapers: Berlingske and Information. These articles, published between January 1, 2015, and June 30, 2016, focused on two political parties: Alternativet (left-wing) and Liberal Alliance (right-wing). The dataset was carefully curated to ensure an equal number of articles for each newspaper-party combination (90 articles per condition), excluding articles that mentioned both parties or were shorter than 60 words.
Key Insights and Ideas
- Sentiment Analysis as an Objective Bias Detector: The study demonstrates that sentiment analysis can move beyond subjective assessments to quantitatively identify potential political biases in news reporting.
- Information's Distinctive Coverage: The newspaper Information showed a clear difference in sentiment when covering Alternativet versus Liberal Alliance, suggesting a more pronounced bias in its reporting compared to Berlingske.
- Berlingske's Balanced Sentiment, Unequal Representation: While Berlingske exhibited no significant difference in sentiment scores between the two parties, it published a substantially higher volume of articles about Liberal Alliance than Alternativet. This highlights that bias can manifest not only in tone but also in the sheer amount of coverage.
- The "Bag-of-Words" Limitation: The study acknowledges that the "bag-of-words" approach, which treats words independently, can miss nuances like irony, sarcasm, and the impact of quotes, potentially leading to misinterpretations of sentiment.
Key Results
- No Main Effect of Newspaper or Party: The analysis found no overall significant difference in sentiment scores between the two newspapers or between the two political parties when considered independently.
- Significant Interaction Effect: A significant interaction effect was found between the newspaper and the party (β = -0.16, p = 0.019). This indicates that the sentiment expressed towards the parties varied depending on which newspaper published the article.
- Information's Bias: Post-hoc tests revealed that Information presented Alternativet with a significantly more positive sentiment (MSS = 0.0152) compared to Liberal Alliance (MSS = 0.0017).
- Berlingske's Neutral Sentiment: Berlingske did not show a significant difference in sentiment scores for articles about Alternativet versus Liberal Alliance.
- Representation Disparity: Berlingske published 63% more articles on Liberal Alliance than Alternativet during the study period, while Information published only 11% more.
Limitations and Future Work
Limitations: * Bag-of-Words Approach: The reliance on AFINN's dictionary and the bag-of-words model means that context, irony, sarcasm, and the source of quotes are not fully accounted for, potentially leading to errors in sentiment scoring. * AFINN Dictionary Limitations: The AFINN dictionary was primarily designed for microblogging (tweets) and may not perfectly capture the nuances of newspaper language. Its creator single-handedly rated words, raising questions about validity. * Limited Scope: The study only analyzed two parties and two newspapers, which is insufficient for a comprehensive understanding of media biases.
Future Work: * Expand the Dictionary: Create a larger, more newspaper-specific sentiment dictionary. * Incorporate Context: Develop methods to account for co-occurrence, context, negations, and modifiers to improve sentiment accuracy. * Analyze More Parties and Newspapers: Broaden the scope to include a wider range of political actors and media outlets. * Investigate Representation Bias: Further explore how the volume of coverage (representation) contributes to political bias, beyond just sentiment. * Web-Mining for Data: Utilize web-mining techniques to gather larger datasets for more robust analysis. * Create Evaluation Datasets: Develop specific datasets for evaluating the performance of sentiment analysis tools on news content.
Summary updated