Multidimensional Analysis of Trust in News Articles (Student Abstract)

Avneet Kaur, Maitree Leekha, Utkarsh Chawla, Ayush Agarwal, Mudit Saxena, Nishtha Madaan, Kalapriya Kannan, Sameep MehtaAAAI 2020 · DOI

Abstract

In the current landscape of extreme political polarization and digital information saturation, public trust in news media has reached historic lows. This research proposes a multidimensional framework to quantify trust by analyzing news articles beyond their immediate textual content. By evaluating content-based linguistic shifts, the historical perspective of individual journalists, and the temporal diversity of topics covered by media houses, the system provides a relative measure of bias against user-defined trusted baselines.

Introduction

While the internet offers unprecedented access to information, it simultaneously complicates the reader's ability to discern authentic reporting from biased narratives. Traditional automated tools for detecting "fake news" often focus exclusively on the text of a single article. However, trust is a multifaceted construct. Bias frequently manifests through the selective inclusion of facts, the framing of perspectives by journalists, and the systemic omission of specific topics by media organizations. This paper introduces an automated system designed to cross-reference these patterns, reducing the manual burden on readers seeking objective information.

Methodology

The proposed system is structured into three distinct analytical modules:

1. Content-Based Analysis

This module evaluates a "target" article against a user-selected "trusted" baseline. It utilizes three primary lexical metrics:

  • Centrality Shift: Using semantic role graphs, the system measures changes in the "importance" (betweenness centrality) of specific entities between versions of a story. This captures how different sources "spin" a narrative by centering different actors.
  • Sentiment and Emotion Shift: The system tracks variations in tone and specific emotional associations (e.g., anger, fear, joy) linked to common entities.
  • Over-representation: It identifies entities present in the target article but absent in the trusted baseline to highlight potentially exaggerated content.

2. Journalistic Assessment

To contextualize a writer's perspective, the system generates "Metro Maps"—visual storylines that link a journalist's current work to their historical output. By extracting entities and relations and applying the CePS algorithm, the system identifies "relationship centers" that illustrate how a journalist's framing of a story evolves over time.

3. Temporal Diversity Analysis

This module addresses "topic selectivity bias" by categorizing articles using Latent Dirichlet Allocation (LDA). By analyzing the distribution of genres and topics over time, the system identifies whether a media house consistently ignores certain issues or over-indexes on others.

Experimental Setup and Results

The system was evaluated using a dataset crawled from three news sources, categorized into National (1,905 articles), Sports (704 articles), and Entertainment (68 articles) genres. In these experiments, "Source-1" was designated as the trusted baseline for comparison against "Source-2" and "Source-3."

Analysis focused on articles falling into the third quartile of bias scores—those representing the most significant deviations from the baseline. The results indicated that Source-3 consistently produced a higher volume of articles with significant representation shifts compared to Source-2 across all genres.

Table 1: Distribution of high-bias articles by genre for Source-2 and Source-3 relative to the Source-1 baseline.

As shown in Table 1, the National category exhibited the highest frequency of bias, with Source-3 showing 853 high-bias articles compared to 594 for Source-2. This suggests that Source-3 is significantly further removed from the representation provided by the trusted baseline.

Discussion and Conclusion

The findings underscore that trust is a relative metric. The system does not attempt to define an objective "truth" but rather allows users to mathematically measure how far various sources deviate from their preferred baseline. A key insight is the utility of "Centrality Shift" as a proxy for narrative spin; even when facts are accurate, changing the central node of a story significantly alters its framing.

Future work will focus on addressing the "trusted source" bottleneck to prevent the reinforcement of existing user biases and expanding the dataset to include a broader range of global media sources to ensure the system's scalability and robustness.

Summary updated

BibTeX
@inproceedings{kaur2020multidimensional,
  author = {Avneet Kaur and Maitree Leekha and Utkarsh Chawla and Ayush Agarwal and Mudit Saxena and Nishtha Madaan and Kalapriya Kannan and Sameep Mehta},
  title = {{Multidimensional Analysis of Trust in News Articles (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i10.7191},
}
Figure