TL;DR
This paper introduces MEAN, a novel neural network architecture that leverages entity information and external knowledge to detect political bias in news articles. By focusing on how entities are discussed, MEAN significantly outperforms existing methods that rely solely on textual content, offering a more nuanced understanding of media bias.
What Problem Does This Paper Address?
In today's media landscape, identifying the political perspective behind news reporting is crucial for understanding information and avoiding echo chambers. While previous work has focused on lexical and syntactic cues, bias is often more subtly expressed through the selective emphasis of certain aspects of a story related to specific entities. This paper tackles the challenge of detecting this nuanced bias by incorporating external knowledge about entities and their relationships into a deep learning model. The authors argue that understanding how entities are portrayed is key to uncovering the underlying political slant, especially when traditional linguistic cues are absent or intentionally masked.
Methods
The core of the proposed solution is the Multi-head Entity Aware Attention Network (MEAN). This architecture builds upon a hierarchical LSTM structure, incorporating entity and relation knowledge in several key ways:
- Entity and Relation Knowledge Representation:
- Wikipedia-Based Entity Representation: Pre-trained embeddings from Wikipedia2Vec are used to initialize entity representations, providing a rich source of background knowledge.
- Text-Based Entity Representation: An entity-level masking task, inspired by BERT, is employed to learn entity representations directly from the news articles themselves. This involves masking entity mentions and training a model to predict them based on context.
- Text-Based Relation Representation: Similar to entity representation, a bidirectional LSTM with self-attention is used to learn representations for entity pairs, capturing the relationships between them.
- Multi-head Entity-Aware Attention Network:
- Hierarchical Structure: The model uses separate encoders for words and sentences.
- Entity-Aware Attention: The attention mechanism is modified to incorporate entity embeddings. This allows the model to weigh words and sentences differently based on their relevance to specific entities or relations, effectively making the attention "entity-aware."
- Multi-head Attention: Inspired by the Transformer architecture, multiple attention heads are used to jointly attend to information at different positions, capturing a richer set of relationships and nuances.
The model learns document representations with respect to "anchor aspects" (top entities/relations in an article) and uses these aspect-oriented representations for classification.
Datasets
The authors evaluate their model on two datasets:
- Allsides: This dataset, collected from news aggregation websites, provides articles covering various events with bias labels (left, center, right) determined by crowdsourced and editorial reviews.
- SemEval 2019 Task 4: Hyperpartisan News Detection: This is an official training dataset for hyperpartisan news detection, where articles are labeled as either hyperpartisan or not.
Key Insights and Ideas
- Entities as Lenses for Bias: The central idea is that bias is often revealed through how entities are discussed, rather than just the words used. Focusing on entities provides a more direct path to understanding the political narrative.
- External Knowledge is Crucial: Leveraging pre-trained embeddings from sources like Wikipedia provides valuable background context that helps disambiguate and understand entities' roles and potential biases.
- Textual Context Alone is Insufficient: The paper demonstrates that while traditional text-based models are useful, they fall short in capturing the subtle biases that MEAN, with its entity-aware approach, can detect.
- Ensembling Boosts Performance: Combining predictions from entity-focused and relation-focused models (MEAN_ENT and MEAN_REL) further improves performance, suggesting that both types of relationships offer complementary signals.
Key Results
- Allsides Dataset: MEAN achieved state-of-the-art results, outperforming baseline models by a significant margin. For instance, MEAN_ELMO_Ensemble achieved a micro F1 score of 82.32 and a macro F1 of 81.30, surpassing other models.
- SemEval Dataset: MEAN also outperformed the winning system of the SemEval 2019 competition, achieving an accuracy of 85.22 with the MEAN_ELMO_Ensemble model.
- Ablation Studies: Removing entity embedding or multi-head attention individually led to performance drops, but removing both dramatically decreased performance, highlighting their complementary importance.
Limitations and Future Work
- Relation Representation Initialization: The authors note that MEAN_REL models performed less optimally than MEAN_ENT, attributing this to the lack of good initialization for relation representations.
- Aspect-Oriented Labels: Obtaining aspect-oriented document representations is expensive. The current approach uses the overall article bias label for each aspect, which is an approximation.
- Future Work: The authors suggest extending the work to learn better relation representations by explicitly modeling entity interactions. They also aim to weigh the importance of different aspects towards the overall article perspective, rather than treating them equally.
Summary updated