TL;DR
This paper introduces an "opinion-aware knowledge graph" (OKG) to improve political ideology detection from text. By integrating sentiment and target entities into a knowledge graph, their method achieves higher accuracy than traditional methods by capturing nuanced ideological stances.
What Problem Does This Paper Address?
Accurately identifying an individual's political ideology from their written text is crucial for understanding public opinion and user behavior on social media. Traditional methods often rely on simple "bag-of-words" approaches, which treat all words equally and fail to capture the subtle differences in how people express their ideologies. For instance, the words "support" and "oppose" carry significant ideological weight, but a bag-of-words model might overlook this distinction if the surrounding words are similar. This paper aims to bridge this gap by developing a more sophisticated method that considers not just the words used, but also the opinions expressed towards specific entities or topics.
Methods
The core of this paper is the Opinion-aware Knowledge Graph (OKG). Here's how it works:
- Opinion Estimation: The process starts by estimating opinions towards entities within a knowledge graph. This involves analyzing text to determine sentiment (positive, negative, neutral) and associating it with specific entities. The key innovation here is to quantify these opinions separately for "liberal" and "conservative" viewpoints.
- Ideology Propagation: The estimated opinions are then propagated across the knowledge graph. This step leverages the existing relationships between entities in a background knowledge base (like DBpedia) to infer opinions on entities that might not have direct textual mentions in the training data. This allows the model to learn from broader contextual information.
- Opinion-aware Knowledge Graph Construction: The OKG is essentially an extended knowledge graph where entities are annotated with ideology-dependent opinion distributions. This graph integrates factual information from a standard knowledge base with subjective opinion data.
- Political Ideology Detection: For a given text (test set), entities mentioned in the text are matched to entities in the OKG. The ideology of the text is then determined by aggregating the ideology-dependent opinions associated with these matched entities. The core idea is that similar opinions on the same entities suggest similar ideologies.
Datasets
The authors evaluated their approach on three real-world corpora:
- U.S. Congressional floor debate (Convote) dataset: Transcripts from US Congress debates, with speakers labeled by political party.
- Ideological Books Corpus (IBC): Sentences from books and articles by authors with known political leanings, annotated for political bias.
- Twitter dataset: Tweets collected from liberal and conservative Twitter accounts.
They used DBpedia as their source for the background knowledge graph.
Key Insights and Ideas
- Nuance over Keywords: The paper strongly emphasizes that political ideology is not just about what topics are discussed, but how opinions are expressed about them. The distinction between "support" and "oppose" is critical, and this is what OKG aims to capture.
- Leveraging Background Knowledge: Integrating external knowledge graphs (like DBpedia) is crucial. It allows the model to infer opinions on entities that might not be explicitly mentioned in a given text, thereby enriching the understanding of ideological stances.
- Opinion Distribution is Key: The idea of quantifying opinion distributions for both liberal and conservative viewpoints on the same entity is central. This allows for a direct comparison and identification of ideological leanings.
- Cross-Domain Applicability: The method shows promise across different types of text, from formal debates to informal tweets, suggesting its robustness.
Key Results
The proposed Opinion-aware Knowledge Graph (OKG) method consistently outperformed baseline methods (Logistic Regression, SVM, and RNNs) across all three datasets. For example, on the Convote dataset, OKG achieved an accuracy of 81.5%, significantly higher than LR (68.3%), SVM (71.4%), and even RNNs (81.5% for RNN1, but OKG's approach is more interpretable and knowledge-driven). Similar improvements were observed on the IBC and Twitter datasets. The authors highlight that OKG's ability to integrate factual and subjective information, and perform joint inference with knowledge bases, is key to its superior performance, especially in handling texts with less obvious keywords.
Limitations and Future Work
The paper acknowledges that the initial opinion estimation relies on a "holistic lexicon-based approach," which might have limitations. While the OKG framework is presented as a significant advancement, the authors do not explicitly detail future work directions beyond the scope of their current experimental setup. However, the nature of knowledge graph construction and opinion propagation suggests potential avenues for exploring more sophisticated opinion extraction techniques, handling more complex ideological spectrums beyond binary liberal/conservative, and integrating temporal dynamics of opinions.
Summary updated