TL;DR
This paper introduces a novel Graph Attention Network (GAT) model that dynamically learns user representations in social media by considering the varying relevance of their social connections for specific NLP tasks. This approach significantly outperforms static methods, particularly in tasks like stance and hate speech detection, by better capturing context-dependent homophily.
What Problem Does This Paper Address?
Understanding users in social media is crucial for many NLP tasks, as their social connections can reveal a lot about their behavior and communication style. Existing methods often represent users statically, treating all their social connections equally. However, this overlooks a key sociolinguistic observation: individuals belong to multiple communities, and the relevance of these communities shifts depending on the context or task. For example, a user's political affiliations might be more relevant for predicting their stance on a political issue than their interest in a sports team, even if both are important aspects of their social identity. Current models fail to capture this context-dependent relevance, leading to suboptimal user representations.
Methods
The authors propose a model that combines linguistic information from a user's text with a dynamically generated social representation. The core innovation lies in the Social Module, which utilizes Graph Attention Networks (GATs).
Here's how it works:
- Initial User Representation: Users are initially represented using Node2Vec, a standard graph embedding technique that learns representations by exploring the social graph. However, Node2Vec treats all neighbors uniformly.
- Dynamic Social Representation with GATs: The GAT encoder then takes these initial representations and dynamically updates them. It uses a self-attention mechanism to assign different weights (attention coefficients) to each of a user's neighbors. This allows the model to learn which connections are most relevant for the specific NLP task at hand.
- Late Fusion: The dynamically generated social representation is then concatenated with the linguistic representation (derived from an LSTM) and fed into a two-layer classifier for the final prediction.
The key novelty is the dynamic exploration and weighting of social connections by GATs, moving beyond static embeddings.
Datasets
The authors evaluate their model on three publicly available Twitter datasets:
- Sentiment Analysis: Task-4 of SemEval-2017, containing ~62k tweets labeled as POSITIVE, NEGATIVE, or NEUTRAL.
- Stance Detection: Task-6 (Subtask A) of SemEval-2016, with ~4k tweets labeled as FAVOR, AGAINST, or NEUTRAL, concerning five topics.
- Hate Speech Detection: A dataset from Founta et al. (2018), containing ~44k tweets labeled as NORMAL or HATEFUL.
For each dataset, they construct an independent social graph where users are nodes and retweets/mentions form edges.
Key Insights and Ideas
- Context Matters: The relevance of social connections is not static; it's highly dependent on the specific task. A user's political community might be crucial for stance detection but less so for sentiment analysis.
- Beyond Static Embeddings: Static user embeddings (like those from Node2Vec) can capture some homophily but fail to adapt to task-specific relevance, leading to a conflation of information when users belong to multiple, differently relevant communities.
- GATs as a Solution: Graph Attention Networks are well-suited to dynamically weigh social connections, allowing the model to focus on the most pertinent neighbors for a given task.
- Homophily Varies by Task: The study found higher homophily (users with similar labels being more connected) in the Stance and Hate Speech datasets compared to Sentiment. This explains why social information is more beneficial for the former tasks.
Key Results
- Improved Performance: The proposed LING+GAT model significantly outperforms baseline models (LING, LING+PV, LING+N2V) on Stance Detection and Hate Speech Detection.
- Sentiment Analysis: Social information did not provide significant improvements for Sentiment Analysis in this study, which the authors attribute to lower homophily in the sentiment-specific social graph.
- LING+N2V vs. LING+PV: Both Paragraph Vector (PV) and Node2Vec (N2V) user representations showed improvements over using only linguistic information (LING). N2V was particularly effective for Stance detection.
- LING+GAT Superiority: The GAT-based model consistently outperformed LING+N2V, demonstrating the advantage of dynamic attention over static graph embeddings. For instance, in hate speech detection, LING+GAT achieved an F1 score of 0.674, compared to 0.656 for LING+N2V.
Limitations and Future Work
- Neighborhood Size: The performance of the GAT model can be affected by neighborhood size; very large neighborhoods can lead to diluted attention values, making it harder to identify truly relevant connections.
- Disconnected Users: A significant portion of users (~800 for Stance, ~2.4k for Hate Speech) were not connected to the relevant sub-community in the social graph. While external nodes were added to mitigate this, it remains a challenge for GAT.
- Future Work: The authors plan to investigate cases where social information is not beneficial and to explore the model's ability to dynamically update user representations for the same author in different contexts, a capability not fully tested in this work due to data limitations.
Summary updated