TL;DR
Current methods for detecting malicious content on social media, which leverage social graph information, perform well on static datasets but fail to generalize to the dynamic nature of online content. This paper introduces a novel, more realistic evaluation setup using few-shot subgraph sampling to better assess model generalization, revealing that existing models struggle significantly in this more challenging inductive setting.
What Problem Does This Paper Address?
Social media platforms are rife with malicious content like misinformation and hate speech. While community models, which combine content analysis with social graph structure, have shown promise, their real-world effectiveness is questionable. This is largely due to current evaluation practices relying on static datasets and transductive learning. These approaches fail to capture the dynamic evolution of online content, user behavior, and graph structures, leading to models that perform well on historical data but poorly on new, unseen content. The authors argue that this mismatch is a key reason why malicious content continues to spread, and they aim to bridge this gap by proposing a more realistic evaluation setup that tests for inductive generalization.
Methods
The core of this paper is a novel few-shot subgraph sampling approach designed to mimic realistic social media scenarios. Instead of using entire graphs, the method samples small, local subgraphs that contain limited context and only a few labeled examples. This "user-centered sampling" ensures locality by focusing on a user's neighborhood and limits context to simulate moderation before widespread dissemination.
To evaluate models under this new setup, the authors employ meta-learning. Specifically, they adapt gradient-based meta-learning algorithms like MAML (Model-Agnostic Meta-Learning) and ProtoMAML (Prototypical MAML) to train community models. These meta-learners are trained to adapt quickly to new tasks (i.e., new subgraphs) with minimal labeled data, making them more suitable for the inductive evaluation setting.
Datasets
The study utilizes three social graph datasets:
- GossipCop: Used for pre-training meta-learners. It contains fact-checked celebrity rumor articles and user interactions.
- CoAID: Used for evaluating generalization to unseen graphs. It comprises articles from the early COVID-19 pandemic.
- TwitterHateSpeech: Also used for evaluating generalization. This dataset focuses on hate speech and has a different relational schema (authorship vs. retweet interactions).
All datasets were rehydrated using the Twitter Academic API.
Key Insights and Ideas
- Static evaluation is misleading: High performance on existing benchmarks doesn't translate to real-world effectiveness because these benchmarks don't reflect the dynamic, evolving nature of online content and social graphs.
- Inductive generalization is crucial: Models need to adapt to new content and graph structures rapidly, especially given the cost of labeling and the need for early detection.
- Few-shot subgraph sampling is a better proxy for reality: This approach better simulates the challenges of content moderation in a live, evolving social network.
- Meta-learning excels in inductive settings: Models trained with meta-learning, particularly those incorporating prototypical initialization, show a greater capacity to generalize to unseen graphs and tasks compared to standard community models.
Key Results
- Existing models struggle: When evaluated using the proposed few-shot subgraph sampling setup on unseen graphs (CoAID and TwitterHateSpeech), standard community models (like the SUBGRAPHS baseline) performed poorly, often near random chance.
- Meta-learners show promise: Meta-learning approaches, especially PROTOMAML, significantly outperformed the baselines in inductive generalization. For instance, on CoAID with k=8, PROTOMAML achieved an MCC of 0.1709, a substantial improvement over baselines.
- GossipCop is not a sufficient benchmark: The paper highlights that strong performance on GossipCop does not guarantee generalization to other datasets, suggesting the dataset might be too static or lack the diversity needed for robust evaluation.
- Homophily plays a role: Analysis of homophily in sampled subgraphs revealed dataset-specific patterns that influence model performance, particularly in the TwitterHateSpeech dataset where heterophily in the 'None' class posed a challenge.
Limitations and Future Work
- Dataset diversity: The authors acknowledge that while GossipCop is large, it represents a single task and a relatively uniform structure. Ideally, meta-training would benefit from a wider variety of distinct graph datasets to foster greater task diversity.
- Class imbalance: An open problem is applying meta-learning to class-imbalanced datasets, which is common in malicious content detection and can pose challenges for meta-learners that reset classification heads.
- Meaningful comparison: The lack of standardized, open-access benchmarks for community models in malicious content detection hinders direct comparison with future work. The authors hope their proposed evaluation setup will encourage the development and adoption of such benchmarks.
Summary updated