Different Absorption from the Same Sharing: Sifted Multi-task Learning for Fake News Detection

Lianwei Wu, Yuan Rao, Haolin Jin, Ambreen Nazir, Ling SunEMNLP 2019 · OpenReview

TL;DR

This paper introduces a novel "sifted multi-task learning" approach for fake news detection that intelligently filters and selects shared features between related tasks (like stance detection). By using gated and attention mechanisms, their method avoids feeding irrelevant or harmful information into each task, leading to state-of-the-art performance on benchmark datasets.

What Problem Does This Paper Address?

The proliferation of fake news is a significant societal problem, with real-world consequences like stock market crashes and election interference. While multi-task learning (MTL) has shown promise in improving fake news detection by leveraging related tasks (e.g., stance detection), existing MTL methods often indiscriminately share all learned features. This can lead to "negative transfer," where irrelevant or even detrimental features from one task confuse and degrade the performance of another. The authors aim to solve this by developing a more discerning way to share information between tasks.

Methods

The core innovation is a "sifted multi-task learning" framework. It builds upon the standard "shared-private" MTL architecture, which separates task-specific (private) features from features shared across tasks. The key novelty lies in the introduction of a selected sharing layer placed after the shared feature extractor. This layer comprises two components:

  • Gated Sharing Cell: Inspired by LSTM/GRU gates, this cell acts as a filter, learning to discard useless or noisy shared features.
  • Attention Sharing Cell: This component uses an attention mechanism to selectively highlight and prioritize shared features that are most beneficial for a specific task.

The model also incorporates Transformer encoders for both shared and private feature extraction, leveraging their ability to capture long-range dependencies and parallelize computations. The final output is generated by combining the private features with the selectively shared features from the selected sharing layer.

Datasets

The authors evaluate their method on two widely used public datasets for fake news detection and stance detection:

  • RumourEval: Contains Twitter conversation threads related to various events, annotated for rumor veracity and stance.
  • PHEME: Similar to RumourEval, it also includes Twitter conversation threads with annotations for rumor veracity and stance.

The paper does not introduce new datasets but focuses on improving performance on existing benchmarks.

Key Insights and Ideas

  • Indiscriminate sharing is detrimental: The paper strongly argues that simply sharing all learned features in MTL can hurt performance by introducing noise. This highlights the importance of feature selection and filtering in multi-task scenarios.
  • Gated and Attention mechanisms are powerful filters: The combination of a gating mechanism (to discard) and an attention mechanism (to focus) proves effective in creating a more refined set of shared features.
  • Transformer encoders are well-suited for this task: The use of Transformer encoders for feature extraction demonstrates their effectiveness in capturing complex textual semantics and long-range dependencies, outperforming LSTM-based approaches.
  • Task-specific feature selection matters: The visualization experiments reveal that the selected sharing layer can indeed filter out irrelevant tokens for one task while retaining important ones for another, demonstrating its ability to learn task-specific nuances from shared information.

Key Results

The proposed sifted multi-task learning method achieves state-of-the-art performance on both the RumourEval and PHEME datasets. Specifically, it boosts the F1-score by over 0.87% on RumourEval and 1.31% on PHEME compared to previous methods. Ablation studies confirm the effectiveness of both the Transformer encoder and the selected sharing layer (especially the combined gated and attention cells).

Limitations and Future Work

The authors acknowledge a few limitations and suggest future research directions:

  • Dataset Size: Performance on RumourEval was slightly less pronounced than on PHEME, potentially due to RumourEval's smaller dataset size, which may not be sufficient for training deep neural networks effectively.
  • Fusion Mechanism: The paper mentions exploring the fusion mechanism of private and shared features as an area for future research.
  • Meta-data Integration: Integrating meta-data of fake news into the model's inputs is suggested as another important direction to further improve performance.

Summary updated

BibTeX
@inproceedings{wu2019different,
  author = {Lianwei Wu and Yuan Rao and Haolin Jin and Ambreen Nazir and Ling Sun},
  title = {{Different Absorption from the Same Sharing: Sifted Multi-task Learning for Fake News Detection}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=B6trz2c9uQB},
}
Figure