Model Generalization on COVID-19 Fake News Detection

Yejin Bang, Etsuko Ishii, Samuel Cahyawijaya, Ziwei Ji, Pascale Fung

Introduction

The COVID-19 pandemic was accompanied by a massive "infodemic," characterized by the rapid spread of misinformation across social media. While Natural Language Processing (NLP) models have achieved impressive accuracy (often exceeding 98%) on specific fake news benchmarks, these models frequently fail when applied to real-world, out-of-domain data. Researchers from HKUST investigated this lack of generalization, proposing methods to make detectors more robust to the evolving nature of misinformation.

The Challenge: Distribution Shift and Overfitting

The core issue in fake news detection is that models often overfit to specific keywords or metadata prevalent in their training sets. For instance, a model trained on early 2020 data might learn to associate "drinking bleach" with fake news but fail to recognize new misinformation patterns emerging months later. This "distribution shift" means that high performance on a static benchmark does not translate to reliable real-world performance.

t-SNE visualizations showing the distribution shift between the FakeNews-19 training set (a), its in-domain test set (b), and the out-of-domain Tweets-19 test set (c).

As seen in the t-SNE visualizations above, while "Real" and "Fake" classes may be well-separated in the training domain, the clusters become significantly more overlapping and shifted in the out-of-domain test set, leading to a collapse in model performance.

Proposed Methodology

The authors explored two primary strategies to enhance model robustness:

1. Robust Loss Functions

Standard Cross-Entropy (CE) loss is sensitive to noisy labels, which are common in social media datasets. The researchers experimented with alternative loss functions designed to handle noise and outliers:

  • Symmetric Cross-Entropy (SCE): Incorporates a reverse cross-entropy term to mitigate the impact of noisy labels.
  • Generalized Cross-Entropy (GCE): A hybrid approach combining the noise-robustness of Mean Absolute Error with the efficiency of Cross-Entropy.
  • Curriculum Loss (CL): A 0-1 loss function that provides a tighter upper bound, specifically targeting outlier robustness.

2. Influence-based Data Cleansing

The most innovative contribution of the study is a radical data cleansing technique. Using turn-over dropout, the authors calculated the "influence" of each training instance—determining whether a specific sample helped or hindered the model's ability to reduce loss on a validation set.

By removing instances with low or negative influence, the authors aimed to distill the training set into a "core" of highly generalizable samples.

t-SNE plot illustrating the selection of influential training samples (darker points) from the full dataset (lighter points) during the cleansing process.

Experimental Results: The 99% Paradox

The study utilized two datasets: FakeNews-19 (10,700 annotated posts for training and in-domain testing) and Tweets-19 (an external set for out-of-domain evaluation).

The results revealed a stark "Generalization Gap." A baseline RoBERTa-large model achieved a 98.13% Weighted F1 on FakeNews-19 but plummeted to 33.65% on Tweets-19.

However, the influence-based cleansing yielded a surprising "Less is More" result. By training on only the 64 most influential samples—effectively discarding 99% of the original data—the model's performance on the unseen Tweets-19 set jumped to 54.33% W-F1. This 20.68-point gain suggests that the majority of "big data" in these benchmarks may actually be noise that prevents models from learning universal features of fake news.

Conclusion and Future Directions

While these techniques significantly improve generalization, they introduce a performance trade-off: the 99% cleansing method reduced in-domain accuracy from 98% to approximately 87%. Robust loss functions like Curriculum Loss offered a more balanced improvement, raising out-of-domain performance to 37.50% without drastic data removal.

Future research will focus on combining adaptive training with influence-based cleansing and refining influence estimation to better handle noise during the initial training phases.

Summary updated

BibTeX
@inproceedings{bang2021model,
  author = {Yejin Bang and Etsuko Ishii and Samuel Cahyawijaya and Ziwei Ji and Pascale Fung},
  title = {{Model Generalization on COVID-19 Fake News Detection}},
  booktitle = {CONSTRAINT @ AAAI},
  year = {2021},
  doi = {10.1007/978-3-030-73696-5\_13},
}
Figure