DAMMFND: Domain-Aware Multimodal Multi-view Fake News Detection

Weihai Lu, Yu Tong, Zhiqiu YeAAAI 2025 · DOI

TL;DR

This paper introduces DAMMFND, a novel framework for detecting fake news across different domains and modalities (text, image). It tackles the challenges of inaccurate domain identification, imbalanced data, and varying modal importance by disentangling domain-specific information and using a domain-aware mechanism to weigh different views for decision-making, achieving state-of-the-art results.

What Problem Does This Paper Address?

The proliferation of fake news poses a significant threat to social media and society. While automated detection methods exist, they often struggle with the complexities of real-world news, which spans multiple domains (e.g., politics, health, entertainment) and utilizes various modalities (text and images). Existing approaches face several hurdles:

  • Inaccurate Domain Identification: Predefined domain labels are often insufficient as news can inherently belong to multiple domains, leading to noisy signals.
  • Imbalanced Multi-domain Data: Some domains have significantly more data than others, which can lead to negative transfer effects where models trained on data-rich domains perform poorly on data-scarce ones.
  • Variable Modal Contributions: The importance of text versus images in determining fake news can vary greatly depending on the domain. For instance, image manipulation might be a stronger indicator in "entertainment" news than in "political" news, where textual analysis might be more critical.

This paper aims to address these challenges by developing a framework that can effectively handle multi-domain and multimodal fake news detection, considering the nuanced interplay between domains and modalities.

Methods

The proposed DAMMFND framework comprises several key components:

  • Multi-view Features Extraction and Aggregation: It leverages pre-trained models (BERT, CLIP, MAE) to extract unimodal embeddings for text and images. These are then fused to create multimodal representations. Three deep extractors generate representations from textual, visual, and multimodal features. Multi-channel aggregation is used to capture diverse perspectives within each modality.
  • Domain Disentanglement: This module aims to separate domain-specific information from the core semantic content of the news. It uses attention mechanisms to create "soft masks" that decompose features into domain and semantic branches. This is further refined using domain classification and domain-uniformity loss functions to ensure disentangled representations.
  • Domain-Aware Multi-View Discriminator (DAMVD): This component recognizes that not all domain features are equally useful for fake news detection. It employs a gating network to filter domain representations relevant to fake news detection and applies these filtered representations to each view (text, image, multimodal). This allows for domain-adaptive detection.
  • Domain-Enhanced Multi-view Decision Layer: This layer addresses the varying importance of different modalities across domains. It uses a Domain-aware Transformer (DAFormer) that integrates domain information to predict domain-specific weights for each view. These weights are then used to aggregate predictions from the individual views, leading to a more informed final decision.

Datasets

The paper evaluates DAMMFND on two real-world datasets:

  • Weibo: Contains 7,532 news articles for training and 1,996 for testing, categorized into nine domains.
  • Weibo-21: A multi-domain dataset with 9,127 articles, also partitioned into training and test sets.

Key Insights and Ideas

  • Domain-Specific Nuances Matter: The paper strongly emphasizes that fake news detection is not a one-size-fits-all problem. The interpretation of news and the indicators of fakeness vary significantly across different domains.
  • Disentangling is Crucial: Separating domain-specific information from semantic content is vital for accurate fake news detection, especially in complex, multi-domain scenarios. This helps reduce noise and negative transfer.
  • Not All Domain Information is Equal: The DAMVD component highlights that simply having domain labels isn't enough; the model needs to learn which domain-specific features are actually predictive of fake news.
  • Dynamic Modality Weighting: The Domain-Enhanced Multi-view Decision Layer is a key innovation, allowing the model to dynamically adjust the importance of text, image, and multimodal features based on the domain, mimicking human reasoning.

Key Results

DAMMFND achieves state-of-the-art performance on both the Weibo and Weibo-21 datasets.

  • Overall Performance: DAMMFND significantly outperforms existing multi-domain and multimodal fake news detection methods in terms of F1-scores and accuracy. For instance, it achieves an accuracy of 94.4% on Weibo and 94.7% on Weibo-21.
  • Ablation Studies: Removing individual components of DAMMFND consistently led to performance degradation, underscoring the effectiveness of each module:
  • Removing Multi-channel Features Aggregation significantly decreased performance.
  • Eliminating domain disentanglement substantially reduced performance.
  • Removing the domain information filtering mechanism in DAMVD led to F1-score decreases.
  • Omitting the three modal views in DAMVD showed that multi-view discrimination is crucial.
  • Removing the Domain-Enhanced Multi-view Decision Layer resulted in notable drops in accuracy.
  • t-SNE Visualizations: Figure 3 shows that DAMMFND generates fewer outliers and exhibits reduced overlap between real and fake news embeddings compared to baseline models, indicating better feature separation.

Limitations and Future Work

  • Limitations: The paper acknowledges that while DAMMFND improves domain identification, the inherent complexity and ambiguity of news data can still pose challenges. The reliance on pre-trained models for feature extraction is standard but can be a bottleneck if the pre-training data doesn't align well with the target domain.
  • Future Work: The authors suggest that future research could explore more sophisticated methods for domain disentanglement and investigate how to better handle scenarios with even more complex domain overlaps or entirely unseen domains. Further exploration into the interpretability of the domain-aware weighting mechanism could also be beneficial.

Summary updated

BibTeX
@inproceedings{lu2025dammfnd,
  author = {Weihai Lu and Yu Tong and Zhiqiu Ye},
  title = {{DAMMFND: Domain-Aware Multimodal Multi-view Fake News Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i1.32036},
}
Figure