Bilinear Learning with Dual-Chain Feature Attention for Multimodal Rumor Detection

Zhang Guo, Haonan Liu, Lin Zuo, Junhao WenMathematics 2025 · DOI

TL;DR

This paper introduces a novel "dual-chain" deep learning framework for detecting rumors on social media by analyzing both text and images. It addresses limitations in existing methods by simultaneously learning semantic and logical features, and crucially, by explicitly modeling the connections between text and images, leading to improved accuracy.

What Problem Does This Paper Address?

The proliferation of social media has made it incredibly easy for rumors to spread rapidly. Manually verifying every piece of information is impossible, making automated rumor detection a critical need. While many approaches now use both text and images (multimodal data) to detect rumors, they often fall short. Existing methods tend to:

  1. Focus too much on surface-level text: They miss logical inconsistencies within the text itself.
  2. Treat text and images as separate: They don't effectively capture how text and images relate to each other.
  3. Ignore semantic mismatches: They overlook instances where the text and image contradict each other, a common tactic in spreading misinformation.

This paper aims to tackle these issues by developing a more holistic approach to understanding multimodal rumor content.

Methods

The core of this work is the Dual-chain Multimodal Feature Learning (DMFL) framework. It operates on two parallel "chains":

  • Basic Semantic Feature Extraction Chain: This chain extracts fundamental semantic meaning from both text and images. It uses a Vision Transformer (ViT) for image encoding and Word2vec with CNNs for text encoding. Crucially, it employs a modal alignment technique to reduce differences between text and image features before fusion, ensuring they are in a comparable representation space.
  • Logical Connection Feature Extraction Chain: This chain focuses on identifying logical inconsistencies. It injects image features into the text's logical feature encoder, allowing it to learn relationships within the text and between the text and images. This is achieved using a cross-attention mechanism.

These two chains then feed into a cross-modal cross-attention module for multi-level fusion of the extracted features. Finally, a classification prediction module uses these fused features to determine if a post is a rumor.

Datasets

The proposed DMFL model was evaluated on two widely-used, real-world multimodal rumor detection datasets:

  • Pheme: An English dataset collected from X (formerly Twitter), focusing on five breaking news stories.
  • Weibo: A Chinese dataset from Sina Weibo, covering actual events and false events from 2015-2016.

The authors filtered these datasets to include only instances with both text and images.

Key Insights and Ideas

  • The importance of logical consistency: Simply understanding the meaning of text and images isn't enough; detecting how they logically connect or contradict each other is vital for identifying rumors.
  • Bridging the modality gap: Explicitly aligning text and image features before fusion is crucial. This helps the model learn more effectively from the combined information, especially when modalities have different characteristics (e.g., short vs. long text).
  • Dual-chain learning is powerful: Separating semantic and logical feature learning into parallel chains allows for a more comprehensive understanding of rumor content.
  • Attention is key for fusion: The cross-modal cross-attention mechanism effectively fuses the learned semantic and logical features, leading to better performance.

Key Results

The DMFL model demonstrated superior performance compared to several state-of-the-art baseline methods on both the Pheme and Weibo datasets.

  • Pheme Dataset: DMFL achieved an accuracy of 0.8883, outperforming the best baseline (MFAN) by 1.65%.
  • Weibo Dataset: DMFL achieved an accuracy of 0.9051, outperforming the best baseline (MFAN) by 3.89%.

Ablation studies confirmed the effectiveness of each component of the DMFL framework, with the full model consistently outperforming variants that removed specific modules like modality alignment or the logical connection chain.

Limitations and Future Work

  • Dataset differences: The paper notes that the Weibo dataset (Chinese, longer texts) shows a different learning pattern than Pheme (English, shorter texts), suggesting that text length and language can influence the importance of different features.
  • Parameter sensitivity: While the paper analyzes the impact of key parameters like loss coefficients and the number of attention heads, further exploration into hyperparameter optimization could be beneficial.
  • Computational complexity: While not explicitly detailed as a limitation, the use of Transformers and multiple complex modules suggests a potentially high computational cost, which might be a consideration for real-time applications.

The authors suggest that their work provides a strong foundation for future research in multimodal rumor detection, particularly by emphasizing the importance of multi-level feature extraction and cross-modal interaction.

Summary updated

Abstract

The rapid growth of social media and online information-sharing platforms facilitates the spread of rumors. Accurate rumor detection to minimize manual verification efforts remains a critical research challenge. While multimodal rumor detection leveraging both text and visual data has gained increasing attention due to the diversification of social media content, existing approaches face the following three key limitations: (1) yhey prioritize lexical features of text while neglecting inherent logical inconsistencies in rumor narratives; (2) they treat textual and visual features as independent modalities, failing to model their intrinsic connections; and (3) they overlook semantic incongruities between text and images, which are common in rumor content. This paper proposes a dual-chain multimodal feature learning framework for rumor detection to address these issues. The framework comprehensively extracts rumor content features through the following two parallel processes: a basic semantic feature extraction module that captures fundamental textual and visual semantics, and a logical connection feature learning module that models both the internal logical relationships within text and the cross-modal semantic alignment between text and images. The framework achieves the multi-level fusion of text–image features by integrating modal alignment and cross-modal attention mechanisms. Extensive experiments on the Pheme and Weibo datasets demonstrate that the proposed method performs better than baseline approaches, confirming its effectiveness in detecting multimodal rumors.
BibTeX
@inproceedings{guo2025bilinear,
  author = {Zhang Guo and Haonan Liu and Lin Zuo and Junhao Wen},
  title = {{Bilinear Learning with Dual-Chain Feature Attention for Multimodal Rumor Detection}},
  booktitle = {Mathematics},
  year = {2025},
  doi = {10.3390/math13111731},
}
Figure