ReMoD: Rethinking Modality Contribution in Multimodal Stance Detection via Dual Reasoning

Bingbing Wang, Zhengda Jin, Bin Liang, Jing Li, Ruifeng Xu

TL;DR

This paper introduces MIND, a novel framework for multimodal stance detection that moves beyond simple feature fusion to a "learning to reason" paradigm. By mimicking human dual-process cognition, MIND uses an intuitive-reflective loop to dynamically adjust how it weighs text and image information, leading to more robust and accurate stance predictions, especially in complex cases like irony.

What Problem Does This Paper Address?

Understanding public opinion on social media is crucial, and Multimodal Stance Detection (MSD) aims to do this by analyzing text and images together. However, existing methods primarily focus on fusing features from different modalities. This approach struggles with nuanced cases where modalities might contradict each other or where subtle cues like irony or cultural context are at play. For instance, a positive text paired with an image of "coffins" might be misinterpreted as favorable if the model doesn't understand the symbolic meaning of "coffins" in political memes as policy failure. Similarly, a text using "What if..." phrasing might be misclassified as "against" if the model focuses only on keywords like "sues" and "block" without grasping the neutral, hypothetical nature of the statement. These "semantic traps" highlight a need for explicit reasoning rather than just feature combination.

Methods

MIND (Meta-cognitive Intuitive-reflective Network for Dual-reasoning) is inspired by the dual-process theory of human cognition, which posits two modes of thought: a fast, intuitive "System 1" and a slow, deliberate "System 2."

  1. Multimodal Knowledge Perception: This initial stage extracts rich semantic information from both text and images, including identifying key entities and retrieving relevant external knowledge (e.g., from Wikipedia). For images, it also grounds semantic entities to specific pixel regions using models like SAM.
  2. Experience-Driven Intuitive Reasoning (System 1): This stage simulates rapid, intuitive judgment. It retrieves relevant past experiences from "Modality" and "Semantic" Experience Pools based on the current text and image inputs. These retrieved experiences act as in-context exemplars to generate an initial stance hypothesis and rationale from text-only, vision-only, and multimodal perspectives.
  3. Meta-cognitive Reflective Reasoning (System 2): This is the deliberate, refining stage.
  4. Experience Formulation: It uses "Modality-CoT" and "Semantic-CoT" to analyze the initial predictions and rationales. This process distills "modality insights" (adaptive fusion strategies) and "semantic insights" (strategies for deeper understanding).
  5. Experience Evolution: The derived insights are used to update and refine the Modality and Semantic Experience Pools. This creates a self-improving loop where the model learns from its own reasoning process.

Datasets

The paper evaluates MIND on the MMSD dataset, which comprises five domains:

  • Multi-modal COVID-CQ (MCCQ)
  • Multi-modal Russo-Ukrainian Conflict (MRUC)
  • Multi-modal Taiwan Question (MTWQ)
  • Multi-modal Twitter Stance Election 2020 (MTSE)
  • Multi-modal Will-They-Won't-They (MWTWT)

Key Insights and Ideas

  • Learning to Reason vs. Learning to Fuse: The core idea is that for complex multimodal tasks like stance detection, explicit reasoning about inter-modal relationships is more effective than simply fusing features. This allows the model to handle ambiguity and contradiction.
  • Dual-Process Cognition as a Model: Mimicking human intuitive and reflective thinking provides a powerful framework for building more robust AI systems that can adapt their reasoning strategies.
  • Self-Improving Experience Pools: The dynamic updating of experience pools is crucial. It allows the model to continuously learn and refine its understanding, making it more adaptable to new or challenging examples.
  • Modality Contribution Rethinking: MIND doesn't assign fixed weights to modalities. Instead, its reflective stage dynamically determines how much to rely on text, vision, or a combination, based on the specific context and target. This is key to overcoming the limitations of static fusion methods.

Key Results

  • MIND significantly outperforms most baseline models on the MMSD benchmark in both in-target and zero-shot settings.
  • In the in-target setting, MIND achieves competitive performance, even against powerful LLMs like GPT4 and GPT4-Vision.
  • In the zero-shot setting, MIND demonstrates remarkable robustness and adaptability, achieving state-of-the-art results on a majority of evaluated datasets, highlighting its superior generalization capability.
  • Ablation studies show that both the Modality Experience Pool (MEP) and Semantic Experience Pool (SEP) are critical, with the MEP being particularly important for zero-shot performance. Removing the reflective CoT module also impairs performance, but to a lesser extent, suggesting it acts as a high-level synthesizer.

Limitations and Future Work

  • Computational Cost and Inference Latency: The deliberate dual-reasoning process, involving multiple calls to a large multimodal model, leads to significant computational overhead and slower inference times. This makes it less suitable for real-time applications.
  • Future Work: The authors suggest exploring techniques like knowledge distillation or model compression to create a more lightweight version of MIND while preserving its core reasoning capabilities.

Summary updated

Abstract

Multimodal Stance Detection (MSD) is a crucial task for understanding public opinion on social media. Existing methods predominantly operate by learning to fuse modalities. They lack an explicit reasoning process to discern how inter-modal dynamics, such as irony or conflict, collectively shape the user's final stance, leading to frequent misjudgments. To address this, we advocate for a paradigm shift from *learning to fuse* to *learning to reason*. We introduce **MIND**, a **M**eta-cognitive **I**ntuitive-reflective **N**etwork for **D**ual-reasoning. Inspired by the dual-process theory of human cognition, MIND operationalizes a self-improving loop. It first generates a rapid, intuitive hypothesis by querying evolving Modality and Semantic Experience Pools. Subsequently, a meta-cognitive reflective stage uses Modality-CoT and Semantic-CoT to scrutinize this initial judgment, distill superior adaptive strategies, and evolve the experience pools themselves. These dual experience structures are continuously refined during training and recalled at inference to guide robust and context-aware stance decisions. Extensive experiments on the MMSD benchmark demonstrate that our MIND significantly outperforms most baseline models and exhibits strong generalization.
BibTeX
@misc{wang2025remod,
  author = {Bingbing Wang and Zhengda Jin and Bin Liang and Jing Li and Ruifeng Xu},
  title = {{ReMoD: Rethinking Modality Contribution in Multimodal Stance Detection via Dual Reasoning}},
  year = {2025},
  eprint = {2511.06057},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.06057},
}
Figure