FRAUD-Net: Fraud News Detection using Sample Uncertainty & Domain Aware Generalized Network

Devendra Patel, Vikas Verma, Shreyas Kumar Tah, Shwetabh Biswas, Soma BiswasWACV 2025

TL;DR

This paper introduces FRAUD-Net, a novel framework for detecting out-of-context misinformation (real images with fake captions). It leverages pre-trained vision-language models, incorporates domain-specific knowledge, and dynamically handles sample uncertainty to improve detection accuracy and robustness, outperforming existing methods.

What Problem Does This Paper Address?

The proliferation of social media has made it easier than ever to spread misinformation. A particularly insidious form is "out-of-context misinformation," where a genuine image is paired with a fabricated or misleading narrative. This tactic requires minimal technical skill to execute but can have significant societal impact. Manually verifying the authenticity of every image-text pair is infeasible, necessitating robust automated detection methods. While recent work has focused on using multimodal information (images, text, and "evidences" gathered from web searches) and powerful pre-trained models like CLIP, there are still challenges. These include handling variable or missing evidence, effectively utilizing domain-specific knowledge (e.g., medical expertise for healthcare news), and dealing with confusing training samples that can degrade model performance.

Methods

FRAUD-Net is designed to address these challenges through three novel modules built upon a frozen CLIP encoder:

  • Evidence-Guided Concept Enhancement Module: This module addresses the issue of variable evidence. It uses transformer blocks to process the input image, text, and their corresponding textual and visual evidences. The goal is to enhance relevant concepts within the input by leveraging the information from these evidences, even if the number of evidences is zero. An "evidence summarization transformer" then creates a global summary vector from all evidences.
  • Domain-aware Classifier Module: Recognizing that different domains (e.g., politics, healthcare) have unique characteristics, this module incorporates domain information. This can be explicitly provided or derived from the input caption using large language models. This allows the model to learn better and dynamically adapt its predictions during testing based on the specific domain context.
  • Uncertain Sample Identification Module: To mitigate the negative impact of confusing training samples on model performance, this module dynamically identifies and discards uncertain samples. It analyzes the consistency of predictions for each training sample over several epochs and discards those with a high rate of incorrect predictions relative to a threshold.

These modules work together, with the output of the evidence enhancement module and domain information being fed into a final classifier.

Datasets

The paper primarily uses the NewsCLIPpings dataset, a large-scale benchmark for out-of-context fake news detection. It also evaluates on the Verite dataset, which includes true, out-of-context, and mis-captioned samples. NewsCLIPpings is derived from VisualNews and involves replacing original images with falsified ones, with evidences collected via cross-modal searches. Verite also uses similar evidence collection methods.

Key Insights and Ideas

  • Evidence is crucial, but robustness to its variability is key: The framework's ability to perform well even with zero or a limited number of evidences highlights its practical applicability in real-world scenarios where evidence might be scarce or unavailable.
  • Domain knowledge significantly boosts performance: Explicitly incorporating domain information allows the model to leverage specialized expertise, leading to more accurate and context-aware predictions. This is a relatively underexplored area in multimodal fake news detection.
  • Handling noisy training data is essential: Dynamically identifying and discarding uncertain samples during training prevents the model from being negatively influenced by ambiguous examples, leading to a more robust and accurate final model.
  • Simplicity can be effective: FRAUD-Net achieves state-of-the-art results without relying on complex external information like entities or additional pre-trained models (beyond CLIP), demonstrating the power of its core architectural innovations.

Key Results

  • State-of-the-art performance on NewsCLIPpings: FRAUD-Net (using the ViT-L/14 backbone) achieves an accuracy of 91.1% on the NewsCLIPpings dataset, outperforming the previous state-of-the-art SNIFFER [12] by 3.1% while requiring significantly less computational resources.
  • Robustness to evidence count: The model demonstrates excellent performance across varying numbers of test evidences, even performing well with a single evidence. It also shows strong performance when trained with zero evidences.
  • Cross-dataset generalizability: FRAUD-Net shows promising performance on the Verite dataset, indicating its potential to generalize across different benchmarks, although further work is suggested in this area.
  • Ablation studies confirm module effectiveness: Removing any of the proposed modules (domain-aware classifier, uncertain sample identification, or specific components of evidence enhancement) leads to a decrease in performance, validating their individual contributions.

Limitations and Future Work

  • Cross-dataset generalizability: While the model shows promise, the paper notes that achieving better cross-dataset generalizability is a direction for future work, as performance on Verite was lower than on NewsCLIPpings.
  • Evidence quality: The paper acknowledges that in some cases, even with evidence, the model can be misled if the evidence is not perfectly aligned or if there's a high correlation between cross-modal elements that creates ambiguity (as seen in one Verite example).
  • Computational requirements: While FRAUD-Net is computationally efficient compared to some SOTA methods, training still takes around 43 minutes on a single high-end GPU. Future work could explore further optimizations.
  • Domain information derivation: While the paper shows how to derive domain information from captions, the accuracy and robustness of this derivation process could be further investigated.

Summary updated

BibTeX
@inproceedings{patel2025fraud,
  author = {Devendra Patel and Vikas Verma and Shreyas Kumar Tah and Shwetabh Biswas and Soma Biswas},
  title = {{FRAUD-Net: Fraud News Detection using Sample Uncertainty \& Domain Aware Generalized Network}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
}
Figure