Abstract
Short video platforms have become significant vectors for misinformation, presenting unique challenges for automated detection. We introduce TGFC-SVFN, a framework that leverages text-based reasoning as a "teacher" to guide the analysis of video and audio modalities. By integrating Large Language Model (LLM) reasoning with diffusion-based counterfactual inference, the system effectively identifies and mitigates spurious correlations and modality biases. Evaluated on the FakeSV benchmark, TGFC-SVFN achieves state-of-the-art performance, demonstrating superior generalization across time and events.
Introduction
Short video platforms blend visual frames, audio tracks, transcripts, and user comments, creating a complex environment for fake news detection. Existing AI models face two primary hurdles: Modality Heterogeneity, the difficulty of fusing disparate data types into a coherent truth score, and Modality Bias, where models latch onto spurious correlations (e.g., assuming the presence of the word "real" in comments implies authenticity). These issues often lead to "teacher model inadequacy," where flaws in the primary guiding modality are propagated through the system.
The TGFC-SVFN Framework
The proposed Text-Guided Fine-grained Counterfactual Inference (TGFC-SVFN) framework addresses these challenges through a multi-stage approach centered on a refined text-based teacher model.
LLM-Enhanced Teacher Reasoning
Text remains the most information-dense modality for news. TGFC-SVFN utilizes ChatGPT-3.5 to generate causality-based reasoning prompts. Instead of processing raw text, the LLM executes a three-step chain of thought: evaluating event reasonableness, checking logic against common sense, and providing an explained judgment. This ensures the teacher model focuses on critical authenticity markers rather than irrelevant linguistic noise.
Diffusion-Based Counterfactual Inference
To eliminate bias, the framework employs a Diffusion Model to simulate "what-if" scenarios. By injecting controlled noise into data features, the model creates counterfactual scenarios. Comparing the model's predictions on real versus noisy counterfactual data allows for the calculation of the "Total Indirect Effect," enabling the system to subtract bias from the final classification.
Knowledge Distillation and Fusion
Video and audio "student" models learn from the debiased text teacher via knowledge distillation. The framework uses Pearson correlation coefficients to capture linear relationships between modalities more effectively than standard methods. Finally, a Multi-head Attention Fusion mechanism merges these refined features for the final verdict.
Experimental Results
TGFC-SVFN was evaluated on the FakeSV dataset, the primary benchmark for short video fake news. The model was tested using two split strategies: By Time (temporal generalization) and By Event (unseen event generalization).
- Performance: The model achieved 91.99% accuracy on time-partitioned data and 81.73% on event-partitioned data, outperforming specialized baselines like MMCAN and SV-FEND.
- Ablation Studies: Removing causality-based prompts or counterfactual inference led to significant performance drops, confirming these components are vital for robust detection.
- Qualitative Analysis: Case studies showed the model correctly identified fake news that misled previous models due to positive sentiment in comments or deceptive keywords.
Conclusion and Future Work
TGFC-SVFN demonstrates that using diffusion models for debiasing and LLMs for reasoning pre-processing significantly improves fake news detection in multimodal contexts. While highly accurate, the framework's multi-stage nature introduces computational complexity and a dependency on the underlying LLM's reasoning quality. Future research will focus on optimizing real-time performance to meet the latency requirements of social media platforms.