Abstract
Detecting fake news on modern video-centric platforms like TikTok and Instagram is significantly more challenging than on traditional social media due to the absence of explicit social cascades (e.g., retweets). This paper introduces HML, a novel framework that reconstructs these hidden social connections using temporal patterns and multimodal content. HML employs an improved Hawkes process for latent network inference, self-supervised contrastive learning for cross-modal alignment, and a heterogeneous graph transformer for final classification. Experimental results on several benchmarks show that HML outperforms state-of-the-art models and even advanced large language models (LLMs) like GPT-4o in identifying sophisticated multimodal fake news.
Introduction
Most existing fake news detection research has focused on platforms like Twitter (X) or Weibo, where "retweet" and "comment" chains are readily visible. These cascades provide a structural "fingerprint" of how news spreads, which is vital for identifying misinformation. However, on modern video-centric platforms such as TikTok, Instagram, or YouTube, these explicit relationships are often obscured by platform mechanics or privacy settings, leaving researchers with a fragmented view of news dissemination.
Furthermore, fake news has evolved into "complex heterogeneous multimodal" content, incorporating deepfakes, edited audio, and misleading captions. Current models often fail to capture the deep relationships between these disparate data types or the latent social context. HML addresses this by inferring a latent social network where an explicit one is missing.
Methodology: The HML Framework
The HML framework operates through three integrated stages designed to capture both content and context:
- Social Latent Network Inference: The model uses an improved Hawkes Process—a self-exciting point process—to estimate the likelihood of influence between news items based on their temporal appearance. It incorporates Gaussian noise to simulate the "forgetting" effect of public attention over time, effectively reconstructing the likely social graph.
- Self-supervised Multimodal Content Learning: To handle text, audio, and video, HML utilizes a "Pruning and Mask" strategy for data augmentation. It employs contrastive learning to align these modalities, ensuring that features from different sources (e.g., a video's audio vs. its frames) are consistent and robust against noise.
- Network-Augmented Graph Transformer: The inferred social relationships and refined multimodal features are integrated into a Heterogeneous Multimodal Fake News Graph Transformer. This component treats news items as nodes and uses an attention mechanism to weigh various relationship types—such as shared authors, locations, or content similarity—to classify news as "Fake" or "Real."
Evaluation and Key Results
The researchers evaluated HML on the FakeSV (short-video) and FVC (Fake Video Corpus) datasets, as well as traditional Twitter and Weibo benchmarks.
- State-of-the-Art Performance: HML achieved 89.14% accuracy on FakeSV and 91.02% on FVC, outperforming previous leaders like MMVD and FakingRecipe by 3-4%.
- The "Plugin" Effect: The Network Inference (NI) module proved highly versatile; when added to older models like EANN or CAFE, it provided an immediate performance boost of up to 10%.
- Superiority over LLMs: In a direct comparison, HML significantly outperformed GPT-4o and Doubao (by 10-15% in accuracy). This suggests that specialized graph-based models are better equipped to handle the structural social context of news dissemination than general-purpose LLMs.
Discussion and Future Work
A key insight of this work is that explicit social graphs are not strictly necessary for effective detection; social influence can be mathematically reconstructed from temporal "heat." Additionally, the self-supervised stage is critical for identifying the subtle cross-modal discrepancies often found in sophisticated fake news.
Despite its success, HML faces challenges in real-time streaming environments due to its batch-processing nature. Future research will focus on improving the computational efficiency of the graph transformer and exploring ways to better "prompt" LLMs to understand the complex multimodal graph structures identified by HML.
Summary updated