TL;DR
Detecting fake news on platforms like TikTok and Instagram is difficult because researchers often cannot see the "social cascades" (who shared what) that traditional detection methods rely on. This paper introduces HML, a framework that reconstructs these hidden social connections using temporal patterns and multimodal content, outperforming state-of-the-art models and even large language models (LLMs) in identifying sophisticated fake news.
What Problem Does This Paper Address?
Most fake news detection research has focused on platforms like Twitter (X) or Weibo, where "retweet" and "comment" chains are visible. These cascades provide a structural "fingerprint" of how news spreads. However, on modern video-centric platforms like TikTok or YouTube, these relationships are often obscured by platform mechanics or privacy settings.
Furthermore, fake news has evolved from simple text to "complex heterogeneous multimodal" content—think deepfakes, edited audio, and misleading video captions. Existing models often just "splice" these different data types together without understanding the deep relationships between them or the social context in which they exist. The authors aim to bridge this gap by inferring a latent social network where an explicit one is missing.
Methods
The HML (Complex Heterogeneous Multimodal Fake News Detection via Latent Network Inference) framework operates in three main stages:
- Social Latent Network Inference (Stage 1): The authors use an improved Hawkes Process (a self-exciting point process) to model how news items influence each other over time. Even without a "retweet" button, they can estimate the "maximum likelihood" of influence based on when news about a specific event appears. They also introduce Gaussian noise to simulate how older news is "forgotten" by the public over time.
- Self-supervised Multimodal Content Learning (Stage 2): To handle text, audio, and video, the model uses a "Pruning and Mask" strategy to augment data. It employs contrastive learning to align these modalities—ensuring that the features extracted from a video's audio track, its frames, and its description are consistent and robust against "noisy" or imbalanced data.
- Network-Augmented Graph Transformer (Stage 3): Finally, the inferred social relationships and the refined multimodal features are fed into a Heterogeneous Multimodal Fake News Graph Transformer. This model treats news items as nodes in a graph, using an attention mechanism to weigh different types of relationships (e.g., same author, same location, or similar content) to make a final "Fake" or "Real" classification.
Datasets
The researchers evaluated HML on several key benchmarks:
- FakeSV: A comprehensive short-video fake news dataset.
- FVC (Fake Video Corpus): A standard dataset for video-based rumor detection.
- Twitter & Weibo: Used to demonstrate that their "Network Inference" module can be used as a "plugin" to improve detection even on platforms where social data is already available.
Key Insights and Ideas
- Inferring the Invisible: The biggest "aha" moment is that you don't actually need a retweet graph to understand social influence. By modeling the temporal "heat" and popularity of an event, the model can mathematically reconstruct the likely connections between news items.
- LLMs Aren't Enough (Yet): The study compared HML against GPT-4o and Doubao. Surprisingly, the LLMs performed significantly worse (often by 10-15% in accuracy). This is because LLMs struggle to grasp the structural social context and the specific distribution of news dissemination that a specialized graph-based model can capture.
- Cross-Modal Discrepancy: Fake news often has "cracks" where the audio doesn't perfectly match the video or the text. The self-supervised contrastive learning stage is specifically designed to catch these fine-grained inconsistencies.
Key Results
- State-of-the-Art Performance: HML achieved 89.14% accuracy on the FakeSV dataset and 91.02% on the FVC dataset, outperforming previous leaders like MMVD and FakingRecipe by roughly 3-4%.
- The "Plugin" Effect: When the authors added their Network Inference (NI) module to older, established models (like EANN or CAFE), those models saw an immediate performance boost of up to 10%.
- Ablation Success: Removing the "Stage 1" latent network inference caused accuracy to drop by about 8%, proving that the social structure is just as important as the content itself for detection.
Limitations and Future Work
While HML is powerful, the authors note a few areas for growth:
- LLM Adaptation: Since LLMs currently struggle with this task, the authors suggest future work should focus on how to better "prompt" or fine-tune LLMs to understand multimodal graph structures.
- Real-time Constraints: The paper focuses on batch processing of datasets; applying this latent inference in a real-time "streaming" environment where news is breaking by the second remains a challenge.
- Computational Complexity: Building and transforming heterogeneous graphs for millions of news items requires significant GPU resources (the study used 4x NVIDIA RTX A6000s). Improving the efficiency of the graph transformer for larger scales is a likely next step.
Summary updated