Abstract
This paper introduces Linguistic Fingerprints Extraction (LIFE), a novel framework for detecting LLM-generated fake news. Unlike traditional methods that analyze surface-level text, LIFE identifies internal statistical shifts in word-level probability distributions when an LLM is prompted to act maliciously. By reconstructing these probabilities for key article fragments, the authors uncover "linguistic fingerprints" that reliably distinguish deceptive content from authentic news, achieving state-of-the-art performance across multiple benchmarks.
Introduction
The proliferation of Large Language Models (LLMs) like ChatGPT and LLaMA has made the generation of convincing fake news effortless. Traditional detectors, which rely on static linguistic features, struggle because LLM-generated text is often indistinguishable from human-written content in terms of coherence and style. This research addresses the challenge by investigating whether LLMs exhibit internal statistical differences when generating real versus deceptive content, even when the surface text appears identical.
Methodology
The LIFE framework shifts the focus from what is written to how an LLM generates it. The process involves three primary stages:
- Key Fragment Extraction: Using sentence-masking to isolate high-impact content that most heavily influences classification.
- Reconstruction Probability Acquisition: The LLM calculates word-by-word probabilities for these fragments while guided by a "malicious prompt" (e.g., "Act as a professional fake news writer").
- Sequence Classification: A hybrid CNN-Transformer network processes these probability sequences to classify the article.
Experimental Setup and Datasets
The authors evaluated LIFE across several datasets, including GossipCop++ (GPT-3.5 generated) and four new benchmarks: GenFake-gpt2, GenFake-gptneo, GenFake-gptj, and GenFake-llama2. To test generalization, the system was also evaluated on the human-written Labeled Unreliable News (LUN) dataset.
Results and Analysis
A core finding is that LLMs assign systematically higher generation probabilities to fake news content than to real news when guided by a malicious persona. This suggests a form of internal statistical bias in the model's predictive patterns when handling deceptive narratives.
As shown in the distributions above, the negative log-probabilities for fake news are consistently lower (indicating higher raw probability) than those for real news. LIFE achieved 95.7% accuracy on GenFake-gpt2 and 93.7% on GossipCop++. Notably, it also generalized well to human-written datasets like LUN, achieving 86.8% accuracy. Ablation studies confirmed that the malicious prompt is essential for amplifying the detection signal; removing it leads to a significant performance drop.
Conclusion
While LIFE requires white-box access to LLM token probabilities, its effectiveness highlights a new frontier in AI safety and misinformation detection. By leveraging the model's own internal "cognitive" fingerprints, researchers can bypass the limitations of surface-level text analysis. Future work will explore the detection of hybrid misinformation, where human and AI contributions are intertwined.
Summary updated