Abstract
Large Language Models (LLMs) frequently generate confident but factually incorrect "hallucinations," posing significant risks in high-stakes domains. We present LLM Factoscope, a novel diagnostic pipeline that detects hallucinations by monitoring a model's internal "physiological" states rather than relying on external database cross-referencing. By analyzing static neuron activations and the dynamic evolution of token probabilities across layers, Factoscope identifies inherent patterns of factual discernment. Our method achieves over 96% accuracy across various architectures, demonstrating that LLMs possess internal signals of truthfulness that can be tapped to improve AI reliability.
Introduction
The primary challenge in deploying LLMs for medicine, law, or education is their tendency to hallucinate. Current mitigation strategies often involve "external verification"—querying massive databases like Wikipedia. However, this approach is computationally expensive and requires constant maintenance of an external truth source. LLM Factoscope explores a different hypothesis: Does the LLM "know" it is lying? Inspired by human polygraphs, this research measures internal states (neuron firing and logit dynamics) to determine if a model is being factual or not.
Methodology
The LLM Factoscope pipeline extracts and analyzes four distinct types of internal features during text generation:
- Activation Maps (Static): Identification of "factual-related" neurons that fire with higher intensity during correct information retrieval.
- Final Output Ranks (Dynamic): Utilizing the "logit lens" technique to track the rank of the final chosen word as it progresses through the model's layers.
- Top-k Output Indices: Analyzing the semantic similarity of candidate words across layers.
- Top-k Output Probabilities: Measuring the confidence levels of top candidate words.
To process these features, the system employs a Siamese network-based architecture. It uses a ResNet18 CNN to analyze spatial patterns in activations and a Gated Recurrent Unit (GRU) for sequential rank data. The model is trained using Triplet Margin Loss, which mathematically clusters factual internal states together while pushing non-factual states into a separate embedding space.
Key Insights into Factual Discernment
Research reveals that factual discernment is not immediate; it primarily occurs in the middle to later layers (typically after layer 15). In early layers, the model filters for general semantic coherence. As processing continues, factual outputs exhibit high stability: the rank of the correct word rises steadily and remains consistent. In contrast, hallucinations are characterized by wild fluctuations in ranks and probabilities in the final layers, signaling internal uncertainty.
Results and Evaluation
Factoscope was evaluated using a custom dataset of 247,246 data points across categories such as Science, History, and Geography. Testing across architectures including GPT2-XL, Llama2 (7B/13B), and Vicuna yielded the following results:
- High Accuracy: The system consistently achieved between 96.1% and 98.3% accuracy in hallucination detection.
- Generalization: A Factoscope trained on history and science data successfully detected hallucinations in unrelated fields like sports, indicating a universal internal signal for factuality.
- Performance: It significantly outperformed existing white-box (SAPLMA) and black-box (SelfCheckGPT) baselines, particularly as model complexity increased.
Conclusion
LLM Factoscope demonstrates that LLMs possess an internal "factual discernment" that can be measured without external tools. While the method assumes the model's training data was predominantly factual, it opens new avenues for real-time model editing and the development of more trustworthy AI systems. Future work will focus on refining these detectors for highly technical, out-of-distribution jargon.
Summary updated