EigenTrack: Spectral Activation Feature Tracking for Hallucination and Out-of-Distribution Detection in LLMs and VLMs

Davide Ettori, Nastaran Darabi, Sina Tayebati, Ranganath Krishnan, Mahesh Subedar, Omesh Tickoo, Amit Ranjan Trivedi

TL;DR

This paper introduces EigenTrack, a novel real-time detector for LLM/VLM hallucinations and OOD errors. It leverages the spectral properties of hidden layer activations, tracked over time, to identify model instability before it manifests as output errors, offering a more interpretable and efficient alternative to existing methods.

What Problem Does This Paper Address?

Large language models (LLMs) and vision-language models (VLMs) are increasingly powerful but suffer from two critical failure modes: hallucination (generating factually incorrect or nonsensical information) and out-of-distribution (OOD) errors (performing poorly when encountering inputs significantly different from their training data). Detecting these failures is crucial for deploying these models in high-stakes applications. Previous methods often relied on surface-level signals (like confidence scores), required multiple generations (increasing latency), or only analyzed static snapshots of the model's internal state. This left a gap for a method that could efficiently and reliably detect these failures in real-time by looking at the dynamics of the model's internal representations.

Methods

EigenTrack's core innovation lies in its temporal spectral analysis of hidden activations. Here's how it works:

  1. Sliding-Window Activation Aggregation: The method monitors hidden activations from selected transformer layers. It then forms a matrix by stacking the activations from the most recent N decoding steps. This captures both cross-layer and short-term temporal information.
  2. Spectral Feature Extraction: From these sliding-window activation matrices, EigenTrack extracts a set of spectral features. These features are derived from the covariance spectrum and are designed to capture the "geometry" of the representation space. Key features include:
  3. Leading eigenvalues: Indicate the strength of dominant representation directions.
  4. Spectral gaps: Measure the separation between eigenvalues, signaling structural changes.
  5. Entropy: Quantifies the dispersion of the spectrum.
  6. Random Matrix Theory (RMT) features: Specifically, divergence from the Marchenko-Pastur (MP) law, which serves as a baseline for random noise. Deviations from this baseline suggest structured, non-random behavior.
  7. Recurrent Spectral Discrepancy Detector: The sequence of extracted spectral feature vectors is fed into a lightweight recurrent neural network (RNN, GRU, or LSTM). This classifier models the temporal evolution of these spectral signatures, learning to identify patterns indicative of hallucination or OOD behavior. The recurrent nature allows it to capture how these spectral properties change over time, providing an early warning.

This approach is novel because it combines RMT-grounded spectral analysis (providing an interpretable link to noise vs. structure) with temporal modeling, operating on a single forward pass without resampling.

Datasets

The paper evaluates EigenTrack on several benchmarks:

  • Hallucination Detection:
    • HaluEval: A benchmark derived from HotpotQA, where factual and hallucinated responses are generated by carefully constructing queries. A novel data-generation pipeline using multiple LLMs is introduced to create these labeled datasets.
  • Out-of-Distribution (OOD) Detection:
    • WebQuestions: Used as the in-distribution dataset.
  • Eurlex: A legal-domain question dataset used as the OOD set, representing a clear domain shift.
  • Multimodal Data: For VLMs like LLaVa, textual prompts are augmented with images from Flickr8k.

Key Insights and Ideas

  • Spectral Geometry as a Proxy for Model Stability: The core idea is that the "shape" or "geometry" of the hidden layer activations, as captured by their spectral properties, changes predictably when a model is about to hallucinate or encounter OOD data. This geometry shifts towards a more "noisy" or less structured state.
  • RMT as a Principled Noise Baseline: Using Random Matrix Theory (specifically the Marchenko-Pastur law) provides a theoretically grounded way to define what "random noise" looks like in terms of eigenvalue distributions. Deviations from this baseline are strong indicators of meaningful structure or, conversely, a loss of structure.
  • Temporal Dynamics are Crucial for Early Detection: Analyzing spectral features over time is key. Hallucinations and OOD issues don't appear instantaneously; they often manifest as a gradual degradation of representation structure. EigenTrack's recurrent classifier captures this evolution, enabling detection before the final output is generated.
  • Interpretability Through Spectral Features: The use of specific spectral features (like KL divergence from MP, leading eigenvalues, spectral gaps) provides a degree of interpretability. These features have theoretical grounding and their behavior can be linked to specific types of model instability.
  • Compactness and Efficiency: EigenTrack achieves strong results with a lightweight recurrent head, making it computationally efficient and suitable for real-time applications, a significant advantage over methods requiring multiple generations or complex analysis.

Key Results

  • High AUROC Scores: EigenTrack consistently achieves high AUROC values for both hallucination detection (0.82-0.94) and OOD detection (0.85-0.96) across various LLMs (LLaMa, Qwen, Mistral) and VLMs (LLaVa).
  • State-of-the-Art Performance: On LLaMa-7B, EigenTrack achieves 0.89 AUROC for hallucination and 0.92 for OOD, outperforming established baselines like HaloScope, LapEigvals, and SelfCheckGPT.
  • GRU Superiority: Gated Recurrent Units (GRUs) consistently perform best among the tested recurrent architectures, highlighting the benefit of gated memory for tracking spectral dynamics.
  • Model Scale Matters: Performance generally improves with model size, likely due to sharper and more stable spectral signatures in larger models.
  • Early Detection Capability: Spectral signatures indicative of failure emerge early in the generation process, allowing for early termination of failing generations and significant computational savings.
  • Accuracy-Latency Trade-off: The method offers tunable parameters (e.g., sliding-window length, response length) that allow users to balance detection accuracy with inference latency.

Limitations and Future Work

  • Hyperparameter Tuning: EigenTrack has several tunable parameters (monitored layers, window size, feature count, recurrent hidden size) that influence the accuracy-latency trade-off. While ablation studies provide guidance, optimal settings may vary.
  • Multimodal Extension: While the framework naturally extends to multimodal architectures, deeper analysis of multimodal specific dynamics is suggested for future work.
  • Theoretical Guarantees: The authors suggest exploring tighter theoretical guarantees for the spectral features and their link to failure modes.
  • Adaptive Strategies: Future work could involve developing adaptive correction strategies and automatic selection of spectral features and window sizes.
  • Broader Applicability: Investigating the application of EigenTrack to other types of model failures or different model architectures is also a potential avenue.

Summary updated

Abstract

Large language models (LLMs) offer broad utility but remain prone to hallucination and out-of-distribution (OOD) errors. We propose EigenTrack, an interpretable real-time detector that uses the spectral geometry of hidden activations, a compact global signature of model dynamics. By streaming covariance-spectrum statistics such as entropy, eigenvalue gaps, and KL divergence from random baselines into a lightweight recurrent classifier, EigenTrack tracks temporal shifts in representation structure that signal hallucination and OOD drift before surface errors appear. Unlike black- and grey-box methods, it needs only a single forward pass without resampling. Unlike existing white-box detectors, it preserves temporal context, aggregates global signals, and offers interpretable accuracy-latency trade-offs.
BibTeX
@misc{ettori2025eigentrack,
  author = {Davide Ettori and Nastaran Darabi and Sina Tayebati and Ranganath Krishnan and Mahesh Subedar and Omesh Tickoo and Amit Ranjan Trivedi},
  title = {{EigenTrack: Spectral Activation Feature Tracking for Hallucination and Out-of-Distribution Detection in LLMs and VLMs}},
  year = {2025},
  eprint = {2509.15735},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.15735},
}

Loading related content...

Figure