FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAG

Maxime Dassen, Rebecca Kotula, Kenton Murray, Andrew Yates, Dawn Lawrie, Efsun Kayi, James Mayfield, Kevin Duh

TL;DR

FACTUM is a new diagnostic framework that detects citation hallucinations in RAG models by analyzing their internal "mechanistic" signals. By monitoring how a model coordinates its "reading" (Attention) and "recalling" (Feed-Forward) pathways, FACTUM outperforms existing black-box and white-box baselines by up to 37.5% in AUC.

What Problem Does This Paper Address?

Retrieval-Augmented Generation (RAG) is the industry standard for grounding LLMs in external data, but it has a "trust" problem: citation hallucination. This occurs when a model makes a claim and cites a source, but that source doesn't actually support the claim.

In long-form RAG (like generating multi-page reports), this becomes even harder to manage due to "attributional drift," where the model loses track of which document provided which fact as the output grows longer. Most current detection methods treat the model as a black box (using another LLM to check the work) or rely on simple uncertainty metrics. The authors argue that citation hallucination isn't just "forgetting" the context; it’s a systemic coordination failure between the model's internal components.

Methods

The authors introduce FACTUM (Framework for Attesting Citation Trustworthiness via Underlying Mechanisms). Instead of looking at the final text, FACTUM probes the residual stream—the internal communication channel of a Transformer—at the exact moment a citation (e.g., "[1]") is generated.

They decompose the model's processing into two primary pathways and measure them using four novel scores:

  1. Contextual Alignment Score (CAS): Measures how much the citation token's internal state aligns with the actual source documents in the attention heads.
  2. Beginning-of-Sentence Attention Score (BAS): Tracks how much the model uses the "attention sink" (the <s> token) to synthesize information before citing.
  3. Parametric Force Score (PFS): Measures the "strength" of the Feed-Forward Networks (FFNs). This represents the model's internal "recall" or parametric knowledge.
  4. Pathway Alignment Score (PAS): A geometric measure of whether the "reading" pathway and "recalling" pathway are pushing the model in the same direction (collaboration) or opposite directions (conflict).

These scores are then fed into interpretable classifiers (like Logistic Regression or LightGBM) to predict if a citation is a hallucination.

Datasets

The study uses the TREC NeuCLIR 2024 dataset, specifically the report generation task.

  • Complexity: Unlike simple Q&A datasets, this requires synthesizing information from 15 different documents into a long-form English report.
  • Models: They evaluated the framework on Llama-3.2-3B-Instruct and Llama-3.1-8B-Instruct.
  • Ground Truth: Labels were generated using a rigorous "LLM-as-a-judge" pipeline (Llama-3.1-70B) validated by human assessors (achieving a high agreement score of $\kappa = 0.68$).

Key Insights and Ideas

  • Parametric Force is Constructive: Conventional wisdom suggests that hallucinations happen because the model relies too much on its internal memory (parametric knowledge) and ignores the context. FACTUM reveals the opposite: correct citations are actually marked by higher parametric force. In healthy RAG, the model uses its internal knowledge to "fact-check" and corroborate the retrieved context.
  • The "Specialization" Shift at Scale: The "signature" of a correct citation changes as models get bigger.
  • In the 3B model, correct citations happen when the Attention and FFN pathways "agree" (high alignment).
  • In the 8B model, the pathways become "orthogonal." They provide distinct, complementary information rather than just repeating each other.
  • The Importance of the Attention Sink: The authors found that the <s> token (attention sink) is vital for information synthesis. Models that "hallucinate" often show a failure to utilize this sink effectively before generating a citation.

Key Results

  • State-of-the-Art Performance: FACTUM significantly outperformed ReDeEP (the previous leading mechanistic baseline) and standard uncertainty metrics like Perplexity and Entropy.
  • AUC Gains: FACTUM achieved a peak AUC of 0.737, representing an improvement of up to 37.5% over baselines.
  • Precision Boost: For the 8B model, FACTUM increased precision from 0.201 to 0.334—a 66% relative improvement—making it far more viable for real-world "guardrail" applications where false alarms are costly.

Limitations and Future Work

  • White-Box Requirement: FACTUM requires access to the model's internal weights and activations. This means it cannot be used on closed-source API models like GPT-4 or Claude without provider cooperation.
  • Token-Level Focus: The current study focuses on the specific citation digit. While this is the "critical link," future work could expand this to analyze the entire sentence leading up to the citation.
  • Architecture Specificity: The framework is built for Transformers. While most LLMs use this architecture, any shift to non-Transformer architectures (like Mamba or State Space Models) would require a redesign of the mechanistic scores.

Summary updated

Abstract

Retrieval-Augmented Generation (RAG) models are critically undermined by citation hallucinations, a deceptive failure where a model cites a source that fails to support its claim. While existing work attributes hallucination to a simple over-reliance on parametric knowledge, we reframe this failure as an evolving, scale-dependent coordination failure between the Attention (reading) and Feed-Forward Network (recalling) pathways. We introduce FACTUM (Framework for Attesting Citation Trustworthiness via Underlying Mechanisms), a framework of four mechanistic scores: Contextual Alignment (CAS), Attention Sink Usage (BAS), Parametric Force (PFS), and Pathway Alignment (PAS). Our analysis reveals that correct citations are consistently marked by higher parametric force (PFS) and greater use of the attention sink (BAS) for information synthesis. Crucially, we find that "one-size-fits-all" theories are insufficient as the signature of correctness evolves with scale: while the 3B model relies on high pathway alignment (PAS), our best-performing 8B detector identifies a shift toward a specialized strategy where pathways provide distinct, orthogonal information. By capturing this complex interplay, FACTUM outperforms state-of-the-art baselines by up to 37.5% in AUC. Our results demonstrate that high parametric force is constructive when successfully coordinated with the Attention pathway, paving the way for more nuanced and reliable RAG systems.
BibTeX
@inproceedings{dassen2026factum,
  author = {Maxime Dassen and Rebecca Kotula and Kenton Murray and Andrew Yates and Dawn Lawrie and Efsun Kayi and James Mayfield and Kevin Duh},
  title = {{FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAG}},
  booktitle = {ECIR},
  year = {2026},
  doi = {10.1007/978-3-032-21289-4\_18},
}
Figure