TL;DR
Circe is a training-free framework that evaluates the correctness of LLM responses without needing reference answers or multiple samples. By fusing internal signals—specifically how attention and feed-forward components interact across layers—with external likelihood signals, it identifies "confident hallucinations" that traditional metrics often miss.
What Problem Does This Paper Address?
Large Language Models (LLMs) are notorious for their ability to generate fluent, plausible, and highly confident responses that are, unfortunately, completely wrong. Evaluating these responses is difficult in real-world applications where reference answers aren't available.
Existing reference-free methods typically rely on output signals, such as token likelihood, verbalized confidence ("I am 95% sure..."), or agreement across multiple sampled responses. However, as shown in Figure 1, these signals can be deceptive: a model might assign high probability to an incorrect answer or consistently repeat the same error across multiple runs.
While some researchers have turned to the model's internal states (hidden-state trajectories), these methods often treat the Transformer block as a black box, overlooking the specific interactions between its two primary engines: the Attention mechanism and the Feed-Forward Network (FFN).
Methods
The authors introduce Circe, a method that scores response correctness by combining three distinct signals:
- Component Interaction Consistency (CIC): This is the paper's core innovation. It measures the directional alignment between attention updates in earlier layers and the changes those updates induce in FFN outputs in later layers. The intuition is that coherent, correct reasoning involves a "consistent" internal dialogue between these components across the depth of the model.
- Representation Geometry (RG): This characterizes the "path" a response takes through the model's layers by measuring normalized displacement and angular changes in the hidden states.
- Likelihood Confidence (LC): An external signal derived from the mean negative log-likelihood (NLL) of the generated tokens, transformed into a bounded score.
These signals are fused into a single Circe Score. The process is "training-free" because it doesn't require optimizing the LLM's parameters or training a complex external classifier; it simply uses a few fusion weights tuned on a small development set.
Datasets
The authors evaluated Circe across five different LLMs (including Qwen2, Llama3, and Mistral variants) and six benchmarks covering four domains:
- Mathematics: GSM8K and MATH.
- Reasoning: CommonsenseQA and TheoremQA.
- Knowledge: MMLU.
- Understanding: Belebele.
They also tested its generalization capabilities using MGSM (multilingual math) and Geometry3K (multimodal geometry problems involving diagrams).
Key Insights and Ideas
- The "Internal Dialogue" Matters: The most significant finding is that correct responses exhibit higher directional consistency between early attention updates and later FFN changes. If the FFNs in later layers "disagree" with the directions established by earlier attention layers, the response is much more likely to be incorrect.
- Harder Tasks, Stronger Signals: Interestingly, Circe’s performance margin over baselines actually increases on more difficult benchmarks (like MATH vs. GSM8K). The authors suggest that difficult questions elicit more complex internal computations, making the patterns of interaction consistency more pronounced and easier to detect.
- Complementarity is Key: The ablation studies show that while CIC, RG, and LC are all useful, they capture different types of evidence. Fusing them provides a much more robust evaluation than any single signal could achieve alone.
Key Results
- Superior Discrimination: Circe outperformed 12 reference-free baselines across all four domains. It improved the AUROC (Area Under the Receiver Operating Characteristic curve) by an average of 7.37% in math and over 5% in knowledge and understanding tasks compared to the strongest baselines.
- Efficiency: Unlike sampling-based methods (like Self-Consistency) that require generating multiple responses, Circe works with a single generation. Its computational overhead is roughly 18% of the base inference time, making it significantly cheaper than multi-pass methods.
- Robustness: The method proved robust to "data ratio" shifts (varying proportions of correct vs. incorrect answers) and generalized effectively to 11 different languages and multimodal (vision-text) inputs without any additional training.
Limitations and Future Work
- White-Box Requirement: Because Circe relies on internal activations (attention updates and FFN outputs), it cannot be used with black-box API models like GPT-4 or Claude 3.5 unless the providers expose these internal signals.
- Domain-Specific Fusion: While the fusion weights are shared across different models, they are currently domain-specific (e.g., different weights for math vs. general knowledge). Future work could explore more universal fusion strategies.
- Linear Approximation: The theoretical justification for CIC relies on a local linear approximation of FFN maps. While this holds well for smooth activations like SiLU, exploring higher-order interactions could potentially yield even stronger signals.
Summary updated