Abstract
Large Language Models (LLMs) frequently produce "hallucinations"—outputs that are fluent but factually incorrect. For developers utilizing black-box APIs, managing these errors is difficult due to the lack of access to internal model probabilities. This paper presents a framework for Unsupervised Conformal Prediction (UCP) that leverages the geometric properties of response embeddings to create calibrated "hallucination gates." By introducing a novel Batch Bootstrap method (BB-UCP) and a conformal alignment technique, the authors provide a mathematically rigorous way to guarantee that filtered response batches meet specific quality thresholds without requiring ground-truth labels at test-time.
Introduction
The deployment of LLMs in high-stakes environments is hindered by their tendency to generate incorrect information with high confidence. Traditional uncertainty quantification often relies on internal logits or gradients, which are unavailable in popular black-box models like GPT-4. While heuristics such as self-consistency and semantic entropy offer some signal, they lack formal statistical coverage guarantees.
Conformal Prediction (CP) typically provides such guarantees (e.g., "the true answer is within this set with 95% probability") but requires labeled calibration data. This research addresses the challenge of performing conformal inference in an unsupervised regime, where ground-truth labels are absent, enabling reliable filtering for generative AI applications.
Methodology
The proposed framework consists of three core components designed to transform geometric signals into statistical gates:
- Atypicality Score: The system analyzes the geometry of multiple sampled responses in an embedding space. By computing a Gram matrix of inner products, it identifies "central" responses with high interaction energy. Responses that appear as outliers are flagged as "atypical," serving as a proxy for potential hallucinations.
- Batch Bootstrap UCP (BB-UCP): To improve data efficiency and stability over standard split-conformal methods, the authors introduce a bootstrapping variant. BB-UCP aggregates residuals across batches, smoothing uncertainty thresholds and producing tighter prediction intervals while maintaining distribution-free guarantees.
- Conformal Alignment: This technique allows users to calibrate a "strictness knob" ($\tau$) using a small amount of historical data. This alignment ensures that a desired objective, such as a specific "factuality lift," is maintained on new batches with a user-defined probability.
Experimental Evaluation
The framework was tested across four diverse Question-Answering (QA) benchmarks: ASQA (ambiguity), NQ-Open (factoid retrieval), HotpotQA (multi-hop reasoning), and AmbigQA (multiple interpretations). The researchers also conducted stress tests involving entropy manipulations and model swaps to verify robustness.
Coverage and Efficiency
A primary goal of conformal prediction is achieving "nominal coverage"—ensuring the empirical error rate matches the user-defined tolerance. As shown in Figure 1, the BB-UCP method consistently meets target coverage across datasets. Furthermore, it produces significantly smaller interval lengths compared to standard split-UCP, indicating higher efficiency in identifying valid responses.
Factuality Alignment
The "alignment" component was evaluated by its ability to reduce hallucination severity. In cross-query alignment experiments, the system demonstrated consistent gains in factuality. On the NQ-Open dataset, the method achieved a median factuality improvement ($\Delta FS$) of approximately 0.209, outperforming simpler per-response detectors.
Discussion and Limitations
The "aha moment" of this work is that the collective shape of LLM responses in embedding space serves as a powerful signal for uncertainty. Bootstrapping provides the necessary stability for real-world deployment where API calls are expensive and samples are limited.
However, several limitations remain. The framework relies on the exchangeability assumption, meaning significant shifts in prompt distributions or model updates could invalidate the guarantees. Additionally, performance is sensitive to the quality of the underlying embedding model. In cases of "consensus collapse"—where a model repeatedly generates the same incorrect answer—the lack of dispersion may reduce the effectiveness of the bootstrap. Future work aims to extend these methods to multimodal settings and develop online recalibration techniques for dynamic data environments.
Summary updated