Unsupervised Conformal Inference: Bootstrapping and Alignment to Control LLM Uncertainty

Lingyou Pang, Lei Huang, Jianyu Lin, Tianyu Wang, Akira Horiguchi, Alexander Aue, Carey E. Priebe

Abstract

Large Language Models (LLMs) frequently produce "hallucinations"—outputs that are fluent but factually incorrect. For developers utilizing black-box APIs, managing these errors is difficult due to the lack of access to internal model probabilities. This paper presents a framework for Unsupervised Conformal Prediction (UCP) that leverages the geometric properties of response embeddings to create calibrated "hallucination gates." By introducing a novel Batch Bootstrap method (BB-UCP) and a conformal alignment technique, the authors provide a mathematically rigorous way to guarantee that filtered response batches meet specific quality thresholds without requiring ground-truth labels at test-time.

Introduction

The deployment of LLMs in high-stakes environments is hindered by their tendency to generate incorrect information with high confidence. Traditional uncertainty quantification often relies on internal logits or gradients, which are unavailable in popular black-box models like GPT-4. While heuristics such as self-consistency and semantic entropy offer some signal, they lack formal statistical coverage guarantees.

Conformal Prediction (CP) typically provides such guarantees (e.g., "the true answer is within this set with 95% probability") but requires labeled calibration data. This research addresses the challenge of performing conformal inference in an unsupervised regime, where ground-truth labels are absent, enabling reliable filtering for generative AI applications.

Methodology

The proposed framework consists of three core components designed to transform geometric signals into statistical gates:

  1. Atypicality Score: The system analyzes the geometry of multiple sampled responses in an embedding space. By computing a Gram matrix of inner products, it identifies "central" responses with high interaction energy. Responses that appear as outliers are flagged as "atypical," serving as a proxy for potential hallucinations.
  2. Batch Bootstrap UCP (BB-UCP): To improve data efficiency and stability over standard split-conformal methods, the authors introduce a bootstrapping variant. BB-UCP aggregates residuals across batches, smoothing uncertainty thresholds and producing tighter prediction intervals while maintaining distribution-free guarantees.
  3. Conformal Alignment: This technique allows users to calibrate a "strictness knob" ($\tau$) using a small amount of historical data. This alignment ensures that a desired objective, such as a specific "factuality lift," is maintained on new batches with a user-defined probability.

Experimental Evaluation

The framework was tested across four diverse Question-Answering (QA) benchmarks: ASQA (ambiguity), NQ-Open (factoid retrieval), HotpotQA (multi-hop reasoning), and AmbigQA (multiple interpretations). The researchers also conducted stress tests involving entropy manipulations and model swaps to verify robustness.

Coverage and Efficiency

A primary goal of conformal prediction is achieving "nominal coverage"—ensuring the empirical error rate matches the user-defined tolerance. As shown in Figure 1, the BB-UCP method consistently meets target coverage across datasets. Furthermore, it produces significantly smaller interval lengths compared to standard split-UCP, indicating higher efficiency in identifying valid responses.

Empirical coverage and interval length across datasets comparing BB-UCP (red) and split-UCP (blue). BB-UCP achieves target coverage with tighter intervals.

Factuality Alignment

The "alignment" component was evaluated by its ability to reduce hallucination severity. In cross-query alignment experiments, the system demonstrated consistent gains in factuality. On the NQ-Open dataset, the method achieved a median factuality improvement ($\Delta FS$) of approximately 0.209, outperforming simpler per-response detectors.

Empirical coverage and median factuality improvement (ΔFS) across datasets, demonstrating the effectiveness of the alignment framework.

Discussion and Limitations

The "aha moment" of this work is that the collective shape of LLM responses in embedding space serves as a powerful signal for uncertainty. Bootstrapping provides the necessary stability for real-world deployment where API calls are expensive and samples are limited.

However, several limitations remain. The framework relies on the exchangeability assumption, meaning significant shifts in prompt distributions or model updates could invalidate the guarantees. Additionally, performance is sensitive to the quality of the underlying embedding model. In cases of "consensus collapse"—where a model repeatedly generates the same incorrect answer—the lack of dispersion may reduce the effectiveness of the bootstrap. Future work aims to extend these methods to multimodal settings and develop online recalibration techniques for dynamic data environments.

Summary updated

Abstract

Deploying black-box LLMs requires managing uncertainty in the absence of token-level probability or true labels. We propose introducing an unsupervised conformal inference framework for generation, which integrates: generative models, incorporating: (i) an LLM-compatible atypical score derived from response-embedding Gram matrix, (ii) UCP combined with a bootstrapping variant (BB-UCP) that aggregates residuals to refine quantile precision while maintaining distribution-free, finite-sample coverage, and (iii) conformal alignment, which calibrates a single strictness parameter $\tau$ so a user predicate (e.g., factuality lift) holds on unseen batches with probability $\ge 1-\alpha$. Across different benchmark datasets, our gates achieve close-to-nominal coverage and provide tighter, more stable thresholds than split UCP, while consistently reducing the severity of factuality severity, outperforming lightweight per-response detectors with similar computational demands. The result is a label-free, API-compatible gate for test-time filtering that turns geometric signals into calibrated, goal-aligned decisions.
BibTeX
@misc{pang2025unsupervised,
  author = {Lingyou Pang and Lei Huang and Jianyu Lin and Tianyu Wang and Akira Horiguchi and Alexander Aue and Carey E. Priebe},
  title = {{Unsupervised Conformal Inference: Bootstrapping and Alignment to Control LLM Uncertainty}},
  year = {2025},
  eprint = {2509.23002},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.23002},
}
Figure