Tight PAC-Bayes Generalisation Guarantees for Large Language Model Judges

Tom A. Lamb, Philip H. S. Torr, Tim G. J. RudnerarXiv 2026 · arXiv

As Large Language Models (LLMs) move from simple chatbots to autonomous agents in safety-critical domains, we face a "Who watches the watchmen?" problem. We increasingly use "LLM Judges" (guardrail models) to monitor other AI systems, but how can we be certain these judges themselves are reliable?

In their recent paper, "Tight PAC-Bayes Generalisation Guarantees for Large Language Model Judges," researchers from Oxford, Toronto, and the Vector Institute provide a mathematical framework to answer this question. They demonstrate that we can obtain formal, "non-vacuous" (meaningful) safety guarantees for LLM judges without needing to set aside precious labeled data for validation.

TL;DR

The paper introduces a practical PAC-Bayes framework to certify the reliability of LLM safety judges. By using a novel, highly compressible fine-tuning method called LoRA-GT, the authors achieve tight generalisation bounds for both classification accuracy and uncertainty calibration, even on datasets with fewer than 1,000 samples.

What Problem Does This Paper Address?

LLM judges like Llama Guard are widely used to block harmful outputs, but they often lack formal guarantees. Standard statistical methods for providing such guarantees usually require a large "held-out" validation set. In the world of AI safety, high-quality labeled data (e.g., examples of successful jailbreaks or subtle hallucinations) is often scarce and expensive.

Furthermore, traditional generalisation bounds for deep learning are often "vacuous"—they might mathematically prove that an error rate is less than 100%, which is technically true but practically useless. This paper seeks to bridge the gap between theoretical learning theory and the practical need for certifiable AI safety.

Methods

The authors leverage PAC-Bayes theory, which relates a model's empirical performance on training data to its "complexity." In this context, complexity is measured by description length: how many bits are required to store the model's learned parameters.

Key Innovations:

  1. LoRA-GT (Global-scale Tied LoRA): Standard Parameter-Efficient Fine-Tuning (PEFT) like LoRA uses many "scale factors" (metadata) that add significant overhead to the description length. LoRA-GT shares a single quantization scale across all layers, drastically reducing the bits needed for certification without hurting performance.
  2. Compression-Based Certification: The judge is trained normally, then its low-rank adapters are quantized and compressed using the Brotli algorithm. This compressed size is plugged into a refined Catoni-style PAC-Bayes bound to calculate a "certificate"—a formal upper limit on the judge's risk in deployment.
  3. Brier Risk Certification: Beyond simple 0-1 classification (safe/unsafe), the authors certify Brier risk, which measures how well-calibrated the judge's probability estimates are. This is crucial for systems that need to "escalate" to a human when the AI is uncertain.
Figure 1: Compression yields tight, non-vacuous PAC-Bayes guarantees for safety oversight models. Left: LoRA-T with per-tensor versus shared global scaling (LoRA-GT), which removes the per-layer quantisation-scale metadata that dominates PEFT adaptor description lengths under aggressive compression. Right: certified bounds and test risk on BeaverTails under 0–1 and Brier losses, where LoRA-GT achieves the tightest certificates among the shown adaptors while retaining competitive test risk.

Datasets

The framework was evaluated across five guardrail models (Llama Guard 3 and Qwen3Guard variants) on four diverse benchmarks:

  • Egida: Jailbreak attempts against safety-aligned models ($n=792$).
  • OpenAI Content Moderation: Standard moderation prompts ($n=1,344$).
  • Aegis: Joint evaluation of user prompts and agent responses ($n=28,984$).
  • BeaverTails: A large-scale safety alignment dataset ($n=300,567$).

Key Insights and Ideas

  • Complexity > Scale: Surprisingly, the tightness of the safety certificate is governed more by the complexity of the adaptation (how much you changed the model) than the scale of the base model (1B vs 8B parameters). This suggests that simple adaptations to capable base models are a winning strategy for certifiable safety.
  • Certification on Small Data: The method yields meaningful bounds even on the Egida dataset ($n=792$). This is a major win for safety researchers who often work with small, specialized datasets.
  • The Quantization Sweet Spot: There is a trade-off between aggressive quantization (which shortens the description and tightens the bound) and model error (which increases as you lose precision). The authors show that for small datasets, 2-bit quantization is often optimal for the tightest certificate.

Key Results

The authors achieved "non-vacuous" bounds across all 40 model-dataset-loss combinations tested.

  • Tightness: On the BeaverTails dataset, the certified 0-1 risk bound for LoRA-GT was as low as 0.169, very close to the actual test risk.
  • LoRA-GT Supremacy: LoRA-GT consistently provided the tightest certificates compared to standard LoRA or other PEFT variants, often by a wide margin.
  • Calibration: The Brier risk certificates provide the first formal generalisation guarantees for the predictive uncertainty of LLM judges.
Figure 2: Tight non-vacuous guarantees for safety oversight models. Certified bounds and test risk for 0–1 and Brier losses across models, datasets, and adaptor families, showing the tightest certificate within each family after accounting for the finite selection cost (Appendix C.3). LoRA, whose bounds are vacuous, is shown at its lowest test risk. Highly compressed adaptors achieve tight non-vacuous guarantees at a small cost in test risk.

Limitations and Future Work

  • Distribution Shift: The certificates are valid only if the "generator" (the model being judged) and the prompt distribution remain fixed. If the underlying AI agent is updated, the judge must be re-certified.
  • Model Scale: The study focused on models up to 8B parameters. Extending these tight bounds to 70B+ parameter models remains an open challenge due to the increased "functional distortion" caused by quantization at that scale.
  • Adversarial Robustness: While the i.i.d. assumption allows for adversarial prompts in the training/cert sample, it does not account for an adaptive attacker who specifically targets the certified judge.

By turning model compression into a tool for mathematical proof, this work provides a practical roadmap for building AI oversight systems that we can actually trust.

Summary updated

Abstract

Large language model (LLM) judges are increasingly used to monitor the outputs of LLM-based AI agents, yet formal guarantees on the reliability of these judges remain largely absent, limiting confidence in their use for safety-critical oversight. We consider certifying such a judge and define its risk over safety classifications of prompt--response pairs drawn from a deployed generator. We apply a PAC-Bayes framework to this setting, refining existing compression-based methodology with the use of tighter bounds and more efficient coding. We show that standard parameter-efficient fine-tuning can yield non-vacuous and even tight certificates for safety judges, for both classification and Brier risk, without holding out data for validation. We introduce LoRA-GT, which shares a single quantisation scale across layers and achieves the tightest certificates with minimal degradation in predictive performance. Across five guardrail models and four benchmarks, we find that certificate tightness is governed more by adaptation complexity than by base model scale. Finally, we show how far quantisation can be pushed before the resulting increase in risk outweighs the bound tightening gained from shorter descriptions. Together, these results provide a practical route toward certifiable scalable oversight and offer insight into how the complexity of safety adaptations shapes generalisation.
BibTeX
@misc{lamb2026tight,
  author = {Tom A. Lamb and Philip H. S. Torr and Tim G. J. Rudner},
  title = {{Tight PAC-Bayes Generalisation Guarantees for Large Language Model Judges}},
  year = {2026},
  eprint = {2611.20818},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20818},
}
Figure