QuadCal: Calibration for In-Context Learning
Its authors withdrew this paper, so there is no full text to read or summarize.
Lists
Only you see this
Abstract
Large language models (LLMs) are increasingly being applied to high-stakes domains with high consequences for errors such as healthcare, drug discovery, law, and finance. However, they are often unstable and highly sensitive to prompt design, which can introduce contextual bias into their predictions. To mitigate this bias, various calibration methods have been developed to prevent overconfident and incorrect predictions. Existing techniques are either confidence-based, relying on heuristics to quantify bias, or likelihood-based, which is theoretically grounded but introduces unnecessary computational overhead. In this work, we introduce QuadCal, a novel supervised likelihood-based calibration method that is up to 40% faster and outperforms the existing likelihood-based approach. Specifically, QuadCal leverages Quadratic Discriminant Analysis (QDA), a supervised algorithm that directly models class-conditioned distributions, making it more efficient. We evaluated calibration methods on GPT-2 models and the more recent Llama and Gemma’s instruction-tuned (IT) models, which are harder to calibrate. Empirically, we show that on average over seven different natural language classification datasets, QuadCal outperforms existing methods on GPT-2 models and is competitive with earlier methods on IT models.
BibTeX
@misc{viswanathan2026quadcal,
author = {Janaki Viswanathan and Nick Howell and Dietrich Klakow},
title = {{QuadCal: Calibration for In-Context Learning}},
year = {2026},
url = {https://openreview.net/forum?id=Rd1bvPbohN},
howpublished = {OpenReview},
note = {Submitted to ICLR 2026},
}Loading related content...