Exact Full Conformal Prediction for Quantized Learners

Conformal prediction wraps any model in prediction sets that contain the true answer with a guaranteed probability, say 90%, using only the assumption that data points are exchangeable. Its most data-efficient form, full conformal prediction, has a cost problem. To decide whether a candidate label belongs in the set, it adds that label to the training data, refits the model, and checks how unusual the candidate looks. For a continuous label this means infinitely many refits. The cheap alternative, split conformal prediction, holds out a calibration set and wastes data. Separately, models are increasingly deployed in low-bit integer form (quantization), where weights are rounded onto a coarse grid and accuracy degrades, sometimes sharply below 3 bits.

This note argues that the rounding which causes quantization's accuracy loss also removes full conformal prediction's cost problem. Consider a learner whose output is a quantized model: fit, then round the weights to a $b$-bit grid. As the candidate label $y$ slides along the real line, the rounded model does not change at all except at a finite set of labels where some weight crosses a rounding boundary. Between those breakpoints the deployed model is fixed, and the conformal test can be solved in closed form. Enumerating the breakpoints therefore yields the exact full conformal set, with the finite-sample coverage guarantee intact, for the model that is actually shipped. With fewer bits the grid is coarser, so there are fewer breakpoints and the computation is cheaper. The sets get wider, and that width becomes a label-free, guarantee-backed measure of what quantization cost.

Core Claim

Full conformal validity requires only that the learner be a deterministic function that treats all points of the augmented dataset symmetrically. "Fit ridge or a linearized head on the augmented data, then round to $b$ bits" satisfies this, so the full conformal set computed for the quantized learner has exact $1-\alpha$ marginal coverage. No approximation is involved in the regime where the pre-rounding solution is an affine function of the candidate label.

For a ridge-type head on frozen features, the pre-rounding weights on the augmented data are exactly affine in the candidate label: $\theta(y) = a + y\,v$. Here $a$ and $v$ are vectors computable once from the $n$ training points and the test input. The deployed weights are $\theta_q(y) = s\cdot\mathrm{round}(\theta(y)/s)$, with $s$ the grid step. Coordinate $j$ changes its rounded value only when $a_j + y v_j$ crosses a half-step boundary. Over a label range of width $R$, it therefore contributes about $R|v_j|/s$ breakpoints. Since $s \propto 2^{-b}$, the total breakpoint count scales as $2^{b} R \sum_j |v_j|$, which is the source of the "fewer bits, cheaper exact inference" prediction.

Between consecutive breakpoints, every training residual is constant. The test residual is $|y - x_{n+1}^\top \theta_q|$, which is piecewise linear in $y$. The conformal p-value on each segment is therefore obtained by a rank comparison. Each breakpoint flips one weight by one grid step, so the training residuals update in $O(n)$.

Relation to Prior Work

The motivation comes from two findings. Conformalized Large Language Models under Configuration Shift runs split conformal prediction with calibration on a high-precision model and deployment on a quantized one. Coverage failed its target in 20% of trials under quantization shift, against a baseline below 3%, and larger calibration sets made this worse. That paper stays entirely within split conformal prediction and treats quantization as an external shift on fixed outputs. We instead put the quantizer inside the learner, so that no mismatch between certified and deployed model exists. Its Recal fix needs 50 labeled target points and still leaves 6–8% undercoverage. On the quantization side, Cherry on Top, QuIP and An empirical study of LLaMA3 quantization document the sub-3-bit collapse. They measure it through perplexity and accuracy, not through calibrated uncertainty.

The method borrows from the fast full conformal literature, which approximates refitting in several ways. Approximating Full Conformal Prediction for Neural Network Regression with Gauss-Newton Influence (ACP-GN) linearizes a network and takes one Gauss-Newton step, so residuals become piecewise linear in $y$ with analytic changepoints. We reuse that changepoint logic, but our breakpoints come from rounding boundaries rather than residual sign changes. In the ridge-head regime our computation is exact, while ACP-GN's is an approximation whose coverage can slip. Stable Conformal Prediction Sets and Leave-One-Out Stable Conformal Prediction bound how far one extra point can move a stable learner and build conservative envelopes. Computing Full Conformal Prediction Set with Approximate Homotopy tracks the solution path in $y$.

All of these assume continuous parameters. For a rounded learner this assumption is the problem, not a convenience. A continuous influence step smaller than half a grid step rounds to zero, and a single boundary crossing produces a jump that Lipschitz stability bounds do not describe.

Full Conformal Prediction under Stochastic Non-Conformity Measure constrains the design. Validity survives randomness only under almost-sure permutation invariance. We therefore restrict the core method to deterministic rounding, meaning round-to-nearest with a scale that is itself a symmetric function of the augmented data. Randomized or order-dependent rounding is treated as an extension to be checked, not assumed.

Research Program

The first component is the exact algorithm and its correctness proof for quantized ridge and kernel-ridge heads on frozen features. It covers a fixed grid step and the common max-absolute-value scale. With that scale, $s(y)$ is a maximum of affine functions and is therefore piecewise affine, so the boundaries remain solvable one segment at a time. Ties are frequent because the rounded models are discrete, and we will handle them with a stated conservative tie rule. The range restriction on $y$ will be handled the way grid-based full conformal methods handle it, with the coverage statement made explicit about that range.

The second component is a mixed-precision variant. Following the observation in Cherry on Top that under 1% of weights dominate loss sensitivity, the most sensitive head coordinates are kept in full precision. Those coordinates add no breakpoints and stay exactly affine in $y$, so the method remains exact while accuracy is protected.

The third component extends the method to small neural heads trained on top of the frozen backbone. The continuous path $\theta(y)$ comes from the ACP-GN linearization, followed by rounding-boundary enumeration. This regime is approximate, and we measure its coverage against brute-force refitting rather than claim validity.

The fourth component is empirical. It maps interval width against bit-width and backbone precision, including backbones quantized with QuIP-style incoherence processing and CherryQ-style protection. The aim is to measure quantization loss as an increase in guaranteed-coverage set size.

Baselines

The naive reference point is split conformal prediction on the quantized deployed model, trained on half the data and calibrated on the other half. It is valid by construction, so it isolates what our method buys in efficiency. It gets the same features, bit-width and rounding rule. We beat it if our intervals are shorter at equal coverage, which prior work leads us to expect at small $n$. Approximating Full Conformal Prediction for Neural Network Regression with Gauss-Newton Influence reports that full-data approximations gave tighter intervals than split conformal at 95% target coverage on 12 UCI datasets. No reported interval-length numbers were found on Lacuna for that comparison.

The second baseline follows the configuration-shift protocol. We run exact full conformal prediction on the full-precision head and then deploy the quantized head, which is the setting whose failure Conformalized Large Language Models under Configuration Shift measured at a 20% undercoverage rate for split conformal. We add that paper's Recal, given 50 extra labeled points, and report its 6–8% undercoverage as the bar. We beat both if our undercoverage rate across repeated splits stays near the nominal level below 3%, with no extra labels.

The strongest existing fast full conformal method is ACP-GN. We run it on the latent full-precision head weights and round afterward, with the same features, damping and label range. Its reported speed is 0.03 seconds against more than 16,000 seconds for exact full conformal prediction on synthetic data, a $10^4$ speedup, from the same paper. We also run LOO-StabCP from Leave-One-Out Stable Conformal Prediction, reported as nearly 100 times faster than replace-one stability methods for 100 test points. We beat them on correctness if our sets equal brute force exactly while theirs do not, and on runtime if we stay within a small constant factor of ACP-GN's per-query cost.

Ground truth is brute-force full conformal prediction for the quantized learner, refitting and rounding on a very fine label grid. The key ablation replaces breakpoint enumeration with a uniform label grid of equal compute. This isolates whether exploiting rounding structure, rather than simply refitting a cheap learner, is what gives exactness.

Why not just…?

Why not just use split conformal prediction on the quantized model? It is valid and simple. Its cost is the discarded calibration data, and this cost shrinks as $n$ grows. That is a real limit on our method's value, not something to argue around. We will vary $n$ from tens to thousands on the UCI sets and report where the interval-length gap closes. We expect the method to matter mainly in the small-sample regime targeted by Stable Conformal Prediction Sets, and we will say so if it does not matter beyond it.

Why not just run exact full conformal prediction in full precision and quantize at deployment? At 8 or 4 bits the rounded head may barely differ, and then this is a genuine threat. Zhu's asymmetry result, where calibrating high and deploying low is the damaging direction, suggests the risk concentrates at 2–3 bits. The bit-width sweep settles the question by reporting the undercoverage rate of this baseline at each precision. We claim value only at the bit-widths where it fails.

Why not just apply ACP-GN or a stability bound to the latent weights? For a rounded learner, a small influence step either vanishes under rounding or crosses a boundary and jumps. The linearization and Lipschitz arguments underlying ACP-GN and LOO-StabCP model neither case. Stability envelopes that must absorb one-step weight jumps should be much looser than the continuous ones. The exact-match test against brute force, together with envelope widths, will show how large this gap is.

Why not just grid-search labels with the cheap quantized learner? At very low bit-widths there are few breakpoints, and a fine grid may catch almost all of them. The equal-compute ablation measures exactly this. If the grid matches our sets at 2 bits, the method's contribution there reduces to certifying exactness rather than to speed.

Proposed Experiments

The UCI regression suite (Yacht, Energy, Boston Housing, Diabetes, California Housing and the remaining datasets from the ACP-GN study) uses quantized ridge and small MLP heads at 2, 3, 4 and 8 bits. We report coverage at 90% and 95% over many random splits, mean and variance of interval length, runtime per query, and breakpoint counts.

The first decisive check is that our sets coincide with brute-force refit-and-round sets on every test point in the exact regime. The second is that breakpoint count grows roughly as $2^{b}$, as the counting argument predicts. If it grows much faster, because scale changes cascade, the cost story fails.

On Oxford-IIIT Pet bounding-box regression, a backbone quantized at several bit-widths feeds a quantized regression head. There we compare region volumes against split conformal and split+refine ACP-GN, which the file reports as the smallest-volume baseline; no numeric volumes were found on Lacuna.

To connect to the configuration-shift result, a 4-way probe head on hidden states of a LLaMA model at several precisions is evaluated on MMLU and MedMCQA. Full conformal prediction needs only four refits per query there, and our method is exact. We report undercoverage rate under Zhu's protocol against split conformal calibrated at the other precision and against Recal. The outcome that would sink the proposal is if the full-precision-then-quantize baseline shows no coverage loss at 2–3 bits on these tasks. In that case the practical case for putting the quantizer inside the learner rests on data efficiency alone.