A quantile network predicts a set of percentiles of an uncertain outcome, such as the 1st, 50th and 99th percentile of tomorrow's electricity load, so that downstream decisions can plan for the bad cases and not only the average. These models are increasingly deployed on constrained hardware, which means they get quantized: their weights and activations are stored in 8-, 4- or 2-bit integers. Mixed-precision quantization gives more bits to the parts of the network that are most sensitive to rounding. The standard way to decide what counts as "sensitive" is to measure how much a single averaged loss would rise. We argue that this averaging systematically starves the extreme percentiles. The tails are exactly where the model has the fewest training examples and exactly where users most need accuracy, yet an averaged sensitivity score is dominated by the dense, easy middle of the distribution.
We propose to treat every (task, quantile level) pair as its own task when allocating bits. Sensitivity is measured separately for each pair on a calibration-relevant scale, and bits are allocated to minimize the worst pair's degradation instead of the average. Concretely, we would build shared-backbone forecasters that predict several targets or horizons at many quantile levels, with non-crossing enforced by construction. We would then show that at an average budget of about 4 bits, worst-pair allocation keeps tail quantiles sharp and correctly positioned where averaged Hessian or divergence-based allocation does not. The comparison always follows a post-hoc recalibration step, because miscalibration alone is cheap to repair and the contest has to be on something recalibration cannot fix.
What the three directions contribute, and what is actually open
Non-crossing quantile methods guarantee that predicted percentiles stay ordered. They do this by predicting a base value plus non-negative increments (I(S)QF, Maddix et al.), increments passed through ELU+1 to avoid adjacent quantiles collapsing onto each other (NQ-Net, Shen et al.), a softmax-then-cumulative-sum head (NC-QR-DQN), or a differentiable sort at the output (SCQRNN, Decke et al.). These papers supply our architecture.
The research file frames the tri-directional gap as "does monotonicity survive quantization?" We think this is mostly the wrong question. If increments are made non-negative and then summed, rounding a non-negative number never makes it negative. Crossing can therefore only appear through an implementation error such as mismatched zero-points across heads, or when the output head is itself unconstrained. The real low-bit failure of incremental heads is the one Shen et al. already identified in full precision: gaps rounding to exactly zero, so distinct percentiles become identical. We will measure crossing and zero-gap rates as a cheap sanity check, not as the headline.
The multi-task lens comes from joint quantile regression in vector-valued RKHSs (Sangnier et al.). That work treats quantile levels as coupled tasks sharing structure, in a kernel model without neural backbones or compression. It also shows that naive post-processing can break the property that the fraction of data below the τ-quantile equals τ. Multi-task training in the neural sense, such as shared backbones with per-task heads and gradient conflict between tasks (RotoGrad), matters here for one reason. Once tasks and quantile levels share layers, bits spent on a shared layer serve all of them, and the allocation becomes a question of whose degradation to minimize.
Mixed-precision allocation supplies the machinery. The options include Hessian-trace sensitivity in NNCF, first-order Taylor loss sensitivity in LBA/DMBQ, and output-divergence or Hessian allocation in a two-task speech pipeline (Mixed Precision DNN Quantization for Overlapped Speech Separation and Recognition). All of these reduce sensitivity to one scalar objective. The closest multi-task quantization work, adaptive scales for multi-task co-training, learns task-adaptive quantization step sizes rather than allocating bitwidths, and it does not handle quantile objectives. None of these works reports how quantization affects individual quantile levels. We found no reported result on Lacuna for quantized quantile regression networks at any bitwidth.
Core Claim
Our hypothesis concerns shared-backbone multi-task quantile networks quantized to roughly 4 bits on average. We expect the dominant error to be displacement of the extreme quantiles of the hardest tasks, not crossing and not degradation of the median. We also expect bit allocations driven by averaged loss sensitivity to be close to blind to this error.
Two mechanisms support the expectation. First, the pinball loss (the standard quantile loss) has an active gradient only on the minority of samples beyond the predicted quantile, a fraction of 1−τ for upper levels. An averaged sensitivity score therefore draws almost all of its signal from central levels. Second, near the true quantile the curvature of the expected pinball loss equals the conditional density of the outcome there. Density is small in the tails, so curvature-based scores label tail-serving parameters as robust, even though a small absolute shift there doubles or halves the rate of exceedances.
Let $\delta_{t,k}(b)$ be the change in empirical coverage at quantile level $\tau_k$ for task $t$ under bit assignment $b=(b_1,\dots,b_L)$ over $L$ layers. Let $r_{t,k}(b)=|\delta_{t,k}(b)|/\min(\tau_k,1-\tau_k)$ be the relative exceedance error, which counts a 0.5-point error at τ=0.99 as seriously as a 25-point error at the median. We allocate bits by solving
$$\min_{b}\ \max_{t,k}\ \sum_{l=1}^{L} s_{t,k}(l,b_l)\quad \text{s.t.}\quad \sum_{l=1}^{L} n_l\, b_l \le B,$$
where $s_{t,k}(l,b_l)$ is the estimated increase in $r_{t,k}$ when layer $l$ alone is quantized to $b_l$ bits, $n_l$ is its parameter count, and $B$ is the total bit budget. The additive-separability assumption is standard in Hessian-based allocation. We test it directly, as described below.
Research Program
The first component is the model family. Each model is a shared encoder, an LSTM or temporal convolution as in LSTM-DLN, feeding per-task incremental quantile heads with ELU+1 gaps over 9 to 23 levels. The levels include 0.01 and 0.99 wherever sample sizes allow. Tasks are forecast horizons and, where available, distinct targets.
The second component is the sensitivity estimator. Per-pair sensitivities are estimated in two ways. The direct way quantizes each layer to each candidate bitwidth and measures coverage shift on a calibration split, which costs $L\times|\text{bitwidths}|$ forward passes and is affordable at this model scale. The cheap way is a first-order estimate in the style of LBA: the gradient of a Huber-smoothed pinball loss for pair $(t,k)$, multiplied by the quantization perturbation and divided by $\min(\tau_k,1-\tau_k)$. For tails, where active samples are scarce, we pool gradients across neighboring levels using the increment structure. Comparing the two estimators tells us whether a cheap proxy is good enough.
The third component is the allocator. The minimax objective above is a small integer program, which we solve exactly or by Lagrangian relaxation. The fourth component is quantization-aware fine-tuning with a straight-through estimator under the chosen allocation, run identically for every allocation policy. We restrict bitwidths to per-layer values in {8, 4, 2} so that the resulting configurations stay executable on runtimes such as NNCF/OpenVINO.
A secondary analysis asks whether per-pair sensitivity vectors disagree mainly in the last shared layers. This multi-task diagnostic would suggest that "tail-serving" capacity is localized. We will report it but do not depend on it.
Baselines
The naive reference points are the full-precision model and uniform INT8 and INT4 post-training quantization. These give the ceiling and the question "is any of this needed?" NNCF reports under 0.5% accuracy drop at INT8 on ImageNet classification (NNCF). We expect INT8 to be near-lossless here too, so our comparisons are made at budgets of 4 bits average and below. If uniform INT4 already preserves tail sharpness, the proposal fails at its premise, and we will say so.
The strongest existing competitors are averaged-sensitivity mixed-precision allocators run at an identical bit budget. We will use a Hessian-trace allocator as in NNCF and the speech-separation work, a KL/output-divergence allocator as in the same speech work and the lossless 4-bit Conformer study, and LBA's first-order criterion (DMBQ/LBA). Each is applied to the sum of pinball losses over all tasks and levels. These methods are the right competitors because they are the established way to spend bits non-uniformly, and the only difference from ours is the aggregation and the scale on which sensitivity is measured. On a two-task speech pipeline, the mixed-precision TCN paper reports lossless accuracy against 32-bit at a 3.3x compression ratio and up to 2.88% absolute WER reduction over uniform baselines. That is the strength of averaged allocation in a multi-task setting. It is not a quantile result, and no quantile-metric number for these allocators exists in the file.
Fairness is controlled in four ways. All policies share the same architecture, training data, calibration split size, fine-tuning epochs and learning-rate sweep. All are subjected to the same post-hoc per-level recalibration: a conformal-style offset per level, applied with an isotonic projection in the style of MultiQT so that ordering is preserved. Performance is compared at matched parameter bits.
Two ablations isolate the key ingredient. The first applies the minimax objective to unnormalized pinball sensitivities, separating the effect of "worst-case over pairs" from the effect of "relative exceedance scale." The second applies the averaged objective to normalized sensitivities.
Beating the baselines means the following. After recalibration, the proposal must show lower tail pinball loss or narrower 98% intervals at equal coverage on the worst task, and lower worst-pair conditional coverage error across input bins. It must also cost at most a small, pre-registered margin in average CRPS (continuous ranked probability score, an overall measure of distributional accuracy), which we set at 1% relative.
Why not just…?
Why not just quantize with any standard allocator and then recalibrate each quantile level post hoc? This is the most serious threat. Per-level offsets repair marginal coverage almost for free, and MultiQT shows that online isotonic tracking drives crossing from 87% of time steps to zero while restoring calibration on the COVID-19 Forecast Hub. Recalibration shifts each quantile by a constant, however. It cannot restore input-dependent sharpness, meaning tails that should be wide in volatile conditions and narrow in calm ones. A quantized backbone that has lost tail-relevant features can only be recalibrated into uniformly wide intervals. For this reason every comparison in this proposal is made after recalibration, on width at matched coverage and on conditional coverage across input bins. If recalibrated averaged allocation matches us on those measures, the idea is refuted.
Why not just keep the quantile heads in full precision? Heads are tiny, so this costs almost nothing, and it removes any head-level rounding or zero-gap risk. We include this as a component of every policy, including the baselines. Our claim is that the tail damage originates in the quantized shared backbone. The "full-precision head, uniform INT4 backbone" configuration therefore serves as an explicit ablation. If it closes the gap, the finding becomes a simpler and still useful deployment rule.
Why not just sort the outputs, or add a crossing penalty? Unconstrained multi-quantile models do cross often: Maddix et al. report crossing in up to 42% of MQ-CNN forecasts, versus 0% for I(S)QF. But crossing is not the problem this proposal addresses. Our heads are monotone by construction, and we expect quantization not to break this. Sorting fixes order, not position, and Sangnier et al. show that post-processing can bias coverage.
Proposed Experiments
The primary benchmarks are the Elec, Traf and Wiki series and M4 used by Maddix et al., with horizons as tasks, and the Norway solar irradiance data from Erdmann et al., with stations and horizons as tasks. The ERCOT 99-level solar forecasts from Ding et al. serve as a stress test of many levels. On the 28-dimensional U-bend and the 20 tabular benchmarks of Sangnier et al., multi-target splits provide non-temporal tasks.
The metrics are per-level coverage and relative exceedance error, tail pinball loss at the 0.01, 0.05, 0.95 and 0.99 levels, interval width at matched coverage, CRPS, weighted quantile loss, and the scaled interval score of Maddix et al. We also report crossing and zero-gap rates and model size in bits.
The first decisive check is whether, on the full-precision model, averaged Hessian scores rank tail-serving layers as low-sensitivity while direct per-pair measurement ranks them high. If the two rankings agree, the premise fails early and cheaply.
The second decisive check sweeps budgets of 6, 4, 3 and 2.5 average bits and plots worst-pair relative exceedance error, after recalibration, against bits for every policy. We expect the gap between policies to open below about 4 bits and to be largest on the tasks with the heaviest tails.
The third check tests the additivity assumption. We compare the predicted degradation of each allocation against the measured degradation. If the correlation is poor, we switch to greedy allocation with re-measurement.
Finally, we run the second check with sensitivities computed from calibration splits of 64, 256 and 1024 series. This tests whether pooling across neighboring levels makes tail sensitivity estimable from small calibration sets, which Liu et al. flag as a known instability of post-training sensitivity metrics (Adaptive Neural Network Quantization For Lightweight Speaker Verification).
References
- Learning Quantile Functions without Quantile Crossing for Distribution-free Time Series Forecasting
- Deep Distributional Learning with Non-crossing Quantile Network
- Non-Crossing Quantile Regression for Distributional Reinforcement Learning
- An Efficient Multi Quantile Regression Network with Ad Hoc Prevention of Quantile Crossing
- Joint quantile regression in vector-valued RKHSs
- RotoGrad: Gradient Homogenization in Multi-Task Learning
- Neural Network Compression Framework for Fast Model Inference
- Distribution-Aware Adaptive Multi-Bit Quantization
- Mixed Precision DNN Quantization for Overlapped Speech Separation and Recognition
- Efficient Low-Bit Quantization with Adaptive Scales for Multi-Task Co-Training
- Multi-Horizon Time Series Forecasting of Non-Parametric CDFs with Deep Lattice Networks
- Lossless 4-bit Quantization of Architecture Compressed Conformer ASR Systems on the 300-hr Switchboard Corpus
- Calibrated Multi-Level Quantile Forecasting
- Adaptive Neural Network Quantization For Lightweight Speaker Verification