Neural network controllers for drones and robots increasingly need to run on small onboard chips, which means storing their weights in 2 to 4 bits instead of 16 or 32. Rounding the weights makes the controller output slightly wrong actions. Inside a feedback loop those small errors add up over time and can push the robot across a safety boundary.
The textbook fix treats the rounding error as a worst-case disturbance and adds a large safety buffer. That buffer is usually so large that the low-bit controller becomes useless, because a safety layer overrides it almost all the time. The usual alternative is to try many bit-width settings and roll each one out in closed loop, which is an expensive search over settings that are not learned during training (hyperparameters).
This note proposes a third route. Unlike wind or payload changes, quantization error is not an unknown external force. It is a fixed function of the state, $\pi_q(x) - \pi(x)$, where $\pi$ is the full-precision policy and $\pi_q$ its quantized copy. Its distribution can be measured almost for free by running both copies on the same logged states.
We use that measurement for two jobs. First, it sets a distribution-free safety margin through conformal prediction, a method that turns held-out errors into a bound that holds with a chosen probability. Second, the same measurement chooses per-layer bit-widths in a single pass, with no closed-loop sweep. An online feedback rule then corrects the margin when the quantized controller drifts into states the calibration never saw. The claim is that this recovers most of the memory savings of 2–3-bit controllers while keeping the long-run violation rate at a user-chosen level, and that it picks configurations comparable to an exhaustive search at a small fraction of its rollout cost.
Core Claim
The three directions in the file meet at one quantity: the margin a safety layer adds to absorb quantization error.
- Worst-case robust control sets that margin from an absolute bound. This is the conservatism the robust-control literature is trying to escape.
- Hyperparameter search sets the bit-widths behind that margin by trial and error.
- Quantization research measures accuracy loss only on static, open-loop benchmarks. QuIP and AlphaQ report perplexity and zero-shot accuracy, with no closed-loop safety evaluation.
The proposal makes the margin a calibrated statistical object instead of a worst-case one. Because the error source is fully observable offline, the same calibration also answers the bit-allocation question.
The closest prior work does not do this. Disturbance-Adaptive Data-Driven Predictive Control uses split conformal prediction and a violation-feedback loop to shrink building-control safety buffers. It reports 11–30% energy savings at a 5% violation target. However, it handles only external disturbances that must be learned from observed violations, and it runs in full precision.
Safe Perception-Based Control with Minimal Worst-Case Dynamic Regret treats neural network output error as a bounded disturbance, but in the worst-case style we argue is too conservative here. It also does not consider quantization.
Our step is to recognize that quantization error is a disturbance with a free oracle. That makes calibration cheap, lets it be done per layer, and lets the same signal drive bit allocation.
Research Program
System. A neural policy is trained in full precision; for some experiments it is an actor-critic policy in the spirit of Actor–Critic Model Predictive Control. The policy is then quantized post-training. Its actions pass through a model predictive safety filter: a short-horizon optimizer that minimally changes the proposed action so that predicted states stay inside constraints that have been tightened by a margin.
Margin. The margin at horizon step $k$ is
$$ m_k = b_{\text{phys},k} + L_k \, \hat{q}_{1-\alpha}, $$
where:
- $b_{\text{phys},k}$ is the usual tightening for physical disturbances;
- $L_k$ is the gain from an action deviation at the first step to state deviation at step $k$, computed from the linearized nominal dynamics the filter already uses;
- $\hat{q}_{1-\alpha}$ is the $(1-\alpha)$ empirical quantile of $\|\pi_q(x) - \pi(x)\|$ over calibration states, with $\alpha$ the target violation rate.
The worst-case alternative replaces $\hat{q}$ with an analytic bound: the product of per-layer rounding bounds and layer Lipschitz constants.
Bit allocation. Per-layer bit-widths $\mathbf{b}$ are chosen by greedily solving
$$ \min_{\mathbf{b}} \ \text{memory}(\mathbf{b}) \quad \text{subject to} \quad \hat{q}_{1-\alpha}(\mathbf{b}) \le \tau, $$
where $\tau$ is the largest deviation the filter can absorb while keeping the tightened feasible set nonempty over the task region. The procedure works as follows:
- Start every layer at 8 bits.
- Lower one layer by one bit at a time, each time picking the layer whose reduction raises the calibrated quantile least.
- Recompute the quantile by re-running only the forward passes on stored states.
- Stop when no further reduction keeps $\hat{q}_{1-\alpha} \le \tau$.
No closed-loop rollouts are needed during this search. This is the step that replaces the outer hyperparameter loop.
Online correction. Once deployed, the quantized policy visits a slightly different state distribution than the full-precision policy did. The exchangeability that conformal guarantees rely on therefore breaks. We borrow the violation-feedback rule from the DAD-DPC work: raise the multiplier on $\hat{q}$ when the running violation rate exceeds $\alpha$, and lower it otherwise.
Because the full-precision policy can be evaluated in the background on each visited state, we can also append fresh deviation samples to the calibration set. This adds evidence directly instead of waiting for violations to occur. The extra forward pass costs something; we measure that cost rather than assume it away.
Quantizers. The pipeline is agnostic to the quantizer. We test round-to-nearest, GPTQ-style adaptive rounding, and QuIP's incoherence processing. QuIP reports that incoherence processing is about 1.5× slower than OPTQ kernels, which matters at kilohertz control rates. We therefore report its loop latency alongside its accuracy benefit instead of assuming it is affordable.
Baselines
Full precision with physical-only tightening. This is the do-nothing reference and upper bound: no quantization, and the filter margins cover only physical disturbances. It sets the achievable reward, tracking error and intervention rate. Every quantized method is reported as a gap to it.
Naive quantization with no extra margin. The same policy is quantized at the same bit-widths and the filter is left unchanged. This shows how much closed-loop violation quantization causes when ignored. If this baseline already stays near $\alpha$ at 3 bits on our tasks, the problem is less severe than we expect, and we will say so.
Worst-case quantization tube. This is the conservative incumbent and the main foil. The margin uses the analytic product of rounding and Lipschitz bounds, in the spirit of the bounded-error treatment in Safe Perception-Based Control with Minimal Worst-Case Dynamic Regret. It is run with identical policies, quantizers, filter and dynamics model.
Beating it means two things at equal bit-width:
- a lower filter intervention rate and better reward or tracking;
- a violation rate that is no more than $\alpha$ plus a pre-registered tolerance of two percentage points over held-out episodes.
Exhaustive closed-loop search. This is the strongest competitor on the allocation question: per-layer bit configurations selected by Bayesian optimization or successive halving, each candidate scored by full closed-loop rollouts. It gets a generous rollout budget, chosen at least 20× our method's (zero-rollout) search plus validation budget. Its best configuration defines the memory–safety frontier we aim to match.
The file reports no result for this on control tasks. On a different problem, Self-Tuning Networks report Bayesian optimization at 72.13 validation perplexity on Penn Treebank against 70.30 for single-run tuning. That shows single-pass tuning can match outer-loop search, but it is not evidence for our setting.
Beating this baseline means landing within 10% of its memory at equal violation and reward, using under 5% of its closed-loop rollouts.
Spectral allocation. The calibration-free AlphaQ rule assigns bits from the heavy-tailedness of each layer's spectrum (AlphaQ), applied to the policy's layers under the same memory budget. It costs nothing to run. If it matches our allocation, the closed-loop measurement adds no value for allocation, and our contribution shrinks to the margin.
Lumped-disturbance adaptation. This ablation isolates our key ingredient. It runs DAD-DPC-style violation feedback on a single margin that covers physical and quantization error together, without the separate deviation oracle. It tests whether measuring $\pi_q - \pi$ directly beats learning the same margin from violations alone.
Why not just…?
Why not just use 8-bit quantization and skip the cliff? For a small multilayer-perceptron policy on a quadrotor, 8 bits may already fit, and then the memory argument is weak. We take this seriously. The margin method helps at any bit-width, but the payoff is largest where memory binds: larger vision-based policies, or several models sharing a microcontroller.
We will report the 8-bit point on every curve, and report how much memory below 8 bits our method actually recovers safely. If the safe region below 8 bits is negligible on these tasks, that is a finding about the regime, and we will state it.
Why not just train the policy with simulated quantization noise, or with quantization-aware training? Training with noise makes the policy tolerate errors, but it gives no numerical margin a safety filter can use, and it costs a new training run per bit setting. That training cost is exactly the hyperparameter overhead we want to avoid. Results on the Actor–Critic MPC racing task also show that randomization-trained model-free policies can still fail out of distribution: they reached a 6.5% success rate under strong gusts, against 83.3% with an online optimizer in the loop.
The approach is complementary rather than competing. We include a noise-injection-trained policy as an extra arm, and also apply our margin to it. If noise training alone matches our violation and reward numbers, the filter-side margin is unnecessary.
Why not just learn the margin from observed violations, as in the building-control work? This is the most serious alternative, because that loop already works for external disturbances. Its weakness is speed: it must wait for violations, and the Polydome deployment needed 19 days to converge. Our method can instead compute the quantization part of the margin before deployment.
The lumped-disturbance ablation settles the question. We compare violations during the first episodes and time-to-convergence of the violation rate. If the lumped loop converges just as fast with equal safety, the deviation oracle is not earning its keep.
Why not just search bit-widths offline with Bayesian optimization? Offline compute is cheap for a small policy, so the overhead argument is weaker here than for large language models. The real objection is that closed-loop scores are noisy and safety-relevant failures are rare. Many rollouts per candidate would therefore be needed to estimate a violation rate near 5%. The exhaustive-search baseline measures this cost directly; if it is small, we will report that.
Proposed Experiments
The main testbed is safe-control-gym. We use 2D-quadrotor trajectory tracking and cart-pole stabilization with state constraints, with its injected wind and mass perturbations as physical disturbances. Its benchmark reports that GP-MPC reaches its best performance with roughly a tenth of the data that PPO or SAC need. The file gives no reported result for quantized controllers on this suite.
For larger policies, where memory matters, we use the velocity-constrained and vision-input tasks of Safety-Gymnasium. There, PPO-Lag is reported to cut safety violations by 98% relative to standard PPO at the cost of a 45% reward drop. We use PPO-Lag-trained policies as the starting full-precision policies.
All arms share the same policies, filter and seeds. We sweep bit budgets from 8 down to 2 bits and violation targets $\alpha$ of 1%, 5% and 10%. We report:
- realized violation rate against target;
- filter intervention rate;
- normalized reward or tracking error;
- weight memory;
- measured control-loop latency;
- the number of closed-loop rollouts each method spent selecting its configuration.
Decisive checks. Four results decide the question:
- At 3 bits and below, the worst-case tube should make the filter intervene so often that reward collapses toward a hand-designed fallback, while the calibrated margin keeps reward near full precision.
- The realized violation rate should track $\alpha$ within tolerance across held-out disturbance seeds, including after the closed-loop distribution shift that the online correction is meant to fix.
- Our zero-rollout allocation should sit on or near the exhaustive search's memory–safety frontier.
- The quantization-specific margin should beat the lumped-disturbance ablation on early-deployment violations.
Failure of the first check would mean worst-case conservatism is not the bottleneck. Failure of the second would mean closed-loop drift defeats calibration. Either outcome is a publishable negative result about deploying low-bit controllers safely.
References
- QuIP: 2-Bit Quantization of Large Language Models With Guarantees
- AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization
- Disturbance-Adaptive Data-Driven Predictive Control: Trading Comfort Violation for Savings in Building Climate Control
- Safe Perception-Based Control with Minimal Worst-Case Dynamic Regret
- Actor-Critic Model Predictive Control: Differentiable Optimization Meets Reinforcement Learning for Agile Flight
- Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions
- Safe-Control-Gym: A Unified Benchmark Suite for Safe Learning-Based Control and Reinforcement Learning in Robotics
- Safety Gymnasium: A Unified Safe Reinforcement Learning Benchmark