Keeping 4-Bit Diagnostic Models Calibrated by Separating the Noise Channel

Small diagnostic models on battery-powered sensors are usually squeezed down to 4-bit integers after training. This step, called post-training quantization, clips and rounds each layer's numbers to ranges measured on clean calibration data. That works in the lab. It is fragile in the field, where train noise, humidity, or a patient who cannot hold still reshapes the model's internal activations and pushes them outside those fixed ranges. We propose applying the logic of Shannon's source-channel separation principle to this problem. That principle says a communication system can be split into a compressor that knows only the data and an error-correcting layer that knows only the noisy wire, with no loss.

By analogy, we split the model in two. A tiny, deliberately non-quantized-in-spirit front end is the only part that sees the environment. Its job is not merely to denoise, but to hand the back end inputs whose statistics match the clean calibration set. The diagnostic back end can then be quantized once, on clean data, and never recalibrated.

The claim is testable and could fail in an informative way. Separation theory predicts when this design should work and when it should break. It should hold when the nuisance (speed, humidity, acquisition quality) is independent of the diagnostic label. It should fail when the two are correlated, as they are in bedside ultrasound, where the sickest patients produce the worst scans. We will measure both regimes. We will build the design on railway and concrete structural-monitoring data, where nuisance is physically controlled, and stress-test it on clinical imaging, where it is not.

Core Claim

A separated edge model gives two things together: a small front end trained to make the back end's layer activations look like they did on clean calibration data, plus a back end quantized to 4 bits with analytical clipping calibrated once on clean data. We claim this combination keeps more accuracy under unseen noise conditions than an end-to-end model of equal compute, whether that model is quantized with noisy calibration data or trained with noise-aware quantization.

The mechanism is specific and measurable. The quantization method of Post training 4-bit quantization of convolutional networks for rapid-deployment (Banner et al.) chooses clipping thresholds analytically, assuming bell-shaped (Gaussian or Laplace) activations of known spread. Its expected clipping-plus-rounding error is therefore a known function of how far the deployed activation distribution drifts from the calibrated one. If the front end removes that drift, the back end's quantization error stays near its clean-data value regardless of the environment.

The second half of the claim comes from Optimality and Approximate Optimality of Source-Channel Separation in Networks. That paper proves separation is optimal or near-optimal across many multi-user networks, and names correlated sources over entangled channels as the case where it becomes suboptimal. We read "channel" as the acquisition environment and "source" as the diagnostic state. The prediction is that the separated design's advantage should shrink, or reverse, as the correlation between nuisance and label rises. We treat this as a falsifiable prediction, not a caveat.

What Prior Work Does and Leaves Open

Three papers motivate the gap.

  • Quantization method. Banner et al. supply the quantization method we reuse. Their combination of analytical clipping (ACIQ), per-channel bit allocation, and bias correction recovers ResNet101 at 4-bit weights and 4-bit activations from 62.6% to 75.0% ImageNet top-1, against 77.2% at full precision. They evaluate only clean 2D images. They also note that correcting activation bias needs input statistics that are costly to track on-device.
  • Real-world drift. Two clinical results show that such drift is ordinary. POCOVID-Net falls from 87.8% on scraped data to 65% prospectively in COVID-BLUeS - A Prospective Study on the Value of AI in Lung Ultrasound Analysis. The synthetic corruptions of MediMeta-C push a biomedical vision-language model, BioMedCLIP, above 100 mean Corruption Error (mCE), according to On the Robustness of Medical Vision-Language Models: Are they Truly Generalizable?.
  • Nobody combines the two. No paper in our search measures what quantization does on top of that drift.

Two structural-monitoring papers supply the front-end design and the controlled nuisance.

  • Front-end design. Acceleration-guided Acoustic Signal Denoising Framework Based on Learnable Wavelet Transform Applied to Slab Track Condition Monitoring (AG-ASDF) trains a learnable wavelet front end (a filter bank whose multi-scale filters are learned from data) on noisy microphone audio. It is supervised by a clean accelerometer that is removed at deployment, and it reaches 95.4% track-degradation accuracy. It runs at full precision. Its loss aligns acoustic features to vibration features, not to the statistics a downstream quantizer was calibrated on. That second target is the ingredient we add.
  • Controlled nuisance. Impact of temperature and relative humidity variations on coda waves in concrete tracks late-arriving ultrasonic "coda" waves. It shows that humidity alone spreads the attenuation quality factor Q by 115%, against 40% for temperature, so humidity can mimic damage. The paper removes this effect with a third-order polynomial, and decoupling becomes ill-posed above about 80% relative humidity. That gives us a physically labeled nuisance axis with a known breaking point.

One paper sets how small the front end must be. Adaptive wavelet distillation from neural networks through interpretations distills neural teachers into sparse wavelet models at roughly 200-fold lower inference cost on biomedical time series. This shows that a wavelet front end can be cheap enough that leaving it at higher precision does not undo the savings.

The closest conceptual neighbor is Distributionally Robust Wireless Semantic Communication With Large AI Models (WaSeCom). It separates robustness to input ("semantic") perturbation from robustness to channel noise, using a two-level worst-case optimization over a Wasserstein ball: the set of input distributions within a fixed transport distance of the training data. It works on CIFAR-10 and text, runs at full precision, and treats the channel as a radio link. We instead treat the sensing environment as the channel and quantization error as the quantity that separation must protect. We also test the correlated-source regime that WaSeCom's strict two-level split does not examine.

Two quality-gating papers show why simply discarding bad inputs is not an acceptable baseline:

Research Program

We will build a two-part model.

Front end. A learnable wavelet filter bank followed by a small per-subband affine normalizer, kept at 8 bits. It is trained with three losses: a reconstruction or guidance loss as in AG-ASDF, a sparsity penalty as in adaptive wavelet distillation, and a calibration-matching loss. The calibration-matching loss penalizes the gap between the back end's per-layer activation distributions on processed noisy inputs and the distributions recorded on clean calibration data. We use the 2-Wasserstein distance (the transport distance between two distributions) between per-channel histograms. The back end is frozen while this loss is optimized.

Back end. A 1D convolutional classifier for structural signals, or an image classifier for clinical data. It is trained once on clean or low-nuisance data, then quantized to 4-bit weights and activations with ACIQ, bias correction, and bit allocation calibrated on that clean data.

Analysis. For each condition we compute per-layer activation drift and plug it into the ACIQ error expression, predicting the quantization error before measuring accuracy. The mechanistic question is whether predicted error tracks measured accuracy loss across conditions, and whether the front end drives predicted error back to its clean value. The separation-theoretic question is how the advantage varies with the mutual information between nuisance and label. We manipulate that correlation directly by resampling the training and test sets.

Baselines

Naive reference. An end-to-end model of matched total bit-operations. We quantize it with uncalibrated uniform 4-bit rounding and evaluate it under each noise condition. Banner et al. establish how bad this is on clean data, at 62.6% on ResNet101. It sets the floor.

Full-precision ceilings. Two ceilings bound what quantization costs.

  • AG-ASDF run at full precision on the slab-track data. Its reported number is 95.4% average accuracy, 2.2 points above plain DeSpaWN, from the AG-ASDF paper.
  • The same end-to-end architecture as the naive reference, at full precision.

Beating AG-ASDF is not the goal. Closing most of the gap between our 4-bit model and it, under held-out speeds, is.

Strongest existing competitor. The end-to-end model quantized with Banner et al.'s full pipeline, with calibration data that includes noisy examples drawn from the training conditions. This is what a careful engineer would ship. We found no reported result on Lacuna for any 4-bit model under structural or clinical corruption, so we will produce these numbers ourselves.

Second competitor. Quantization-aware training with noise augmentation, meaning the model is trained while simulating 4-bit rounding.

Fairness rules. All models share the same data splits, the same total inference bit-operations (front end included), and the same hyperparameter search budget, measured in training runs.

Win condition. Across held-out nuisance conditions, the separated model must beat both competitors by more than the seed-to-seed standard deviation. It must do so while staying within the competitors' compute. This must hold in the independent-nuisance regime.

Ablations. Two ablations isolate the key ingredient. The first removes the calibration-matching loss, leaving a front end trained only to denoise. The second recalibrates the back end on front-end outputs instead of clean data. If the first ablation performs as well as the full model, then ordinary denoising is enough and our distinctive claim fails.

Why not just…?

Why not just calibrate the quantizer on noisy data? This is the most serious threat. On conditions seen during calibration it may match our method. But widening the clipping range to cover noisy activations coarsens the rounding steps on clean ones. Banner et al.'s analysis makes this explicit: the ACIQ-optimal clip trades clipping error against rounding error for a single assumed distribution. A mixture of conditions is a poor fit to any single bell-shaped prior. The decisive comparison is on held-out nuisance levels: unseen train speeds, humidity bands outside calibration, and MediMeta-C corruption severities not used for calibration. If noisy calibration holds up there too, we will report that the separation step is unnecessary.

Why not just use quantization-aware training with noise augmentation? It needs full retraining for every deployment and every sensor. That defeats the rapid-deployment motivation of post-training quantization. The COVID-BLUeS result also shows that end-to-end training on augmented data can still latch onto dataset-specific shortcuts. Still, it is a legitimate competitor, and it is run with an equal budget. If it wins on held-out conditions, we will say so.

Why not just reject low-quality inputs? In the heart-ultrasound study, rejection would discard 73% of acquisitions. DLIFS shows that the rejected images are enriched for pathology. We will report accuracy with rejection alongside coverage, meaning the fraction of patients who receive any prediction. A gate that wins only by refusing the hardest cases does not count as winning.

Why not just stay at 8 bits? If 8-bit end-to-end models show no drift-driven loss under noise, our motivation largely disappears, and we will report that outcome. We therefore also run every condition at 8 bits, to show whether the failure we target exists at all.

Proposed Experiments

Slab-track data (AG-ASDF). The nuisance axis is train speed (20, 40, 60, and 80 km/h, 20 kHz audio). We train and calibrate at two speeds and test at the other two. The accelerometer is available as a guidance signal during training only. Accuracy is the metric, alongside per-layer activation drift.

Concrete climate-chamber coda-wave data. The nuisance axis is humidity. We train and calibrate below 60% relative humidity and test in the 60–80% and above-80% bands, where decoupling is known to be ill-posed. The tasks are regression of velocity change against an imposed damage proxy and the associated attenuation error. We expect separation to fail above 80% relative humidity. Showing that it does is part of validating the theory-derived failure prediction.

We are not certain both structural datasets can be obtained. If only one is available, the other nuisance axis is dropped rather than simulated.

Clinical controlled arm (MediMeta-C). We use the five modalities and seven corruptions at graded severities, reporting mCE. The reference point is RobustMedCLIP's cell-microscopy mCE of 70.1, a full-precision, low-rank adapted (LoRA) model. We ask how much of that robustness survives at 4 bits under each baseline and under our method.

Clinical real-shift arm (COVID-BLUeS). This arm uses prospective bedside lung ultrasound, where image quality is plausibly correlated with disease. It is the planned test of the correlated regime. The prediction is that our method's advantage disappears or reverses there. The tabular blood-marker model at 84% is the practical bar any imaging model must be compared against.

Correlation sweep. On the slab-track data, we resample the dataset to set a chosen mutual information between speed and degradation class, from zero to high. We then plot the separated model's advantage against that quantity. A downward trend confirms the separation-theoretic reading. A flat trend means the benefit comes from something other than decoupling. We will state that plainly.

All compute results are reported in bit-operations and measured microcontroller latency, with front-end cost included.