Active learning picks the unlabeled examples a model is most unsure about and asks a labeler for them. At the network edge, that labeler is often a remote "teacher" model, and the chosen examples must first cross a noisy wireless link. Current systems split this into two steps: first decide which examples to query, then decide how to transmit them, usually by giving every example the same number of channel uses. This note argues that the split fails for the same reason Shannon's separation of compression from error correction fails when blocks are short and noise is high. The value of an example to the learner depends on how faithfully it arrives. How faithfully it arrives depends on how much of the channel it gets. Neither decision can be made well without the other.
The concrete proposal is an acquisition rule that chooses a set of examples and a per-example channel budget together. The budget is the number of transmitted symbols, allocated through a truncatable neural joint source-channel code, meaning an encoder that maps an image straight to noisy analog channel symbols, so that more symbols give a cleaner reconstruction. The rule maximizes the learner's expected information gain about its own parameters per channel use. This gain is computed with respect to the corrupted version the teacher will actually label, so noise the channel will predictably add is not mistaken for useful model uncertainty. The central prediction can be tested directly. Under a fixed total symbol budget, joint allocation should beat the best "select, then transmit uniformly" pipeline at low signal-to-noise ratio and small budgets. The gap should shrink toward zero as SNR and budget grow, which mirrors how separation becomes optimal only asymptotically.
Core Claim
The closest existing work is Batch Selection and Communication for Active Learning with Edge Labeling (CC-BAKD). It selects batches with BatchBALD, which scores a batch by the mutual information between its unknown labels and the model parameters. It then compresses inputs with a shared linear PCA projection plus linear mix-up across the batch, sends them over a noisy channel, and trains on the returned noisy soft labels with a weighted free-energy objective. It shows that communication-aware selection can be dramatically cheaper than naive querying. Its limits are specific, though. Compression is linear and applied at a fixed ratio to every example. The acquisition score accounts only for linear reconstruction distortion; the authors note that quantization noise is not modeled. Nothing lets the learner spend more of the channel on one example than another.
Our claim is that the selection problem should be posed over pairs (example $i$, symbol budget $k_i$):
$$ \max_{S,\ \{k_i\}} \; I\big(\theta;\ \{\tilde y_i\}_{i\in S} \,\big|\, \mathcal{D}\big) \quad \text{s.t.} \quad \sum_{i\in S} k_i \le N . $$
Here $\theta$ are the learner's parameters and $\mathcal{D}$ its current labeled data. The term $\tilde y_i$ is the teacher's label for the reconstruction $\hat x_i(k_i, h)$ that emerges after $k_i$ channel symbols under channel state $h$, and $N$ is the round's total symbol budget. Because the information gain is taken over $\tilde y_i$ rather than the clean label, the objective automatically discounts examples whose informative detail would be destroyed at low $k_i$. It also discounts label noise that comes from the channel rather than from the data. This is exactly the three-way confusion flagged as open in the research file: an edge case, a corrupted image, and a deep fade can all show up as "high received entropy." The receiver knows its SNR, so the channel-induced part can be predicted and removed from the score instead of rewarded.
Research Program
The work has three parts, each with a clear deliverable.
The first part is a small theory model that makes the separation gap visible in closed form. We use Bayesian linear regression with a Gaussian prior, examples sent over an additive Gaussian channel with $k_i$ uses each, and linear minimum-mean-square-error reconstruction. In this model the posterior-covariance reduction from each (example, budget) pair is explicit. Optimal allocation becomes a water-filling-like problem over the eigen-directions the learner is least sure about. We will derive the ratio between the joint optimum and the best "choose by uncertainty, then split symbols equally" policy. The expected form comes from the fact that the reconstruction-noise variance scales like $(1+\mathrm{SNR})^{-k_i}$ for a Gaussian source sent with $k_i$ channel uses. That suggests the ratio should approach one exponentially fast in SNR times per-example budget and differ most when budgets are a few symbols per example dimension. This motivation role is the one played by the classical results that separation breaks outside the isolated, asymptotic setting: Separation of source-network coding and channel coding in wireline networks, and the finite-blocklength behavior in A Second-Order Converse Bound for the Multiple-Access Channel via Wringing Dependence. Those works measure loss in distortion; we measure it in parameter information gain, which is the quantity an active learner actually spends.
The second part is the transmission mechanism. It is a truncatable deep joint source-channel code: an autoencoder whose first $k$ output symbols give a usable reconstruction for every $k$ on a grid. It is trained with a loss averaged over random truncation points and random SNRs. The method borrows from Neural Joint Source-Channel Coding (NECST), which trains an end-to-end discrete autoencoder over noisy channels and reports graceful rather than cliff-like degradation. NECST, however, uses a fixed code length, which is precisely the limitation we remove. Distributed Deep Joint Source-Channel Coding over a Multiple Access Channel (DeepJSCC-NOMA) likewise fixes the bandwidth ratio and optimizes pixel fidelity, with no notion of downstream utility. Our encoder has no utility objective of its own. Utility enters entirely through the allocation rule. This keeps the ablations clean.
The third part is the acquisition algorithm. For each candidate, the learner holds an ensemble or MC-Dropout copy of its model and a local emulator of the channel code. It estimates the information gain at several budgets by pushing emulated reconstructions through the model and a lightweight proxy of the teacher. It then allocates symbols greedily by marginal gain per symbol, which is a reasonable heuristic because the objective is submodular in the example set for BALD-style scores. The learner may also send "zero symbols," which means skipping the example. Following Privacy-preserving and Uncertainty-aware Federated Trajectory Prediction for Connected Autonomous Vehicles, which found that clients with median rather than maximal aleatoric uncertainty are the most useful, we expect the rule to avoid the noisiest examples. Unlike that work, it does so because of an explicit channel-aware score rather than a fixed quantile heuristic. To keep scoring cheap, we score with an early intermediate head, as in Understanding the Training Speedup from Sampling with Approximate Losses (SIFT). The overhead concern raised in Active Learning Solution on Distributed Edge Computing is real, so we report scoring compute alongside channel uses.
Baselines
The naive reference is random selection with equal symbols per example, run through the same truncatable encoder and channel and the same learner. Every method below gets the same total symbol budget per round, the same number of rounds, the same seed set, and the same hyperparameter search budget. The seed set is at least 20 random labels per class-balanced start, matching the cold-start threshold reported in Active Learning Solution on Distributed Edge Computing.
The strongest existing method is CC-BAKD from Batch Selection and Communication for Active Learning with Edge Labeling, reimplemented with its PCA plus mix-up compression, BatchBALD selection, and weighted free-energy update. On MNIST it reports about 61% accuracy at N = 784 symbols with B = 4 and R = 0.99. Its uncompressed baselines BAKD and RAKD need roughly 65×10^4 symbols for similar accuracy. Beating it means reaching higher MNIST accuracy at N = 784 symbols, or reaching 61% with fewer symbols, averaged over at least five seeds and at more than one SNR. On CIFAR-10 the file reports no CC-BAKD number, so there we compare curves we generate ourselves.
The separated neural pipeline is BatchBALD selection followed by our own truncatable encoder at a fixed per-example length, with the length tuned per SNR. It is the direct embodiment of the "choose, then transmit" split that the proposal claims is suboptimal, and it uses the same encoder as our method.
The digital separation pipeline quantizes each image, protects it with an ideal channel code at the finite-blocklength normal-approximation rate, and uses BatchBALD on the decoded inputs. It is generous to separation because it assumes an ideal code.
The key ablation is channel-aware BatchBALD at fixed length. It scores examples on emulated noisy reconstructions exactly as we do but gives each the same budget, so the only thing it removes is variable allocation. A second ablation allocates budgets by total predictive entropy instead of information gain, which isolates the channel-noise disentanglement. A noiseless-channel upper bound shows how much headroom exists.
Why not just…?
Why not just use standard digital compression with capacity-approaching error correction and run BatchBALD on the decoded data? At high SNR and long blocks this is expected to be nearly optimal, and our theory model predicts the gap vanishes there. That regime is a control, not a weakness. At low SNR, NECST Neural Joint Source-Channel Coding reports being 20x more bit-efficient than WebP with an ideal channel code at a 40% bit-flip rate, and digital schemes fall off a cliff below their decoding threshold. The SNR sweep in our experiments tests whether that reconstruction advantage turns into a learning advantage.
Why not just compute BALD on simulated noisy reconstructions and keep lengths fixed? This is the most serious threat. It captures the disentanglement idea without the allocation machinery, and it may recover most of the gain. It is included as the key ablation. If it matches our method within seed noise across the SNR and budget grid, the honest conclusion is that channel-aware scoring suffices and variable allocation is not worth its cost. The theory model tells us where to look for the difference: small per-example budgets, where skipping some examples to send others well should matter most.
Why not just send many more random examples at low quality? Early in training, quantity may win. Active Learning Solution on Distributed Edge Computing notes that random sampling matches uncertainty sampling before a seed set exists, and gains typically narrow once accuracy saturates. Our allocation rule can choose this policy itself by spreading symbols thinly, so we will report when it does. We compare against random selection with a tuned uniform length at every round, not just at the end.
Proposed Experiments
The main experiment sweeps SNR from −5 to 15 dB on AWGN and block-Rayleigh channels and total per-round budgets on MNIST, which is the CC-BAKD setting. The teacher is a pretrained classifier returning soft labels on received reconstructions. We then repeat on CIFAR-10, the benchmark used for DeepJSCC-NOMA, with a ResNet teacher and a small CNN learner. The metrics are test accuracy versus cumulative channel symbols, the area under that curve, symbols needed to reach a target accuracy, and scoring compute per round.
Three decisive checks follow. The first is whether the joint-minus-separated gap is significant at low SNR and small budget and shrinks toward zero at high SNR, as the Gaussian model predicts. A gap that does not shrink would point to an implementation artifact rather than a separation effect. The second is whether the full method beats channel-aware fixed-length BatchBALD. If not, variable allocation is abandoned in favor of the simpler rule. The third checks the disentanglement directly. We inject known label corruption into a subset of the pool and verify that our score, unlike total-entropy allocation, stops spending symbols on those examples as the channel worsens.
A two-device extension over a shared multiple-access channel, reusing the superposition decoder design from DeepJSCC-NOMA, is attempted only if the single-link results are positive.
References
- Batch Selection and Communication for Active Learning with Edge Labeling
- Separation of source-network coding and channel coding in wireline networks
- A Second-Order Converse Bound for the Multiple-Access Channel via Wringing Dependence
- Neural Joint Source-Channel Coding
- Distributed Deep Joint Source-Channel Coding over a Multiple Access Channel
- Privacy-Preserving and Uncertainty-Aware Federated Trajectory Prediction for Connected Autonomous Vehicles
- Understanding the Training Speedup from Sampling with Approximate Losses
- Active Learning Solution on Distributed Edge Computing