Semantic Packets as Free Doping: Coupled Erasure Coding for Two-Stream Neural Feature Links

Brain decoders that reconstruct images now split their output into two streams. One is a small semantic vector that says what is in the scene, such as a CLIP embedding. The other is a much larger structural map that says where things are, such as a variational-autoencoder latent or a depth or edge sketch. When this two-stream representation has to cross a lossy, low-latency wireless link in short packets, the two streams fail differently. Losing the semantic packet makes the generator draw the wrong object. Losing scattered structural packets mostly blurs or misplaces regions, and the generator can partly fill those in. We propose one packet-level code that links the two failure modes. The structural stream is protected by a spatially coupled erasure code, which decodes as a front that sweeps along a chain of linked blocks. The semantic packets are placed at known positions in that chain and given strong extra protection of their own. Once they are recovered, they act as anchors that restart the sweep. The protection we would pay for the important stream anyway then does a second job, keeping the bulk stream decodable.

The second part is analysis. The sweeping front, called the decoding wave, has stochastic models that predict when it stalls. These models exist only for memoryless erasures. Real wireless brain-computer-interface links lose packets in bursts, and burst loss is the most damaging pattern for neural decoding (Effects of Packet Loss on Neural Decoding Effectiveness in Wireless Transmission). We would extend the wave-stall theory to bursty packet erasures and to the short chain segments that frequent anchors create. That turns it into a design tool: it predicts how much structure is lost, and where, for a given anchor spacing, latency budget and burst statistics.

Core Claim

At matched total redundancy and matched decoding deadline, embedding the strongly protected semantic packets as doping points inside a spatially coupled code for the structural stream should give lower structural loss and better end-to-end reconstruction than coding the two streams separately with an optimized rate split. A regime-switching extension of the wave first-passage model should predict the resulting structural loss accurately enough to choose anchor spacing without Monte Carlo sweeps.

Three pieces of prior work set up this claim. Variable-node doping, meaning periodically inserting known bits into a streaming coupled chain, launches two decoding waves with some probability $\psi_d$ and otherwise does nothing (On Doped SC-LDPC Codes for Streaming). The catch is that doped bits are known to the receiver in advance, so they carry no data and cost rate. Our change is to make the doped symbols real payload, the semantic packets, whose reliability comes from an outer code they would need regardless. A second line of work models decoding under iteration limits and sliding windows as a race between a diffusing wave front and an absorbing barrier, using a Fokker-Planck equation (Finite-Length Scaling of SC-LDPC Codes With a Limited Number of Decoding Iterations). We reuse this race model, but that paper assumes memoryless erasures and homogeneous bits. Third, belief propagation and the simpler peeling decoder resolve the same symbols on the erasure channel (Finite-length scaling based on belief propagation for spatially coupled LDPC codes). This is why we work at the packet-erasure layer: packets that fail their checksum are true erasures, so the rigorous erasure-channel theory applies directly. We do not need the still-open AWGN extension.

Research Program

The code. The structural latents are quantized into short packets, a few hundred channel uses each, matching the short-packet regime of Cross-layer Transmission Design for Tactile Internet. Each packet is one variable symbol in a packet-level spatially coupled code with chain length $L$, coupling memory $m$ and $N$ symbols per position. Every $d$ positions, a group of semantic packets occupies the variable slots a doping scheme would normally fill with known symbols. The semantic packets are also covered by a short outer erasure code spread across the chain, so a burst at one position cannot wipe them out. Decoding has two stages. First the outer code recovers semantic packets as soon as enough of their parity has arrived. Then sliding-window peeling over the structural chain treats every recovered semantic packet as known.

We also test one design choice: ordering structural latents coarse-to-fine outward from each anchor. If the wave stalls between anchors, the coarse layout near the anchors survives and the lost content is the fine detail that a diffusion generator can best hallucinate.

The theory. In the existing model, the number of degree-1 check nodes $r(t)$ at the wave front behaves like an Ornstein-Uhlenbeck process, meaning a noisy quantity pulled back toward a mean. The wave stalls when $r$ first hits zero. Under Gilbert-Elliott bursts, a two-state good/bad Markov model of the link, both the pull and the noise depend on the hidden channel state $s(t)$:

$$dr = -\theta_{s(t)}\,(r - \mu_{s(t)})\,dt + \sigma_{s(t)}\,dW_t.$$

Here $\theta_s$ is the mean-reversion rate in state $s$, $\mu_s$ is the steady-state mean of $r$ in that state, and $W_t$ is a standard Brownian motion. The stall probability over one anchor segment of length $d$ becomes a first-passage problem with Markov-switching coefficients. It can be solved by a pair of coupled Fokker-Planck equations, one per channel state, with an absorbing boundary at $r=0$ and a moving window boundary from the decoding deadline.

The anchor success probability $\psi_d$ is no longer a constant. It factors into the probability that the outer semantic code decodes in time and the probability that the wave survives the local dip in degree-1 checks next to the anchor. Both depend on the same burst process, so they are correlated, and the model must keep that correlation. When $d$ is close to $m$, the wave never reaches its steady state. We therefore also derive the transient first-passage law from the anchor boundary condition, which is the short-chain gap the existing scaling laws leave open. As in the shifted Gaussian model, we expect to need one calibration simulation per ensemble to fix the noise parameter, and we will report that dependence plainly.

The end-to-end model. The theory outputs the distribution of how many structural packets are lost, and where, between anchors. We feed these loss masks into a frozen two-stream reconstruction pipeline. Following the provenance protocol of Brain-Measurable Diffusion Decoding, cross-attention tokens carry the semantic stream and VAE latents carry the layout. Missing latent regions are inpainted by the generator. The semantic stream comes from a MindEye-style contrastive head (Reconstructing the Mind's Eye) and the structural stream from its low-level VAE pipeline.

One honest caveat about the setting: fMRI reconstruction is not itself transmitted wirelessly. We use the Natural Scenes Dataset as a realistic stand-in for a two-stream source with unequal importance. We use the macaque motor dataset for a case that genuinely is wireless and streaming.

Baselines

The naive reference is uncoded transmission of both streams over the same Gilbert-Elliott link, with lost packets handed to the decoder as erasures. This sets the floor. For motor data it should reproduce the reported collapse of acceleration $R^2$ to about 73% of its original value at 10% packet loss (Effects of Packet Loss on Neural Decoding Effectiveness in Wireless Transmission).

The strongest competitor is separate coding with an optimized unequal rate split. The semantic packets get a strong short code. The structural stream gets an uncoupled packet-level LDPC erasure code, or alternatively a standard uniformly doped coupled code. The split between them is searched over the same total redundancy. This is what a careful engineer would build first. Our closest proxy for how good strong short codes are: Efficient Error-Correcting Codes in the Short Blocklength Regime reports that non-binary LDPC and Polar+CRC on a (128,64) code come within 0.1 to 0.5 dB of the random-coding-union bound at block error rate $10^{-4}$. That number is for AWGN bit-level coding. We found no reported result on Lacuna for packet-level coupled erasure codes under Gilbert-Elliott bursts. For fairness, every scheme gets the same total overhead, the same decoding deadline expressed in packet slots, the same peeling-decoder complexity cap, and the same grid search over rates or spacing. Beating this baseline means a statistically significant gain at matched rate and latency in at least one of two places: residual structural packet loss across the tested burst regimes, or end-to-end CLIP two-way identification and SSIM.

The key ablation keeps the coupled structural code and the anchor positions but removes the coupling between streams. The anchor slots are filled with ordinary known-bit doping, and the semantic stream is sent separately with the same outer code. This isolates whether reusing semantic payload as doping is what produces the gain. A second ablation removes coarse-to-fine ordering.

The clean-channel ceiling is the reconstruction quality of the same frozen pipeline with no loss. Published clean numbers give the scale: MindDiffuser reports a CLIP score of 0.765 and SSIM of 0.354 (MindDiffuser), and the provenance-audited decoder reports 99.1% CLIP two-way identification on Subject 01 (Brain-Measurable Diffusion Decoding). We will recompute the ceiling with our own pipeline rather than compare against these figures directly.

Why not just…?

Why not just code the streams separately with a stronger code on the semantic packet? This is the real threat, and the proposal is built so that it can lose to it. Separate coding is simpler and uses mature short codes. Our only argument against it is mechanistic: in the separate design, the semantic stream's redundancy does nothing for the structural stream, while in ours it replaces doping that would otherwise cost rate. If the matched-rate comparison shows a gain below the across-seed noise, we will report that the coupling is not worth its complexity.

Why not just retransmit lost packets? Retransmission roughly halved packet loss in a wireless BCI device (Effects of Packet Loss on Neural Decoding Effectiveness in Wireless Transmission). However, a round trip does not fit millisecond-scale closed-loop budgets, which is why cross-layer short-packet designs rely on forward error correction and proactive dropping instead (Cross-layer Transmission Design for Tactile Internet). We will include ARQ under the same deadline, so it gets as many retries as fit in the latency budget, and report where it wins, likely at loose deadlines.

Why not just protect only the semantics and let the diffusion model inpaint all lost structure? This is a serious alternative, because generators are good at plausible fill-in. But semantic-only conditioning is known to get position, scale and orientation wrong (MindDiffuser), and VAE latents are what anchor layout (Brain-Measurable Diffusion Decoding). We run this baseline at matched rate. If SSIM and layout metrics match ours, structural coding is unnecessary at the tested loss rates.

Why not just run Monte Carlo instead of building a theory? For a single operating point, Monte Carlo is fine. The theory matters for the design search over anchor spacing, window size and burst regimes, where the Fokker-Planck approach replaced days of cluster simulation with minutes on a laptop (Finite-Length Scaling of SC-LDPC Codes With a Limited Number of Decoding Iterations). We validate the theory against Monte Carlo and only trust it where they agree.

Why not just learn joint source-channel coding end to end? Learned wireless image transmission exists, for example KeYang8/WITT, but we found no reported result on Lacuna for it on neural-feature streams or under packet bursts. It also offers no analytic loss guarantee. We treat it as optional, as a stretch baseline on the analog-channel variant only.

Proposed Experiments

The first experiment validates the theory. On packet-level ensembles modeled on the (5,10) and (4,12) protographs, we sweep anchor spacing $d$, window size, and Gilbert-Elliott parameters, including burst lengths comparable to $d$. We compare the predicted per-segment stall probability and structural block error rate against Monte Carlo. Success means a log-error prediction gap no worse than the memoryless model shows in its own published regime. The critical test is the short-segment case $d \approx m$.

The second experiment compares codes at matched total overhead and deadline: the coupled-anchor scheme, both ablations, separate coding, ARQ, and uncoded transmission. We report residual semantic and structural loss rates.

The third experiment is end to end on the Natural Scenes Dataset, using the 982-image shared test set for Subject 01. We run the frozen two-stream pipeline under sampled loss masks and report CLIP two-way identification, AlexNet-2 identification and SSIM against loss rate. A further check replaces sampled masks with theory-predicted mask statistics, to test whether the analytic model can stand in for simulation when predicting reconstruction quality.

The fourth experiment is a wireless sanity check on the Sabes macaque reaching data. Spike-bin features are split into a slow stream and a fast stream, and we test whether anchoring recovers acceleration decoding under burst loss, where uncoded transmission is known to hurt most.

The decisive check is the second experiment under bursty loss. If coupled anchors do not beat the optimized separate-coding split at matched rate and latency, the core claim fails, whatever the theory achieves.