When a car completes its depth map from LiDAR points sent over a wireless link, by another vehicle or by a roadside unit, some packets will be lost. Standard networking offers two remedies, and both are costly. Retransmission adds latency. Erasure coding, which sends extra algebraic combinations of packets so that lost ones can be rebuilt, adds decoding computation that can exceed the transmission time itself on embedded hardware. This proposal starts from a simple observation: a depth completion network, a model that fills in a dense depth map from a camera image plus a few thousand measured depth points, is already an error-correcting decoder for most of those points. If a point sits on a flat road surface, the network will predict its depth almost exactly from its neighbours and the image. Protecting that point with coding wastes bandwidth and CPU time. The points that genuinely need protection are the few the network cannot reconstruct. These are points on thin poles, at object boundaries, and isolated returns in otherwise empty regions.
We propose to measure this "irreplaceability" directly. Each point gets a score for how much the receiver's completion error rises when that point is lost. The sender computes the scores cheaply and uses them to split the stream. A small irreplaceable subset gets lightweight, block-partitioned parity protection, which is cheap to decode. The remaining points are sent uncoded and spread across packets, so that any loss looks to the network like random subsampling. The claim is that, at equal bandwidth overhead and equal decoding budget, this split gives lower depth error at object boundaries than either uniform erasure coding or no coding at all. It should also give better downstream detection in cooperative perception.
Core Claim
The unit of reliability for sparse-to-dense depth completion should be the point's contribution to reconstruction error, not the packet. Once losses are arranged to look like random point dropout, most of the stream can be left uncoded, because completion networks tolerate dropout. Coding is then needed only for a small fraction of points. For that fraction, a systematic code decodes cheaply: the original points travel unmodified, and the parity is split into many tiny independent blocks. Decoding therefore costs roughly linear time rather than the cubic Gaussian elimination of monolithic codes. The new ingredient is the protection target: points are ranked by predicted completion error when absent, rather than by detection confidence or uniformly.
Where This Sits
Collaborative perception supplies both the application and the gap. BM2CP reprojects LiDAR points from neighbouring vehicles into the ego camera and densifies them with a completion network. It reports 83.72% AP on OPV2V, but it assumes the points arrive losslessly. Where2comm decides what to send using spatial confidence maps and cuts communication volume by up to 100,000x. It prunes by detection confidence and offers no protection against loss. OPV2V shows that features compress 4096x at roughly a 3% accuracy cost, again over an idealized channel. We keep their sharing setups and add a lossy channel. We also replace "importance for detection" with "irreplaceability for completion" as the criterion for protection rather than for transmission.
The coding literature constrains the design. Block-Diagonal and LT Codes for Distributed Computing With Straggling Servers shows that monolithic MDS codes (codes that recover the data from any sufficient subset of pieces) are about 20x slower to decode on large matrices than codes partitioned into small independent blocks. The communication cost of partitioning is negligible. Wireless Network Coding: Opportunities & Challenges documents the cubic decoding cost of random linear network coding. NCRAWL shows that XOR-only coding cuts CPU use 2x to 7x relative to COPE. We reuse the block-partitioning idea and the XOR-first arithmetic rather than inventing a new code. Coded Computing at Full Speed motivates using partial arrivals rather than waiting. Our analogue is that the completion network makes use of every uncoded point that arrives.
The depth completion literature supplies the receiver and explains why irreplaceability is uneven. Propagation models, which spread measured depth to neighbouring pixels along image-guided affinities, degrade gracefully under random sparsity. These include CSPN, NLSPN (741.68 mm RMSE on KITTI) and DySPN (709.12 mm). They still fail when whole regions empty out, as Estimating Depth from RGB and Sparse Sensing shows at 0.01% density. Depth Coefficients for Depth Completion shows that errors concentrate at depth discontinuities. There, boundary smearing cut Easy 3D-detection AP from 64.9% to 36.7%. This is the evidence that a few boundary points carry disproportionate value, and it motivates the proposal.
Research Program
The sender assigns each point $i$ an irreplaceability score $r_i$. The score is a regression target defined as the expected increase in the receiver's local completion error when point $i$ is dropped but its neighbours are kept. We generate training labels offline on KITTI and OPV2V. We drop random subsets of points, run a frozen completion network (NLSPN or DySPN), and record the absolute residual $|\hat d_i - d_i|$ at the dropped locations. Here $d_i$ is the measured depth and $\hat d_i$ the depth the network predicts for that pixel without the point. A small point-cloud network at the sender, optionally given the sender's own camera image, learns to predict $r_i$. It does not need the receiver's image, which the sender does not have in the cooperative setting.
The sender then solves a knapsack-style allocation. The receiver's expected error $E$ must be kept low under a parity budget:
$$\min_{S}\; \sum_{i \notin S} p\, r_i \quad \text{s.t.}\quad \rho\,|S| \le B,$$
In this formula, $S$ is the set of protected points, $p$ is the packet loss rate, $\rho$ is the parity overhead per protected point, and $B$ is the bandwidth overhead budget. The objective counts only the unprotected points, because protected ones are assumed to be recovered. The solution is simply to protect the highest-$r_i$ points until the budget runs out.
Protected points go into small blocks of $k$ packets, each with a few XOR or $\mathbb{F}_{2^8}$ parity packets. Decoding therefore costs $O(k^2)$ per block with $k$ around 8 to 16, a size chosen following the block-diagonal result. Unprotected points are interleaved pseudo-randomly across packets, so that a lost packet removes scattered points rather than a contiguous image region.
We will build three pieces: the loss-label generator, the irreplaceability predictor, and a packetizer/decoder in C running on an embedded ARM CPU. The analysis asks one question. As loss rate and burstiness vary, at what point does protecting the top fraction stop helping, and how does that fraction compare with the share of points lying on depth discontinuities?
Baselines
The naive reference is uncoded best-effort transmission with contiguous packetization, which is how points are naturally serialized by scan line. Lost packets become missing scan segments, and the completion network runs on whatever arrives. This is the configuration BM2CP implicitly deploys once losses appear.
The strongest simple competitor is uncoded transmission with random interleaving, spending the bandwidth budget $B$ on sending more points instead of parity. It isolates the claim that the completion network is a decoder, and it may win at low loss. The comparison uses the same completion network, the same total bits per frame, and the same loss traces. Beating it means lower boundary tMAE at 10–30% bursty loss, with RMSE no worse.
The strongest existing coding approach is uniform protection at the same budget $B$, run as either Reed–Solomon over all packets or block-partitioned random linear coding. For this baseline we report both error and decode time on the same ARM core. To beat it, our method must show equal or lower RMSE at a decode time at least several times lower, or lower error at equal decode time.
The detection-oriented alternative ranks points for protection by a Where2comm-style spatial confidence rather than by $r_i$, under an identical coding and packetization pipeline.
Two ablations isolate the key ingredient. The first replaces the learned $r_i$ with a hand-crafted geometric proxy, namely local depth variance (a discontinuity detector). The second uses random selection of the protected set. If the geometric proxy matches the learned score, the learned predictor is unnecessary, and we will report that as the result.
For absolute reference, the best clean-channel numbers on Lacuna are DySPN's 709.12 mm RMSE on KITTI and BM2CP's 83.72% AP on OPV2V. We found no reported result on Lacuna for any depth completion or cooperative perception method under packet loss, so all loss-channel numbers will be produced here.
Why not just…?
Why not just interleave points and let the network fill in the gaps? This is the real threat, and it is part of the proposal, since unprotected points are interleaved. What interleaving cannot fix is the loss of an irreplaceable point, such as a lone return on a pedestrian or a pole edge. The Depth Coefficients results suggest that such boundary errors drive downstream detection failure. If interleaving alone matches our method on boundary tMAE and OPV2V AP across loss rates, we will conclude that coding is unnecessary for this task. That would itself be a useful negative result.
Why not just use a standard Reed–Solomon code on everything? The block-diagonal results show monolithic MDS decoding running about 20x slower than partitioned codes on large inputs. Uniform coding also spends parity on road-surface points the network reconstructs anyway. The equal-budget comparison, with decode time measured on embedded hardware, settles whether the targeting matters or only the partitioning does.
Why not just retransmit lost packets? At V2X frame rates, a retransmission round trip competes with the frame deadline. NCRAWL's measurements of ACK/retransmission CPU overhead, and SDR-RDMA's analysis of retransmission under loss, both indicate cost at speed. We will include ARQ with one retransmission attempt under a fixed per-frame deadline and report deadline-miss rate alongside error.
Why not just protect regions Where2comm already marks as confident? Detection confidence marks regions that contain objects, whereas irreplaceability marks the specific points that cannot be inferred. A car's flat door panel is confidently detected yet highly predictable. The confidence-ranked baseline tests this difference directly.
Proposed Experiments
On KITTI Depth Completion we emulate a sensor-to-compute link. LiDAR points are packetized and passed through Gilbert–Elliott bursty loss traces, a standard two-state model alternating good and bad channel periods. Loss ranges from 1% to 40% at mean burst lengths of 1 to 8 packets. We also run sparsified 16-line inputs, where points are scarcer and irreplaceability should be higher. Metrics are RMSE, MAE, and boundary tMAE. We run the same protocol on NYUv2, with sparse samples at 500 points, to check indoor generality.
On OPV2V, and on DAIR-V2X for real-world localization noise, we reproduce BM2CP-style cooperative depth generation. Each neighbouring vehicle's points cross the lossy link, and we report [email protected] and [email protected] against bits sent. For every configuration we report decode latency and energy on an embedded ARM core.
The decisive check is a curve of boundary tMAE against bandwidth overhead $B$ for four configurations: interleave-only, uniform coding, confidence-ranked coding, and irreplaceability-ranked coding. The idea succeeds if, at 10–30% bursty loss, irreplaceability-ranked protection of a small fraction of points (we expect under 15%) reaches the boundary error of uniform coding at a fraction of its decode time and clearly beats interleave-only. The same result must hold on 16-line KITTI and on OPV2V AP. It fails if interleave-only or the geometric-variance proxy matches it throughout.
References
- BM2CP: Efficient Collaborative Perception with LiDAR-Camera Modalities
- Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps
- OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication
- Block-Diagonal and LT Codes for Distributed Computing With Straggling Servers
- Wireless Network Coding: Opportunities and Challenges
- A modular framework for implementing joint wireless network coding and scheduling algorithms
- Coded Computing at Full Speed
- Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network
- Non-local Spatial Propagation Network for Depth Completion
- Dynamic Spatial Propagation Network for Depth Completion
- Estimating Depth from RGB and Sparse Sensing
- Depth Coefficients for Depth Completion