A self-driving car's LiDAR (a spinning laser that measures distance by timing reflected pulses) produces a 3D point cloud that a neural network labels point by point as car, road, pedestrian, and so on. In rain, snow, or fog, many pulses never come back. The network then sees a car with most of its points missing, and that sparse remnant looks like a bush. The network does not become uncertain. It confidently calls the car a bush. Selective prediction, which lets a model abstain on inputs it is likely to get wrong, is the natural safety valve here. But its usual trigger is low output confidence, and that trigger is the one this failure disables.
This note proposes a different trigger. The information the network lacks sits in the points that are absent. A LiDAR fires a known, fixed grid of beams, so for every point cloud we also know which beams returned nothing and where the local point count falls short of what that range and elevation normally produce. Standard backbones throw this information away, because they only ever see the points that came back. We propose an abstention score that combines the network's confidence with these sensor-level "missing evidence" measurements. We calibrate it separately within each range band, so that the car does not simply give up on everything distant, and so that the error rate on accepted points carries a finite-sample guarantee. The central claim is testable: on real adverse-weather LiDAR, missing-return evidence catches confident errors that softmax, ensembles, and range-aware thresholds do not.
Core Claim
Two findings in the file set up the problem. Rethinking Data Augmentation for Robust LiDAR Semantic Segmentation in Adverse Weather broke weather damage into point drop, occlusion, geometric perturbation, and intensity distortion, and found that point drop and geometric perturbation cause most of the performance collapse. Adaptive Augmentation-Aware Latent Learning for Robust LiDAR Semantic Segmentation (A3Point) showed that heavy point drop produces "semantic shift": the degraded geometry really does resemble another class. These papers motivate the proposal. Together they imply that the most harmful corruption is also the one confidence scores are structurally blind to, since a semantically shifted input is not ambiguous to the network.
Our claim is that point drop is directly observable at inference time from the sensor's own scan pattern, independent of the learned features. A selection function that reads this signal will lower selective risk at equal coverage, and the gain will concentrate on the semantic-shift errors that confidence-based selection accepts.
Research Program
The first component computes sensor-evidence features for every returned point. We project each sweep into the LiDAR's native range image, which is a grid indexed by beam and azimuth. From that grid we compute three quantities over a local window around each point. The first is the fraction of beams with no return. The second is the ratio of the observed local point count to the count expected at that range and elevation. We estimate the expected count from clear-weather SemanticKITTI statistics, so it absorbs the ordinary drop in density with distance described in Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation and SemanticKITTI. The third is the count of near-sensor returns in empty space along the beam, which is the typical signature of spray, snowflakes, and edge-bleeding points. None of these features needs a new network. They cost roughly one pass over the range image.
The second component is a small selection head. It takes the frozen segmenter's softmax output together with the evidence features and predicts whether the segmenter's label is wrong. We train it on labeled calibration-free data that has been corrupted with physically simulated drop and jitter. We deliberately do not temperature-scale the segmenter. Learning Point-wise Abstaining Penalty for Point Cloud Anomaly Detection (LiON) reported a "calibration trap" in which standard calibration hurt LiDAR abstention, and that finding constrains our design.
The third component supplies the guarantee. It follows Selection with Guaranteed Risk (SGR) from Selective Classification for Deep Neural Networks, which picks an acceptance threshold on a held-out split using a binomial tail bound. We apply SGR separately within each range band $b$. For a target risk $r^*$ and failure probability $\delta$, the threshold $\theta_b$ is chosen so that
$$ \Pr\Big[\, \mathbb{E}\big[\ell(f(x),y)\mid g_{\theta_b}(x)=1,\ \rho(x)\in b\big] > r^* \Big] \le \delta, $$
where $f$ is the segmenter, $\ell$ is the 0/1 point error, $g_{\theta_b}$ accepts a point when its selection score exceeds $\theta_b$, and $\rho(x)$ is the point's range. The guarantee holds only when the calibration and test data are exchangeable. We therefore report two settings. In the first, we calibrate on a held-out SemanticSTF split, where the guarantee applies. In the second, we calibrate on corrupted SemanticKITTI and test on SemanticSTF, where the guarantee can fail and we measure by how much.
Abstention happens at two levels. At the point level we accept or reject each point. At the object level, for "thing" classes, we group points into connected predicted segments and abstain on a segment when the mean point score inside it falls below threshold. An object-level abstention is what a planner would actually act on.
This proposal differs from the closest prior work in three ways. LiON learns a per-point abstention penalty, but it targets out-of-taxonomy objects synthesized from ShapeNet CAD models. We target errors on in-taxonomy classes under real weather, and we use sensor evidence that LiON's features never see. PointCVaR scores points by the tail of the network's attribution gradients and deletes the risky ones, but it still forces a label at 100% coverage. It also cannot flag points that are missing, because it can only remove points that exist. PointASNL and TANet suppress noise inside the architecture and expose no residual uncertainty. Our method is compatible with any of these backbones.
Baselines
The naive reference point is the segmenter at full coverage with no abstention. Its point error rate on SemanticSTF sets the ceiling that any abstention method must beat at every coverage level.
The strongest predictor we can build on is MinkowskiNet trained with Selective Jittering and Learnable Point Drop. Rethinking Data Augmentation reports 39.5 mIoU on SemanticKITTI → SemanticSTF for this model. UniMix reports 37.2 mIoU in its domain-adaptation setting. Every selection method will sit on top of this same frozen, augmentation-trained segmenter, so that differences reflect the selection rule and not the predictor. A Cylinder3D backbone serves as a replication check. We found no reported result on Lacuna for selective risk, risk-coverage curves, or area under the risk-coverage curve on SemanticSTF or SemanticKITTI-C, so we will establish the reference numbers ourselves. The only reported SGR figure in the file, 1% error at 78% coverage on CIFAR-10, is not comparable to our setting.
The strongest existing selection method is a set of confidence scores run through the same range-banded SGR. These are softmax response and entropy (the approach EHRSQL uses for clinical queries), MC-dropout variance as in Tousignant et al., and a five-member deep ensemble. We add a LiON-style learned abstention head trained on backbone features only, with the same augmented data and training budget as our head. All selection heads get the same hyperparameter search budget and the same calibration split. Our method beats this group if it lowers area under the risk-coverage curve and achieves higher guaranteed coverage at $r^*$ values of 5% and 10% in both the near and far range bands, with no band dropping below the baselines' coverage.
Three ablations isolate the key ingredient. The first removes the evidence features and keeps range-banded SGR. The second keeps the evidence features but uses a single global threshold. The third feeds the ray-drop map to the backbone itself as an extra input channel and uses plain softmax on top. The third ablation is the critical one, as explained below.
Why not just…?
Why not just use range-banded thresholds on softmax? Banding by range removes the obvious failure where a single threshold rejects everything distant, and it may recover much of the gain. It is our first ablation. However, A3Point's semantic-shift finding predicts that confident errors from point drop occur at every range, since weather drop comes on top of range sparsity. If softmax with range-banded SGR matches our method on SemanticSTF, the evidence features add nothing and we will report that result. The decisive diagnostic is the share of confidently accepted errors that fall in high-drop regions.
Why not just train a learned confidence head on the backbone's features, as LiON does? A voxel or point backbone receives only the points that returned, so it has no direct view of beams that returned nothing. It can at best infer absence from low density, and low density is confounded with ordinary range falloff. The head-on-features baseline tests this. If it matches our head, the backbone already encodes the missing-return information implicitly.
Why not just give the backbone the ray-drop map as input and keep softmax selection? This is the most serious threat, because it could produce a better segmenter and a better-ordered softmax in one step. We run it as the third ablation. If it matches our method, the contribution shrinks to "feed the missing returns to the network." That would still be a useful finding, but a smaller one, and we would report it that way. We expect it to fall short for a specific reason. A3Point argues that training a network to map heavily dropped geometry onto its true label corrupts feature learning, so the network is pushed to stay confident. A separate error predictor has no such pressure.
Why not just denoise or augment harder? Denoisers such as Score-Based Point Cloud Denoising move existing points but cannot restore dropped ones, and their iterative refinement is slow. The best augmented models still sit below 40 mIoU on SemanticSTF. Our method uses the strongest augmented model as its predictor, so it builds on augmentation rather than competing with it.
Proposed Experiments
The primary benchmark is SemanticKITTI → SemanticSTF. We hold out part of the SemanticSTF labeled data as a calibration split, and we stratify every result by weather type (fog, rain, snow) and by range bands of 0–20 m, 20–40 m, and beyond 40 m. The metrics are area under the risk-coverage curve, coverage attained at guaranteed risk $r^*$ with $\delta = 0.05$, the realized violation rate over resampled calibration splits, and coverage on safety-critical classes (pedestrian, cyclist) both overall and at range.
SemanticKITTI-C lets us break results down by known corruption type. We predict gains concentrated in drop-type corruptions and near zero on pure jitter. ModelNet-C from Ren et al. serves as a negative control. Its objects have no scan pattern, so no ray-drop features exist there, and any gain on it would point to a leak in our pipeline rather than a real effect.
Three checks decide whether the claim holds. First, among errors that softmax selection accepts at 80% coverage, a majority should fall in the top quartile of our drop score, and our method should reject them at higher rates. Second, our method should beat the backbone-with-ray-drop-channel ablation on area under the risk-coverage curve. Third, in the cross-domain calibration setting, we report how far realized risk exceeds $r^*$ for every method. We expect evidence features to shrink that excess, because missing returns are a physical quantity whose meaning stays the same when the weather changes, whereas a network's confidence does not. If none of the three checks holds, the idea fails, and the simpler range-banded softmax rule should be the one deployed.
References
- Rethinking Data Augmentation for Robust LiDAR Semantic Segmentation in Adverse Weather
- Adaptive Augmentation-Aware Latent Learning for Robust LiDAR Semantic Segmentation
- Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation
- SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences
- Learning Point-wise Abstaining Penalty for Point Cloud Anomaly Detection
- Selective Classification for Deep Neural Networks
- PointCVaR: Risk-Optimized Outlier Removal for Robust 3D Point Cloud Classification
- PointASNL: Robust Point Clouds Processing Using Nonlocal Neural Networks With Adaptive Sampling
- TANet: Robust 3D Object Detection from Point Clouds with Triple Attention
- UniMix: Towards Domain Adaptive and Generalizable LiDAR Semantic Segmentation in Adverse Weather
- EHRSQL: A Practical Text-to-SQL Benchmark for Electronic Health Records
- Prediction of Disease Progression in Multiple Sclerosis Patients using Deep Learning Analysis of MRI Data
- Score-Based Point Cloud Denoising
- Benchmarking and Analyzing Point Cloud Classification under Corruptions