Self-explaining classifiers justify a decision by pointing at evidence. A prototype network says "this patch of the input looks like that patch of a training example." These models are now being moved from images to 3D point clouds. The parts of a point cloud that look most distinctive (sharp edges, spikes, thin protrusions) are also where real depth sensors fail. When a laser footprint or camera pixel straddles a foreground edge and the background behind it, the sensor reports "flying pixels": points that float in empty space between two surfaces. When the viewing ray grazes a surface almost tangentially, small errors become large 3D displacements. We hypothesize that an interpretable 3D model trained on real scans will often pick these artifacts as its class-defining prototypes. Its explanations would then be faithful to the model but unfaithful to the object.
We propose to measure this failure and to repair it with a small, inspectable "trust gate." Each point is scored from a few physical cues that the outermost-point-filtering literature already uses as hard thresholds: the angle between the sensor ray and the surface, the depth jump to neighboring pixels, and whether other views confirm the point. Only trusted patches may serve as prototypes or contribute to matching. The central bet concerns the trade-off between accuracy and interpretability. A learned soft gate on physical cues should remove artifact prototypes about as well as an aggressive hand-tuned pre-filter, while keeping the thin real structures (chair legs, lamp stems, poles) that such filters are known to destroy. The gate's few weights stay readable as part of the explanation.
Core Claim
There are two claims, and the first can fail independently of the second. First, an unmodified prototype point-cloud classifier trained on scans containing sensor edge artifacts places a substantial fraction of its prototypes on artifact points. We call this fraction the artifact prototype rate. It is higher than the share of artifact points in the data. Second, a gate of the form
$$g_i = \sigma\left(w_0 - w_1\,\cos\theta_i^{-1} - w_2\,\rho_i + w_3\,s_i\right),$$
conditions prototype matching and prototype projection. Here $\theta_i$ is the angle between point $i$'s viewing ray and the depth-image surface direction, $\rho_i$ is the ratio of the local depth jump to the pixel footprint, $s_i$ is the number of other views that confirm the point, $\sigma$ is the logistic function, and the weights $w_1, w_2, w_3 \ge 0$ are constrained so that the gate can only distrust points for physically sensible reasons. With this gate, the artifact prototype rate falls to near the level of a tuned hard filter, thin-structure retention improves over that filter, and accuracy on real scans stays within a small margin of an equally sized black-box model. We compute the cues in the 2D depth image before back-projection. This sidesteps the known instability of nearest-neighbor normal estimation at sharp boundaries.
Where this sits
The geometric side supplies the cues, and its limitations motivate the work. Fixed-threshold boundary filters appear across many pipelines. Examples include discarding mesh triangles with long projected edges in Ambient point clouds for view interpolation, applying a depth-discontinuity filter before volumetric fusion in Efficient 3D Reconstruction and Streaming for Group-Scale Multi-Client Live Telepresence, removing points whose rays lie within 20° of the surface in Building Footprints Extraction from Oblique Imagery, and pruning projected background "point bleeding" in ZOPP. All of these work as decoupled preprocessing. All carry the documented risk of deleting thin genuine structure. None asks what a downstream model, interpretable or not, does with the artifacts it lets through. We reuse their cues but turn each threshold into a learned, signed weight. Geometric measure theory offers stability guarantees for curvature-based cleaning (Stability of Curvature Measures), but these assume positive reach. Sharp, occluded scan geometry violates that assumption, so this route constrains the problem without solving it.
The interpretable side supplies the method and the failure mode. This looks like that introduced prototype layers whose prototypes are periodically projected onto real training patches. That projection step is where an artifact can become an "explanation," and it is the step we constrain. Concept Bottleneck Models route predictions through named concepts, but 3D scans have no concept labels, so we work with prototypes. Two 3D interpretable models are the closest prior work, and both were evaluated only on clean CAD shapes. PointMask learns a free per-point keep-probability with an information-bottleneck penalty and resists injected synthetic bias. Its gate is unconstrained by physics, and real sensor artifacts are never tested. GPr-Net builds prototypes from hand-designed high-frequency geometric signatures, and its authors flag its nearest-neighbor grouping as vulnerable to outliers. Those same high-frequency signatures are exactly what flying pixels mimic. Attention-Based Point Cloud Edge Sampling shows that edge points carry most of the class information. That finding sharpens the threat, because spurious edges would score highly too. Revisiting Point Cloud Classification motivates the real-data regime. Its background-aware branch separates object from clutter, but it does so inside a black box and without sensor geometry. Our proposal differs from all of these in one concrete way: the explanation unit (the prototype) may only be grounded on points that a physically constrained, human-readable gate trusts.
Research Program
We first build a controlled testbed with ground-truth artifact labels. Using a virtual depth scanner, we re-render ModelNet40 meshes from multiple views. The scanner simulates beam-footprint mixing at depth discontinuities, which produces flying pixels, and range noise that grows with grazing angle. We keep the camera poses and per-pixel labels. For each mesh we also mark thin-structure points: mesh regions whose local thickness is below a few pixel footprints. For real data, we work from the RGB-D sources behind ScanObjectNN. These are ScanNet and SceneNN, whose frames carry camera poses. We re-extract object point clouds with per-point rays and multi-view support counts. Where an exact correspondence to ScanObjectNN instances cannot be recovered, we report results on a re-extracted ScanNet object subset that uses ScanObjectNN's 15 classes. In the real data, artifact labels are not available. We use a proxy instead: a point counts as an artifact if no other view within tolerance confirms it. We validate this proxy on the synthetic set, where the truth is known.
The model has a PointNeXt-S encoder (PointNeXt) and a prototype layer on its local grouped features. Prototype similarities are combined by a linear class layer, as in ProtoPNet. The gate enters in two places. First, a patch's feature is gate-weighted when pooled, so distrusted points contribute little to similarity. Second, during the prototype projection step, a prototype may only be projected onto a training patch whose mean gate exceeds a threshold. The gate has four weights and is trained jointly with the rest of the model. A light penalty keeps it near 1 on average, so it cannot discard the object wholesale. Because the gate has only a handful of signed weights, it can be printed and checked. A user can see, for example, that it distrusts points with grazing angle beyond roughly some learned angle.
Baselines
The naive reference point is the same prototype network with no gate and no filtering, trained on raw artifact-laden scans. It establishes the artifact prototype rate that the rest of the study has to explain. If this rate is not clearly higher than the fraction of artifact points in the data, the first claim fails, and we will report that.
The strongest accuracy competitor is PointNeXt itself, the black-box version of our own encoder. On ScanObjectNN's hardest variant, PB_T50_RS, it reports 87.7% overall accuracy (PointNeXt). For context, BGA-PN++ reports 80.2% on that variant (Revisiting Point Cloud Classification). We found no reported result on Lacuna for any prototype-based point-cloud classifier on ScanObjectNN, and none for PointMask or GPr-Net on real scans, so we run those ourselves. All models share the same encoder size, augmentation recipe, epochs, and hyperparameter search budget. On the original ScanObjectNN split we expect to be compared against 87.7%. On our re-extracted data we will retrain PointNeXt under identical conditions. Staying within about two points of accuracy of it while cutting the artifact prototype rate counts as an acceptable trade-off. Matching it would be a strong result.
The strongest filtering competitor is a hard pre-filter followed by the same prototype network. The filter applies grazing-angle, depth-jump-ratio, and multi-view-support thresholds of the kind used in Building Footprints Extraction from Oblique Imagery and Efficient 3D Reconstruction and Streaming. To give it its best chance, we tune its thresholds by grid search on the validation set, with the same budget the gate receives. Beating it means a lower or equal artifact prototype rate together with higher thin-structure retention and per-class accuracy on thin-part classes, at matched overall accuracy.
The learned-masking competitor is PointMask's variational per-point gate inserted at the same location (PointMask). It tests whether physics is needed at all. We also run GPr-Net on the synthetic artifact set, because its high-frequency signatures make it the model most exposed to the failure we hypothesize.
Two ablations isolate the key ingredients. The first replaces the four-weight physical gate with an unconstrained per-point network that receives the same cues. This tests whether the sign constraints and low capacity matter, or whether they merely cost accuracy. The second keeps the gate in pooling but removes the restriction on prototype projection. This tests whether grounding prototypes is the operative mechanism.
Why not just…?
Why not just pre-filter with tuned thresholds and then train any interpretable model? This is the real threat, and on artifact removal alone it may well match us. The literature's own stated limitation is that hard thresholds on depth gradients and elongation delete thin real structures. The tuned-threshold baseline lets us measure that cost directly, using the synthetic thin-structure labels. If the tuned filter matches the gate on both artifact prototype rate and thin-structure retention, we will conclude that the gate is unnecessary. The contribution would then reduce to the diagnostic finding and a recommended filtering protocol for interpretable 3D models.
Why not just let an information-bottleneck mask learn what to drop, as PointMask does? A free mask keeps whatever predicts the label. On real scans, boundary artifacts can be label-correlated: a chair's thin legs systematically produce flying pixels against the floor. In that case a free mask has no reason to discard them. PointMask's evidence comes from synthetic bias on CAD models, not from sensor artifacts. The PointMask baseline and the unconstrained-gate ablation test this directly. If either matches the physical gate on artifact prototype rate, the physical constraint is not earning its place.
Why not just train PointNeXt and explain it with saliency maps? Saliency shows where a model looks, not which reference evidence it uses, and it can be unfaithful to the model's actual computation (Self-Explainable Neural Networks without Performance Loss, This looks like that). We still report saliency mass on artifact points for PointNeXt. If the black box concentrates on artifacts too, that strengthens the case that the problem is not specific to prototypes.
Why not just use the background-aware segmentation branch? That branch separates object from clutter (Revisiting Point Cloud Classification). Flying pixels, however, sit on the object's own boundary, not in the background. We include BGA-PN++ in the real-data comparison and measure its artifact saliency.
Proposed Experiments
We report overall and mean-class accuracy on the synthetic artifact set and on the real re-extracted scans. We report the artifact prototype rate: the fraction of prototypes whose projected training patch is mostly artifact points, scored against ground-truth labels in the synthetic data and against the multi-view proxy in the real data. We report thin-structure retention, the fraction of thin-part points with gate above threshold or surviving the filter. Finally, we run a deletion test. We remove each top prototype's supporting points from test inputs and measure the drop in confidence. If explanations are faithful to the object, removing real parts should hurt and removing artifact support should not.
The first decisive check is the ungated prototype network on the synthetic set. Its artifact prototype rate must exceed the base rate of artifact points, and the excess should be largest for GPr-Net's high-frequency features. The second check is the comparison against the tuned hard filter on thin-part classes such as chair, lamp, and plant. The gate must retain clearly more thin-structure points without a higher artifact prototype rate. The third check transfers the gate, trained only on simulated artifacts, to real scans. The learned weights should stay physically sensible. In particular, the grazing-angle weight should keep its sign and roughly its magnitude, and the multi-view proxy should agree with ground truth well enough on synthetic data to trust it on real data. If the first check fails, we publish the negative diagnostic. If the second fails, we publish the tuned filter as the recommended practice and drop the gate.
References
- Ambient point clouds for view interpolation
- Efficient 3D Reconstruction and Streaming for Group-Scale Multi-client Live Telepresence
- Building Footprints Extraction from Oblique Imagery
- ZOPP: A Framework of Zero-shot Offboard Panoptic Perception for Autonomous Driving
- Stability of Curvature Measures
- This looks like that: deep learning for interpretable image recognition
- Concept Bottleneck Models
- PointMask: Towards Interpretable and Bias-Resilient Point Cloud Processing
- GPr-Net: Geometric Prototypical Network for Point Cloud Few-Shot Learning
- Attention-Based Point Cloud Edge Sampling
- Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data
- PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies
- Adaptive Concept Discovery for Interpretable Few-Shot Text Classification