Semi-supervised 3D detection trains a model on a few labeled scans plus many unlabeled ones. A "teacher" network guesses boxes on the unlabeled scans, and a "student" trains on those guesses, called pseudo-labels. The guesses are usually in roughly the right place but wrong in exact position, size and heading. Current methods respond by scoring each box and throwing away the ones that look bad. We propose to fix the boxes instead. A small attention-based network, the refiner, looks at the points around a coarse box and predicts a correction to its center, size and heading. The refiner is trained only on labeled scans, using ground-truth boxes that have been deliberately corrupted to resemble the teacher's typical mistakes. It never trains on pseudo-labels, so it cannot learn to agree with the teacher's errors. Running the refiner from several jittered starting boxes gives a second benefit: if the runs converge to the same box, the pseudo-label is probably reliable, and if they scatter, it is not.
The motivation is a well-documented mismatch. Classification confidence says little about how well a box is placed. Decoupled Pseudo-Labeling measures a correlation of only −0.196 between 2D score and depth error, and 3DIoUMatch finds objectness and class scores miss localization quality. Field practice still treats localization noise as a reason to discard data, when it is the kind of error a geometric refiner could correct.
Core Claim
Our claim is that pseudo-label noise in 3D detection is mostly correctable error in box position, size and heading, not unrecoverable error. A refiner trained outside the teacher–student loop can correct much of it. The quantity the refiner computes anyway, namely whether repeated refinement from different starts converges, is a better test of box reliability than a learned IoU head (a branch that predicts a box's overlap with the unknown true box). Concretely, let $r$ be the refiner and $b^{(m)}_0$ the teacher box perturbed by the $m$-th of $M$ random jitters. Let $b^{(m)}_K$ be the result after $K$ refinement steps. We score reliability as
$$s = \frac{2}{M(M-1)} \sum_{m<m'} \mathrm{IoU}_{3D}\big(b^{(m)}_K, b^{(m')}_K\big),$$
the mean pairwise 3D overlap of the converged boxes. The student trains on the average of the converged boxes, weighted by $s$. Two things are new relative to the file. First, pseudo-labels are actively corrected, including heading. Second, the corrector is kept out of the feedback loop by construction.
Relation to Prior Work
Several papers motivate the gap by showing that localization noise is the bottleneck, and all of them filter rather than repair. 3DIoUMatch adds an IoU-prediction head and uses it to gate and suppress boxes; we use the convergence score for the same gating role but also move the box. Adaptive pseudo-labeling with ACCS selects boxes by class-wise confidence percentiles and leaves regression untouched. Channel Augmentation with TED scores boxes by agreement across transformed copies of the same scan. That is the closest scoring idea to ours. The difference is that its agreement comes from the teacher's own correlated predictions, whereas ours comes from an independently trained corrector.
The nearest "repair" work is Diffusion-SS3D. It iteratively denoises box sizes and class labels with a diffusion model, a generator that learns to turn noise into samples step by step. However, it pins box centers to sampled surface points and explicitly drops heading because angle ambiguity destabilizes diffusion. We correct centers and heading directly. To keep heading stable, we regress a residual angle as sine and cosine inside a box-aligned frame, so the refiner never generates absolute orientation. PERE repairs uncertain pseudo-labels by deleting or replacing the points inside them, which edits the data rather than the box. DPL projects away noisy depth gradients and verifies depth with a flat-ground homography check, an assumption that fails on indoor tables and shelves.
The refiner's mechanism comes from Direction 1. Pointformer shows that attention can shift surface points toward object centers, which is the offset between where LiDAR returns land and where the box center is. DepthSSC shows that rotation, scale and translation can be predicted for queries. SimDistill uses deformable attention to correct geometric misalignment. Two findings constrain the design. The file notes that attention offsets drift when objects have very few points, and DepthSSC reports failure on small non-rigid classes. We therefore give the refiner an abstain output and rely on the convergence score to catch drift, rather than assuming attention always helps.
Self-training work (Tri-net, Self-Ensembling Attention Networks, Sener et al.) explains why the refiner must stay outside the loop. Errors that feed back into training compound, especially in early rounds.
Research Program
The first step measures what the teacher actually gets wrong. We split the labeled set into folds, train a teacher on all but one fold, and record the residuals between its boxes and ground truth on the held-out fold. The residuals cover center offset, size ratio and heading error, and are recorded per class and per bin of point count. This gives an empirical noise model that matches the teacher's real errors. We expect it to show errors that pull the center toward the visible surface and heading flips near 180°, neither of which isotropic jitter captures.
The second step builds the refiner. It takes the points inside an enlarged box, expressed in that box's coordinate frame. Learnable tokens for the center and the eight corners attend to these points through cross-attention and output a residual for center, size and heading, along with an abstain logit. The same weights are applied for $K$ steps. Training uses ground-truth boxes from labeled scans, corrupted by noise drawn from the measured noise model, under a curriculum of increasing corruption. Because each labeled object can be corrupted many times, even 1% label splits yield many training pairs. The refiner is frozen before self-training begins.
The third step plugs the frozen refiner into standard teacher–student pipelines. We use VoteNet indoors and PV-RCNN and TED outdoors, and change only how pseudo-labels are produced and weighted. The splits are artificial, so we have hidden ground truth for the "unlabeled" scans. That lets us measure pseudo-label quality itself, not just final student accuracy, across self-training rounds.
Baselines
The naive reference point is the supervised-only detector trained on the labeled split, together with plain teacher–student training at a fixed confidence threshold. All other results are reported as gains over these.
The strongest filtering competitor indoors is 3DIoUMatch, which uses triple-confidence filtering and IoU-guided lower-half suppression. It reports +7.7 [email protected] and +8.5 [email protected] over SESS on ScanNet at 10% labels. The file gives no absolute value, so we will rerun its released configuration on our splits. The strongest repair competitor is Diffusion-SS3D, which reports a 9.0-point [email protected] gain on SUN RGB-D at 1% labels and more than 5.4 [email protected] on ScanNet at 5% labels. Again, no absolute numbers were found on Lacuna for these settings. Outdoors, the strongest competitor is Channel Augmentation with TED, which reports +22.5% mAP over pretrained TED on KITTI at 1% labels and +19.9% at 2% labels. ACCS reports roughly a 24.3% relative gain at 10% labels.
For fairness, every method shares the same backbone, labeled and unlabeled splits (three seeds), number of self-training iterations, and hyperparameter search budget. Our compute total includes refiner training and the $M$-start refinement passes, and we report wall-clock time next to accuracy. Beating a baseline means a gain larger than seed variance on [email protected] or Moderate $AP_{3D}$. A gain only at the looser [email protected] would not count, because localization repair should show up most at strict thresholds.
Three ablations isolate the key ingredients. In the first, the refiner is trained on pseudo-labels inside the loop rather than frozen; this tests the confirmation-bias claim. In the second, the refiner is trained with isotropic jitter instead of the measured noise model. In the third, we keep the refiner's corrections but replace the convergence score with a 3DIoUMatch-style IoU head, which separates the value of correcting boxes from the value of scoring them.
Why not just…?
Why not just iterate the teacher's own second-stage box head? Two-stage detectors such as PV-RCNN already contain a box head that refines region proposals, and running it several times from jittered starts would provide both refinement and a convergence score at no extra cost. This is the most serious threat to the proposal. Our argument is that this head is co-trained on pseudo-labels within the loop and was trained only on the detector's own proposal distribution, so it should inherit the teacher's bias. That is an empirical claim, so we run this variant with identical $M$ and $K$. If it matches our frozen refiner, we will report that the contribution is multi-start convergence alone and drop the separate network.
Why not just average the teacher's boxes over augmented views? Test-time augmentation with box fusion is cheap, and Channel Augmentation already exploits agreement across transformations. Its views all pass through the same network, however, so systematic errors such as the pull toward the visible surface agree across views and survive averaging. We compare directly on hidden-ground-truth center error. The prediction is that averaging reduces random error but not this surface-biased center error, while the refiner reduces both.
Why not just soft-weight uncertain boxes or lower the threshold? Combating Noise weights the loss by region uncertainty and performs well in 2D. Down-weighting changes how much a box counts but never moves it. Most mistakes sit near IoU 0.5, so these boxes either pass on their misalignment or are effectively discarded. Our pseudo-label IoU histograms will show whether repair actually moves mass from the 0.3–0.5 range to above 0.7. If it does not, this alternative wins.
Proposed Experiments
We evaluate on ScanNet (10%, VoteNet) and SUN RGB-D (1% and 5%, VoteNet), which allow direct comparison with 3DIoUMatch and Diffusion-SS3D. We also evaluate on KITTI (1%, 2% and 10%, with PV-RCNN and TED), reporting Easy, Moderate and Hard $AP_{3D}$. SUN RGB-D and KITTI have oriented boxes and test heading repair. ScanNet has axis-aligned boxes and tests center and size repair only.
Three checks decide the outcome. The first is pseudo-label quality on the hidden ground truth: mean 3D IoU, center error, heading error, and recall at IoU 0.5 and 0.7, before and after refinement, broken down by point count. The second tracks these errors across self-training rounds for the frozen refiner and for the in-loop refiner. The confirmation-bias claim predicts that error stays flat for the frozen version and grows for the in-loop version. The third tests whether the convergence score $s$ predicts true IoU better than the 3DIoUMatch head, measured by rank correlation on held-out pseudo-labels. All three are reported for objects with fewer than ten points, because the file notes that attention refinement tends to fail on such sparse objects.
The proposal fails if refined pseudo-labels do not raise [email protected] or Moderate $AP_{3D}$ beyond seed variance over 3DIoUMatch and Channel Augmentation, or if iterating the teacher's own box head matches it.
References
- Decoupled Pseudo-labeling for Semi-Supervised Monocular 3D Object Detection
- 3DIoUMatch: Leveraging IoU Prediction for Semi-Supervised 3D Object Detection
- Semi-Supervised 3d Object Detection Via Adaptive Pseudo-Labeling
- Semi-Supervised 3D Object Detection With Channel Augmentation Using Transformation Equivariance
- Diffusion-SS3D: Diffusion Model for Semi-supervised 3D Object Detection
- Pseudo Label Refinery for Unsupervised Domain Adaptation on Cross-dataset 3D Object Detection
- 3D Object Detection With Pointformer
- DepthSSC: Monocular 3D Semantic Scene Completion via Depth-Spatial Alignment and Voxel Adaptation
- SimDistill: Simulated Multi-Modal Distillation for BEV 3D Object Detection
- Tri-net for Semi-Supervised Deep Learning
- Self-Ensembling Attention Networks: Addressing Domain Shift for Semantic Segmentation
- Learning Transferrable Representations for Unsupervised Domain Adaptation
- Combating Noise: Semi-supervised Learning by Region Uncertainty Quantification