A lidar sensor (a laser scanner that returns a cloud of 3D points) sees a parked truck 150 m ahead as two or three points. Five seconds later, as the car drives toward it, the same truck appears as hundreds of points. Driving logs therefore already pair sparse far-field views with dense near-field views of the same static scene, and nobody has to annotate that pairing. We propose to use it as a self-supervised target. A masked autoencoder, which is a network trained to fill in deliberately hidden parts of its input, sees a single masked lidar sweep. It must predict what that region will look like once later sweeps, aligned to the current frame using the vehicle's measured motion, have filled it in. The model learns to complete far-field geometry from real future observations, not from simulated degradation. At deployment it still takes one sweep and adds no latency.
This matters because long-range perception fails in two ways at once. Point density falls off roughly with the square of distance, and more than 85% of annotations sit within 100 m (Towards Long-Range 3D Object Detection for Autonomous Vehicles). Label-efficient pretraining is the natural answer to scarce labels. Existing lidar masked autoencoders, however, reconstruct the current sweep, so their target is no denser than their input exactly where density is the problem.
Core Claim
Our central claim is that future-densified reconstruction gives larger gains than current-frame reconstruction specifically in the 100–225 m range, under 1–10% label budgets. We expect the gain to grow with distance because the information gap between a far-field input and its future-densified target grows with distance, while near-field targets change little.
The closest prior work reconstructs the present. MAELi thins range-image scan lines to imitate lower angular resolution and splits space into occupied, empty, and unknown voxels so that occluded regions go unpenalized. Its thinning is uniform across range, and its target is the current sweep. BEV-MAE masks top-down grid cells and predicts point density as a proxy for distance. NOMAE predicts occupancy in fixed 9×9×9 voxel neighborhoods, which are too small to span the gaps between far-field points. T-MAE is the only temporal method. It reconstructs the current frame using a past frame 0.3 s earlier as a reference, which gives a short window and no densification target, and it reports no distance-stratified results.
We reuse two things from this work. From MAELi we take the occupied/empty/unknown loss design. From T-MAE we take the lesson that ego-motion supplies free extra views. What none of these methods does is make the target denser than the input in a way tied to distance.
Research Program
Building the target. For an anchor sweep at time $t$, we transform the sweeps from $t$ to $t+K$ into frame $t$ coordinates using the dataset's ego-poses, with $K$ spanning up to about 5 s. Each voxel is then labeled by ray casting:
- Occupied if it receives returns in a consistent majority of the sweeps that observe it.
- Empty if rays pass through it consistently.
- Unknown if it is never observed, or if it is occupied in some sweeps and traversed in others.
The inconsistent case is the important one. Moving objects produce exactly this pattern, so they fall into unknown space instead of producing the smeared "ghost" geometry that raw frame stacking creates. This is the failure T-MAE cites as its reason for working in representation space instead of stacking points.
Training. The encoder sees only the masked anchor sweep, which is what it will see at test time. The decoder predicts target occupancy, and the loss is
$$\mathcal{L} = \sum_{v \in \mathcal{O} \cup \mathcal{E}} w(d_v)\,\mathrm{BCE}\big(\hat{o}_v, o_v\big),$$
where:
- $\mathcal{O}$ and $\mathcal{E}$ are the target's occupied and empty voxels;
- $o_v \in \{0,1\}$ is the target occupancy of voxel $v$, and $\hat{o}_v$ is the predicted probability that it is occupied;
- $\mathrm{BCE}$ is binary cross-entropy;
- $w(d_v)$ is an optional weight that depends on the voxel's range $d_v$. It defaults to 1 and is studied as an ablation.
Unknown voxels incur no loss, following MAELi.
Decoder. To avoid the memory cost of dense 3D decoding, we use a lightweight top-down decoder in the style of BEV-MAE. It predicts a column of height bins per grid cell.
Analysis. Three questions drive the analysis:
- How the gain depends on range and on the horizon $K$. We expect diminishing returns once the vehicle has passed the object.
- How sensitive the method is to ego-pose error. We inject rotation noise into the alignment, because a fraction of a degree displaces a target at 200 m by meters.
- What happens to moving objects. They supply no target, so if gains on moving vehicles appear, they would show that static-scene training transfers to moving objects.
Baselines
Do nothing. The naive reference point is the same detector trained from scratch on each label fraction. MAELi reports +10.79 mAP over from-scratch with 1% of Waymo labels. BEV-MAE reports +7.22 mAP with 5%. These numbers show how much room pretraining can open up.
Strongest self-supervised competitors. These are MAELi, BEV-MAE, and T-MAE. Each is pretrained on the identical unlabeled Argoverse 2 lidar sequences with the same backbone, the same pretraining compute (matched GPU-hours), and the same fine-tuning schedule and tuning budget. MAELi is the most dangerous of the three because it already models sensor thinning and unknown space. T-MAE is the only one that uses temporal context.
Strongest long-range method. This is Range Experts with range-adaptive loss weighting, plus camera-generated virtual points, from Towards Long-Range 3D Object Detection for Autonomous Vehicles. On Argoverse 2 vehicles at 100–150 m it reports 39.3% AP, against 29.2% for the plain FSD detector. That result uses full supervision and adds more than 1700 ms per frame. We found no reported result on Lacuna for label-efficient self-supervised pretraining on Argoverse 2 at any range. Our own runs of MAELi and BEV-MAE will therefore be the label-efficient comparison points.
What counts as beating them. We count it as a win if, at 5% labels:
- our 100–150 m and 150–225 m vehicle AP exceeds the best matched self-supervised baseline by a margin larger than the seed-to-seed spread across three seeds;
- 0–50 m AP stays within that spread;
- our 10%-label far-range AP approaches the 39.3% full-supervision figure without the camera pipeline.
Ablations isolating the key ingredient. The first ablation replaces the future-densified target with the current sweep, keeping everything else fixed. That reduces our method to MAELi-style reconstruction on our decoder. The second ablation is described under the first "Why not just…?" question, because it is the main threat to the proposal.
Why not just…?
Why not just simulate far-field sparsity by thinning near objects in proportion to range, and reconstruct the dense original? This is the obvious extension of MAELi's spherical masking to distance-dependent masking, and it is a real threat. It needs no sequences and no pose alignment. Our bet is that real far-field returns differ from thinned near-field returns in ways simulation misses: partial surfaces, edge-dominated hits, intensity falloff, and beam-footprint mixing. We will test it directly. We will train a version with range-proportional synthetic thinning and a dense current-frame target, matched in compute. If it equals our far-range AP, the simpler method wins and we will report that. A hybrid that uses synthetic thinning plus the future target would also tell us whether the two are complementary.
Why not just stack past sweeps at the input? Stacking raw sweeps densifies static structure but smears moving objects, as T-MAE notes. It also depends on pose accuracy at test time. Our method uses the future only to build training targets, and the consistency filter keeps dynamic smear out of the loss. Every baseline and our model will be run both with single-sweep input and with the standard short multi-sweep input. This tests whether the pretraining gain survives once the input is already somewhat densified.
Why not just reweight distant errors or split near and far experts? Loss reweighting cannot add geometry that a three-point cluster lacks. In the long-range study, the large gain came only when camera virtual points densified the input, not from Range Experts alone. We will add range-adaptive weighting on top of both the from-scratch and pretrained models to check whether the two effects are additive.
Why not just use 2D foundation-model pseudo-labels, as MixSup does? MixSup reaches 97.31% of full supervision on Waymo with 10% boxes plus SAM-derived clusters. It also notes that cluster centers drift for distant, partially observed objects, and its pipeline depends on calibration and lighting. Camera detection mAP falls 55–58% at night on nuScenes. We will report MixSup-style fine-tuning on top of each pretrained backbone. This tests whether better far-field geometry reduces pseudo-label damage at long range, which is the open tension between decoupled-supervision and reconstruction work.
Proposed Experiments
Data and pretraining. We pretrain on the unlabeled sequences of the Argoverse 2 lidar dataset. We fine-tune on the Argoverse 2 sensor dataset with 1%, 5%, 10%, and 100% of labeled frames. We report AP in four range bins: 0–50, 50–100, 100–150, and 150–225 m. FSD is the detector, so results are comparable to the 29.2% and 39.3% reference points.
Second sensor. We repeat the 5% setting on Waymo, the same budget at which BEV-MAE and T-MAE report results, re-binning by range from their protocols. We will also run a transfer test from Waymo pretraining to nuScenes fine-tuning, because cross-sensor transfer is BEV-MAE's acknowledged weakness.
Decisive checks. The proposal stands or falls on three results:
- The far-range gain over the current-frame-target ablation increases monotonically with range.
- The synthetic-thinning control does not close that gap.
- Gains degrade gracefully under injected pose noise up to the error levels typical of production odometry.
If the gain is flat across range, the idea is wrong. Any benefit would then be generic pretraining and not far-field completion.
References
- Towards Long-Range 3D Object Detection for Autonomous Vehicles
- MAELi: Masked Autoencoder for Large-Scale LiDAR Point Clouds
- BEV-MAE: Bird's Eye View Masked Autoencoders for Point Cloud Pre-training in Autonomous Driving Scenarios
- Multi-Scale Neighborhood Occupancy Masked Autoencoder for Self-Supervised Learning in LiDAR Point Clouds
- T-MAE : Temporal Masked Autoencoders for Point Cloud Representation Learning
- MixSup: Mixed-grained Supervision for Label-efficient LiDAR-based 3D Object Detection
- nuScenes: A multimodal dataset for autonomous driving
- Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting