TL;DR
CLOCs-R extends the candidate-level fusion framework by incorporating radar data to assist camera and LiDAR in long-range object detection. While it successfully boosts detection accuracy at mid-ranges (30–50m), the study reveals that sparse radar returns alone struggle to distinguish distant vehicles from environmental noise, highlighting a significant "recall bottleneck" in late-fusion systems.
What Problem Does This Paper Address?
For autonomous vehicles to operate safely at highway speeds, they must detect objects hundreds of meters away. However, standard sensors have a "long-range problem":
- LiDAR becomes extremely sparse with distance; a car that returns 2,000 points at 14m might return only 39 points at 63m.
- Cameras lose depth precision as objects get smaller in the frame.
- Radar can see out to 250m and measure velocity directly, but its data is "noisy," lacks height information, and often produces "ghost" returns from static objects like guardrails.
The authors aim to bridge this gap by adding radar evidence to CLOCs (Camera-LiDAR Object Candidates), a framework that rescores potential detections by looking at how well 2D camera boxes and 3D LiDAR boxes overlap.
Methods
CLOCs-R uses a "candidate-level" (or late) fusion approach. Instead of fusing raw sensor data, it takes the outputs of pre-trained, "frozen" detectors and tries to improve their confidence scores.
The pipeline works as follows:
- Candidate Generation: A 3D LiDAR detector (SECOND) and a 2D camera detector (Cascade R-CNN) propose potential objects.
- Radar Association: Radar returns are accumulated over multiple sweeps and compensated for the car's own motion. These returns are matched to 3D candidates using a "gate" that widens with distance to account for sensor error.
- Feature Extraction: For every 3D candidate, the system calculates five radar features:
- Support: How many radar points fell in the box.
- Doppler Speed & Spread: The median velocity and how much that velocity varies (to distinguish moving cars from static clutter).
- Range Residual & Offset: How well the radar's distance measurement aligns with the 3D box's position.
- Fusion & Rescoring: These features are appended to a "pair tensor" representing camera-LiDAR overlaps. A small neural network then learns to output a new, more accurate confidence score for each 3D box.
Datasets
- KITTI: Used primarily to verify that the implementation could reproduce the original CLOCs results.
- aiMotive: The primary evaluation dataset. It is specifically designed for long-range perception, featuring annotations out to 200m, which is significantly further than standard datasets like nuScenes (70m) or KITTI (70.4m).
Key Insights and Ideas
- The "Recall Bottleneck": Because CLOCs-R is a late-fusion method, it can only rescore objects that the 3D LiDAR detector already found. If the LiDAR detector misses a distant car entirely because it only has 2 or 3 points, no amount of radar fusion can recover it.
- Residual Scoring is Essential: The authors found that simply replacing the original 3D detector's score with a fusion score actually hurt performance at long range. They introduced "residual scoring"—adding the fusion network's output to the original score—which allowed the model to keep the 3D detector's strengths while using radar/camera data as a corrective signal.
- Radar's Identity Crisis at Range: At 100–200m, radar returns from real cars look remarkably similar to returns from false detections (noise). The "Doppler speed" feature helps, but the signal-to-noise ratio is much lower than at closer ranges, making it a weak discriminator for the fusion network.
Key Results
- Mid-Range Boost: Radar consistently improved Bird’s-Eye-View Average Precision (BEV AP) in the 30–50m range, raising it from 0.857 to 0.870.
- Long-Range Marginality: Beyond 50m, the gain from radar was small (+0.003 in center-distance AP). At these distances, the system was essentially "statistically tied" with the LiDAR-only baseline.
- Efficiency: The radar feature extraction and fusion network add only about 19ms of latency per frame (on a single CPU thread/GPU), making it a lightweight addition to existing perception stacks.
Limitations and Future Work
The primary limitation is the recall ceiling mentioned above; future work might explore "cued" detection where radar proposals tell the 3D detector where to look more closely. Additionally, the authors noted a "label shift" between training and validation data that made strict IoU metrics difficult to satisfy.
The paper suggests that better long-range performance will require more balanced training data (currently, only 5% of training positives are in the 100–200m range) and perhaps earlier-stage fusion to help the detectors propose candidates they would otherwise miss.
Summary updated