Abstract
As cellular networks transition toward 6G, "joint communication and sensing" (JCAS) is becoming a critical capability, enabling base stations to act as radar systems. A major hurdle in deploying these systems is the difficulty of labeling radio heatmaps for training. This paper presents a novel self-supervised framework, Masked Contrastive Learning (MCL), which uses synchronized camera data to teach radio networks how to localize objects. By aligning cross-modal representations and utilizing visual masks to isolate targets, the system generates high-quality self-labels for training radio-only localizers. Our approach achieves state-of-the-art performance on both synthetic (MaxRay) and empirical (CRUW) datasets, demonstrating a viable path for autonomous sensing in future networks.
Introduction: The Challenge of Radio Sensing
The evolution of 6G promises a shift from pure communication to integrated sensing, where infrastructure can detect pedestrians, vehicles, and drones. However, deep learning for radio sensing is bottlenecked by data annotation. Unlike visual images, raw radio heatmaps are unintuitive for humans, making manual labeling nearly impossible. While synthetic data offers a starting point, models must eventually adapt to real-world environments where ground truth is unavailable. This research addresses the "label-free" localization challenge by leveraging the shared physical geometry between co-located cameras and radio sensors.
Methods: Masked Contrastive Learning (MCL)
The proposed solution is a three-stage pipeline designed to transfer knowledge from the visual domain to the radio domain without human intervention.
- Cross-Modal Alignment: The system employs two backbone neural networks—a visual encoder and a radio encoder. Through contrastive learning, these networks are trained to produce similar mathematical representations when observing the same scene.
- Visual Target Masking: To prevent the model from learning irrelevant background features, MCL introduces a "visual target mask." By using standard object detectors (e.g., YOLO) on the camera feed, the system forces the radio network to focus specifically on the features corresponding to the identified object.
- Self-Labeling and Deployment: Once aligned, the visual encoder generates "self-labels" (pseudo-labels) for the radio data. These labels are used to train a dedicated radio-only localizer network. Post-training, the camera is no longer required, allowing the system to perform localization using only radio waves.
Datasets: MaxRay and CRUW
The framework was validated using two distinct datasets:
- MaxRay: A custom, high-fidelity synthetic dataset generated via ray tracing in Blender. It simulates 6G conditions at 28GHz with 800MHz bandwidth across diverse urban scenarios, providing perfect ground truth for benchmarking.
- CRUW: A real-world dataset featuring synchronized camera and radar data from a mobile platform, used to test the model's robustness against empirical noise and lower resolutions.
Key Results
The MCL approach establishes a new state-of-the-art for label-free localization across both synthetic and real-world benchmarks.
- Synthetic Performance: On the MaxRay dataset, MCL achieved a median localization error of 0.94 meters, a significant improvement over traditional Constant False Alarm Rate (CFAR) methods (2.7 meters).
- Real-World Robustness: On the CRUW dataset, MCL reached a median error of 2.56 meters, outperforming the previous leading self-supervised method, RODNet (3.28 meters).
- Self-Correction and InfoMin: A key finding was that the final localizer network often outperformed the noisy self-labels it was trained on. Furthermore, the study validated the "InfoMin" principle; providing a small amount of context around the visual target (a positive mask offset) yielded the best localization accuracy, as seen in the performance curves.
Discussion and Future Work
The success of MCL highlights the potential of "vision as a teacher" for radio sensing. The authors noted that performance is currently limited more by hardware resolution (antenna array size) than by algorithmic capacity. Future work will focus on expanding target diversity to include complex human motion (micro-Doppler effects) and improving performance in extremely dense urban environments where target overlap is frequent.
This research represents a significant step toward autonomous, "plug-and-play" 6G sensing infrastructure capable of learning and understanding its environment without manual supervision.
Summary updated