TL;DR
Researchers have introduced Fog-FWL, the first large-scale dataset for Full-Waveform LiDAR (FWL) designed to "see" through fog. By providing raw optical waveforms instead of processed point clouds, along with a physics-based simulator, this work enables the development of models that can recover objects hidden by dense fog—objects that standard LiDAR sensors would simply ignore.
What Problem Does This Paper Address?
Standard LiDAR is the "eyes" of most autonomous vehicles, but it has a major blind spot: fog. In foggy conditions, water droplets scatter laser light, creating a wall of "backscatter" noise near the sensor while simultaneously weakening the signal reflecting off actual objects (like cars or pedestrians).
The real bottleneck, however, is peak detection. Commercial LiDARs don't output raw data; they process the incoming light signal and only keep the strongest two or three "peaks" to generate a 3D point cloud. In dense fog, the fog's backscatter is often stronger than the reflection from a distant car. Consequently, the car's signal is discarded at the hardware level, making it impossible for downstream AI to "denoise" a point cloud that doesn't contain the object in the first place.
Full-Waveform LiDAR (FWL) records the entire raw intensity profile over time, retaining these weak, hidden returns. Until now, progress has been stalled by a lack of data: the only existing real-world foggy FWL dataset contained a measly 13 frames.
Methods
To bridge this gap, the authors developed a two-part dataset foundation:
- Fog-FWL-Real: A massive real-world benchmark captured using an in-house SPAD (Single-Photon Avalanche Diode) LiDAR. They used a fog machine to create localized, heterogeneous fog banks, capturing how light behaves in complex, non-uniform scattering environments.
- Fog-FWL-Simulator: Since labeling raw waveforms is incredibly labor-intensive, the team built a sophisticated physics-based simulator. Unlike previous simulators that just added uniform noise, this pipeline (shown in Figure 1) uses:
- CARLA for diverse 3D driving scenes.
- Time-resolved volumetric rendering (via Mitsuba 3) to simulate multiple scattering and light transport through heterogeneous fog plumes.
- SPAD Receiver Modeling to account for hardware-specific effects like "pile-up" (where early photons mask later ones) and "dead time."
Datasets
The Fog-FWL suite consists of:
- Fog-FWL-Real: 7,527 frames across 58 sequences. This is 579x larger than any previous real-world foggy FWL dataset. It includes 10,719 3D bounding boxes and histogram-level labels for Object, Fog, and Noise.
- Fog-FWL-Sim: 12,000 synthetic frames designed for training, allowing for a strict "sim-to-real" evaluation where models are trained on synthetic data and tested on the real-world benchmark.
As shown in the comparison table below, Fog-FWL is the first to provide the trifecta of raw waveforms, fog conditions, and 3D detection labels at scale.
Key Insights and Ideas
- The "Invisible" Car: The authors quantified exactly how much information is lost to peak detection. In "Hard" fog conditions, the actual object return is among the top three strongest peaks for only 34.9% of rays. If you only look at the single strongest peak (a common LiDAR setting), you only see the object 11.6% of the time.
- Heterogeneity Matters: Real fog isn't a uniform "soup"; it exists in patches and banks. The authors' use of a plume model for both real and simulated data is critical because it creates ray-dependent attenuation, which is much harder for sensors to handle than uniform fog.
- Waveform-Level Supervision: By using a "fog-free" reference waveform for every scene, the authors can label exactly which time-bins in a foggy waveform contain an object return, even if that return is buried deep under noise.
Key Results
The study provides a sobering look at the limitations of current technology while offering a path forward. The primary result is the dataset itself, categorized by three difficulty levels (Easy, Moderate, Hard) based on signal attenuation.
The data in Figure 4 underscores the necessity of this research: in Hard fog, nearly 65% of the time, the object is completely missing from the top three peaks. This proves that standard point-cloud-based perception is fundamentally limited in adverse weather.
Limitations and Future Work
While a massive step forward, the authors note a few areas for growth:
- Artificial Fog: The real-world dataset uses a fog machine, which may have different particle size distributions than natural meteorological fog.
- Sensor Specificity: The data is captured with a specific SPAD-based LiDAR. While the simulator helps generalize, different LiDAR architectures (like linear-mode APDs) might produce different waveform characteristics.
- Scope: The current dataset focuses on nearby vehicles (10–20m). Future iterations could expand to higher speeds, longer ranges, and more diverse objects like pedestrians or cyclists in natural fog settings.
Summary updated