AURORA-KITTI: Any-Weather Depth Completion and Denoising in the Wild

Yiting Wang, Tim Brödermann, Hamed Haghighi, Haonan Zhao, Christos Sakaridis, Kurt Debattista, Valentina Donzella

Abstract

Robust depth completion is fundamental to real-world 3D scene understanding, yet existing RGB-LiDAR fusion methods degrade significantly under adverse weather, where both camera images and LiDAR measurements suffer from weather-induced corruption. In this paper, we introduce AURORA-KITTI, the first large-scale multi-modal, multi-weather benchmark for robust depth completion in the wild. We further formulate Depth Completion and Denoising (DCD) as a unified task that jointly reconstructs a dense depth map from corrupted sparse inputs while suppressing weather-induced noise. AURORA-KITTI contains over 82K weather-consistent RGBL pairs with metric depth ground truth, spanning diverse weather types, three severity levels, day and night scenes, paired clean references, lens occlusion conditions, and textual descriptions. Moreover, we introduce DDCD, an efficient distillation-based baseline that leverages depth foundation models to inject clean structural priors into in-the-wild DCD training. DDCD achieves state-of-the-art performance on AURORA-KITTI and the real-world DENSE dataset while maintaining efficiency.

Introduction

Depth completion is the process of generating a dense, high-resolution 3D map from sparse LiDAR point clouds and RGB images. While current models excel in clear conditions, they often fail in "the wild" due to adverse weather. Sensors face a dual challenge: camera images lose contrast and structural cues due to rain or fog, while LiDAR sensors suffer from "backscatter" (false returns from water particles) and signal attenuation.

Most research treats depth completion as a simple hole-filling task. This paper argues that real-world applications require Depth Completion and Denoising (DCD)—a unified task that reconstructs geometry while simultaneously filtering out weather-induced artifacts and "ghost" points.

The AURORA-KITTI Benchmark

AURORA-KITTI addresses the lack of large-scale, weather-diverse datasets for depth completion. It contains 82,177 weather-consistent RGB-LiDAR pairs, covering eight compound conditions (Day/Night combined with Clear/Fog/Rain/Snow) across three levels of severity. It also includes specific "lens occlusions" like raindrops or snowflakes stuck to the camera lens.

AURORA-KITTI dataset samples showing diverse weather conditions for both RGB images and LiDAR point clouds across fog, rain, and snow scenarios.

To ensure realism, the authors utilized physics-consistent simulations. Fog is modeled using Koschmieder’s law to accurately reflect visibility, and LiDAR simulations account for beam divergence and particle interaction. This cross-modal consistency ensures that if an image appears foggy, the corresponding LiDAR data correctly reflects the associated signal loss and noise.

DDCD: Distillation-based Depth Completion and Denoising

To solve the DCD task, the authors propose DDCD, a student-teacher framework designed to inject clean structural knowledge into models processing noisy data:

  • The Teacher: A frozen "Depth Foundation Model" (such as Depth Anything V2). During training, the teacher processes a clean version of the scene to provide a "structural prior," establishing what the environment should look like without weather interference.
  • The Student: The active model being trained, which processes the corrupted RGB and LiDAR inputs.
  • Loss Functions: The framework employs a Scale-and-Shift Invariant (SSI) loss to align the student's predictions with the teacher's geometry, alongside Residual Gradient Regularization to maintain sharp object boundaries despite blurry or noisy inputs.
The DDCD architecture: A frozen Depth Foundation Network provides a dense structural prior (Pseudo Label) from clean RGB to guide the Depth Completion Network during training on noisy inputs.

Experimental Results

DDCD achieves state-of-the-art performance on the AURORA-KITTI benchmark, reducing the Root Mean Squared Error (RMSE) by 35.1% compared to previous zero-shot methods like Marigold-DC. Crucially, when tested on the real-world DENSE dataset, DDCD outperformed competitors, proving that synthetic training on AURORA-KITTI translates effectively to real-world physics.

Beyond accuracy, the model is highly efficient, running at 48.6 ms per frame on modern hardware. This makes it a viable candidate for real-time integration into autonomous vehicle perception pipelines.

Key Insights and Future Work

A significant finding of this research is that data quality trumps architecture. Retraining existing architectures on weather-aware, physically consistent data provided a larger robustness boost than designing complex new architectures trained only on clean data. By using LiDAR to anchor the metric scale and foundation models to denoise the shapes, DDCD effectively bridges the gap between sensor reliability and environmental noise.

Future work may involve utilizing the textual descriptions included in AURORA-KITTI. Integrating Large Language Models (LLMs) could allow depth models to "understand" the context of the weather (e.g., "heavy snow at night") to further refine denoising strategies.

Summary updated

Abstract

Robust depth completion is fundamental to real-world 3D scene understanding, yet existing RGB-LiDAR fusion methods degrade significantly under adverse weather, where both camera images and LiDAR measurements suffer from weather-induced corruption. In this paper, we introduce AURORA-KITTI, the first large-scale multi-modal, multi-weather benchmark for robust depth completion in the wild. We further formulate Depth Completion and Denoising (DCD) as a unified task that jointly reconstructs a dense depth map from corrupted sparse inputs while suppressing weather-induced noise. AURORA-KITTI contains over \textit{82K} weather-consistent RGBL pairs with metric depth ground truth, spanning diverse weather types, three severity levels, day and night scenes, paired clean references, lens occlusion conditions, and textual descriptions. Moreover, we introduce DDCD, an efficient distillation-based baseline that leverages depth foundation models to inject clean structural priors into in-the-wild DCD training. DDCD achieves state-of-the-art performance on AURORA-KITTI and the real-world DENSE dataset while maintaining efficiency. Notably, our results further show that weather-aware, physically consistent data contributes more to robustness than architectural modifications alone. Data and code will be released upon publication.
BibTeX
@misc{wang2026aurora,
  author = {Yiting Wang and Tim Brödermann and Hamed Haghighi and Haonan Zhao and Christos Sakaridis and Kurt Debattista and Valentina Donzella},
  title = {{AURORA-KITTI: Any-Weather Depth Completion and Denoising in the Wild}},
  year = {2026},
  eprint = {2603.14701},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.14701},
}
Figure