SkyShield: Occupancy as a Safety Interface for Low-Altitude UAV Autonomy

Jie Gao, Jie Ma, Kaihui Lin, Kai Ye, Miaohui Zhang, Pingyang Dai, Liujuan CaoarXiv 2026 · arXiv

TL;DR

SkyShield introduces the first front-view monocular semantic occupancy benchmark specifically designed for low-altitude UAV flight (below 20 meters), where rapid 6-DoF attitude changes and thin obstacles make traditional vision models fail. Along with a 36K-sample dataset, the paper proposes a new kinematics-aware evaluation metric (KAR-mIoU) and a geometry-first baseline model (SkyOcc) that explicitly integrates drone attitude to prevent feature misalignment.

What Problem Does This Paper Address?

When an autonomous drone flies below 20 meters in an urban environment, the stakes are incredibly high. It must navigate a chaotic airspace filled with tree branches, power lines, lampposts, and buildings. To do this safely, the drone needs a dense, 3D understanding of what space is occupied and what space is free.

While 3D semantic occupancy mapping has taken off in autonomous driving, transferring these models to Unmanned Aerial Vehicles (UAVs) isn't straightforward. Autonomous cars enjoy stable, near-planar motion on flat roads. Drones, by contrast, experience severe kinematic volatility—their cameras constantly pitch, roll, and shift in altitude. Traditional perception models suffer from "perspective collapse" or misalignment because they assume a stable sensor rig. Furthermore, standard evaluation metrics like mean Intersection-over-Union (mIoU) treat all voxels equally, meaning a missed power line right in front of the drone is penalized the same as a misclassified building voxel a hundred meters away.

Figure 1: Motivation. Low-altitude UAV scenarios expose the limitations of traditional UAV perception and driving- oriented occupancy models.

Methods

The authors tackle these challenges with a three-part contribution:

  1. KAR-mIoU (Kinematics-Aware Risk mIoU): A new, safety-centric metric that re-weights voxel errors based on the drone's current velocity and trajectory. It uses an Omnidirectional Worst-Case Time-to-Collision (OWC-TTC) formulation to heavily penalize false negatives and false positives within the drone's immediate flight corridor, while leaving distant or unreachable errors unweighted.
  2. Attitude-Aware Spatial Projection: Instead of treating the drone's rapid pitch and roll changes as auxiliary noise, the baseline model, SkyOcc, builds frame-wise 6-DoF camera extrinsics directly into the 2D-to-3D projection chain. This ensures visual features align perfectly with the 3D voxel space regardless of how much the drone tilts.
  3. Safety-Prior Voxel Optimization: To prevent rare but critical hazards (like thin wires or poles) from being ignored by the loss function, the authors utilize static safety-prior class weights combined with a Lovász-Softmax loss to optimize for sparse, long-tail structures.
The SkyOcc pipeline utilizing attitude-compensated projection and spatiotemporal feature fusion.

Datasets

The authors introduce SkyShield, a high-fidelity dataset built using the CARLA simulator.

  • Scale: 36K front-view UAV samples collected at 25Hz across 8 urban scenes and 10 weather conditions.
  • Annotations: Each sample pairs a monocular RGB image with frame-wise 6-DoF UAV poses, dynamic camera geometry, and dense front-frustum semantic occupancy labels.
  • Ground Truth Generation: To build reliable labels without cheating during inference, the simulation environment utilizes dual, blind-spot-free LiDARs to capture dense geometry, which is then voxelized and filtered to match the front camera's frustum.
The coordinate-consistent pipeline used to construct the SkyShield dataset.

Key Insights and Ideas

  • The Voxel-Safety Imbalance: A core finding is the massive discrepancy between 2D image richness and 3D voxel volume. In the SkyShield dataset, free space accounts for 82.78% of all voxels, while safety-critical "thin structures" (poles, wires) occupy a mere 0.26%. Models optimized purely for global voxel accuracy will easily ignore these thin structures, creating a silent but deadly safety hazard.
  • Attitude is a Geometric Cue, Not Noise: In high-roll or high-pitch maneuvers, the visual horizon tilts violently. Explicitly embedding the 6-DoF pose into the projection matrix stabilizes the temporal feature queue, allowing the network to cleanly fuse historical observations without "ghosting" artifacts.

Key Results

The proposed SkyOcc baseline was evaluated against a modified BEVFormer backbone on the SkyShield test set:

  • Overall Performance: SkyOcc improved the standard mIoU from 25.08% to 28.91% and boosted the safety-focused KAR-mIoU from 15.04% to 19.10%.
  • Critical Categories: The biggest gains were seen in sparse, safety-critical classes. Performance on thin structures jumped from 5.57% to 12.80% (+7.23), and vulnerable road users (VRUs) climbed from 2.95% to 12.09% (+9.14).
  • Ablation Insights: Incorporating attitude-aware projection provided the single largest immediate boost to geometry-sensitive foreground classes, proving that explicit geometric stabilization is vital for aerial perception.

Limitations and Future Work

The primary limitation is that SkyShield is entirely simulation-based. While CARLA provides excellent environmental diversity and perfect ground truth, it cannot fully replicate real-world physical phenomena like motion blur, rolling-shutter distortion, lens contamination, structural vibrations, or rotor-induced aerodynamic airflow (which can shake the camera rig).

The authors note that bridging this sim-to-real gap is the logical next step, alongside exploring uncertainty-aware predictions and closed-loop perception-planning integration.

Summary updated

Abstract

For low-altitude Unmanned Aerial Vehicle (UAV) autonomy, 3D spatial understanding is not merely a perception objective, but the safety interface between human instructions and physical flight. In human-scale urban airspace below 20 meters, thin geometry, occlusions, vegetation, and urban clutter define whether an aerial agent can safely enter the space ahead. However, existing UAV datasets mainly provide 2D annotations or 3D boxes, while driving-oriented occupancy benchmarks assume stable ground-level sensor rigs. Both miss the defining regime of low-altitude flight: a front-facing monocular camera observing occupied and free space from a moving aerial body with frame-wise changing 6-DoF pose and camera extrinsics. To bridge this gap, we introduce SkyShield, to the best of our knowledge the first front-view monocular semantic occupancy benchmark for urban UAV flight below 20 meters. Built on CARLA, SkyShield contains 36K front-view UAV samples across diverse urban scenes and weather conditions, pairing each image with frame-wise 6-DoF UAV pose, frame-wise dynamic camera geometry, UAV states, and front-frustum semantic occupancy labels. We further propose KAR-mIoU, a UAV-centric and dynamics-aware metric that re-weights voxel-level evaluation by kinematic reachability and time-to-collision, revealing safety-critical risks hidden by conventional mIoU. To tackle this challenging new setting, we provide SkyOcc, a geometry-first monocular baseline that integrates frame-wise UAV attitude into projection, fuses temporal occupancy features, and applies safety-prior optimization to preserve sparse collision-critical structures. Together, SkyShield, KAR-mIoU, and SkyOcc establish occupancy as a safety interface for low-altitude aerial autonomy. Code and dataset will be released publicly.
BibTeX
@misc{gao2026skyshield,
  author = {Jie Gao and Jie Ma and Kaihui Lin and Kai Ye and Miaohui Zhang and Pingyang Dai and Liujuan Cao},
  title = {{SkyShield: Occupancy as a Safety Interface for Low-Altitude UAV Autonomy}},
  year = {2026},
  eprint = {2606.00747},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.00747},
}
Figure