DarSwin-Unet: Distortion Aware Encoder-Decoder Architecture

Akshaya Athwale, Ichrak Shili, Émile Bergeron, Ola Ahmad, Jean-François Lalonde

Introduction

Wide-angle and fisheye lenses are indispensable in robotics and autonomous driving due to their expansive field of view. However, these lenses introduce significant geometric distortions that violate the translational equivariance assumption inherent in standard Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). While conventional methods often "undistort" images into perspective views, this process frequently results in resolution loss and severe stretching artifacts.

DarSwin-Unet addresses these challenges by introducing a novel encoder-decoder transformer architecture that incorporates the physical distortion profile of the lens directly into its structure. This approach enables "zero-shot" generalization, allowing the model to accurately process images from lenses with distortion characteristics unseen during training.

Architecture and Methodology

The core of the proposed system is the DarSwin-Unet, an extension of the distortion-aware radial Swin Transformer (DarSwin) into a full U-Net framework for pixel-level tasks such as depth estimation.

The DarSwin-Unet architecture, featuring radial patch partitioning and azimuth patch expanding layers for distortion-aware depth estimation.

Key architectural components include:

  1. Radial Patching: Instead of a standard Cartesian grid, the image is divided into radial patches based on the lens's specific distortion profile. This allows the network's internal structure to align with the physical properties of the lens.
  2. Azimuth Patch Expanding: To facilitate the decoder's upsampling process while preserving radial geometry, the authors designed a specialized Azimuth Patch Expanding layer.
  3. k-NN Mapping: Because internal processing occurs in polar-radial space, a final k-Nearest Neighbors (k-NN) layer maps features back to a standard Cartesian grid for the final output.

Optimized Sampling Strategy

A critical challenge in extending radial transformers to pixel-level tasks is ensuring uniform pixel coverage. Previous sampling methods often left "holes" or sparse regions in the image, particularly under varying distortion levels. The authors introduced an optimized mathematical sampling function, $g(\theta)$, designed to spread samples as uniformly as possible across the image.

Comparison of sampling strategies across different distortion levels (xi). The proposed g(theta) function ensures more uniform coverage.

This optimization is vital for maintaining high performance in dense prediction tasks like depth estimation, where sparsity significantly degrades results. By warping the model's receptive field rather than the image itself, the architecture achieves unprecedented generalization across diverse lens types.

Experimental Evaluation

The model was evaluated using a synthetic benchmark derived from the Matterport3D dataset. High-resolution panoramas were used to generate 175° field-of-view crops with varying distortion levels simulated via the Unified Camera Model (UCM).

Key Results

  • Zero-Shot Robustness: DarSwin-Unet demonstrates remarkable stability when tested on distortion levels different from those used in training. While standard baselines like Swin-Unet experience sharp performance drops, DarSwin-Unet's accuracy remains consistent.
  • Superior Accuracy: The model consistently outperforms state-of-the-art baselines across the entire spectrum of distortion levels ($\xi \in [0, 1]$) on standard depth metrics.
  • High-Fidelity Reconstruction: The k-NN mapping layer achieves a Mean Absolute Error (MAE) of only 0.8% when projecting features back to the standard image grid.

Conclusion

DarSwin-Unet demonstrates that integrating lens geometry directly into the model architecture is more effective than pre-processing images to fit standard models. Future work will focus on reducing the reliance on pre-calibration by predicting distortion parameters end-to-end and improving computational efficiency for higher resolutions.

Summary updated

BibTeX
@inproceedings{athwale2025darswin,
  author = {Akshaya Athwale and Ichrak Shili and Émile Bergeron and Ola Ahmad and Jean-François Lalonde},
  title = {{DarSwin-Unet: Distortion Aware Encoder-Decoder Architecture}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
  doi = {10.1109/wacv61041.2025.00840},
}
Figure