DaFIR: Distortion-Aware Representation Learning for Fisheye Image Rectification

Zhaokang Liao, Wengang Zhou, Houqiang Li

Abstract

This paper focuses on fisheye image rectification. Existing learning-based solutions learn image representations that mix distortion features and content features. Since the distortion feature dominates the rectification process, we propose a novel distortion-aware representation learning framework, which decouples the distortion feature from the content feature, for fisheye image rectification. Specifically, we first pre-train a Vision Transformer with a supervised pre-text task, which regresses the distortion distribution map of a distorted image. The pre-training equips the Vision Transformer with the ability to capture distortion-related patterns. After that, the pre-trained model is fine-tuned to predict the pixel-wise flow map to rectify the fisheye images. Extensive experiments are conducted to evaluate our approach and verify our idea of feature decoupling. The experiment results demonstrate the state-of-the-art performance of our approach compared to existing algorithms, as well as its generality on real-world images. Our source code is publicly available at https://github.com/lzk9508/DaFIR.
BibTeX
@article{liao2024dafir,
  author = {Zhaokang Liao and Wengang Zhou and Houqiang Li},
  title = {{DaFIR: Distortion-Aware Representation Learning for Fisheye Image Rectification}},
  journal = {IEEE Trans. Circuits Syst. Video Technol.},
  year = {2024},
  url = {https://openreview.net/forum?id=w3X6wcpMk5},
}

Loading related content...

Figure