DEFDVO: Deformable Deep Visual Odometry for Equirectangular Images

Burak Sevsay, Egor Bondarev, Erkut AkdagarXiv 2026 · arXiv

TL;DR

DEFDVO is a novel deep visual odometry (VO) framework designed specifically for $360^\circ$ equirectangular images. By baking spherical geometry directly into feature extraction, patch selection, and bundle adjustment, it achieves a massive $4.8\times$ improvement in trajectory accuracy over previous state-of-the-art methods.

What Problem Does This Paper Address?

Visual Odometry (VO)—the process of estimating a camera's motion by analyzing a sequence of images—is a cornerstone of robotics and augmented reality. While most VO systems are designed for standard "pinhole" cameras with a limited field of view, $360^\circ$ (omnidirectional) cameras offer a huge advantage: they see everything at once, meaning they rarely lose track of the environment even during rapid rotations.

However, $360^\circ$ images are usually stored in an Equirectangular Projection (ERP)—the same "unwrapped" format used for world maps. This format introduces three major headaches for standard computer vision algorithms:

  1. Latitude-dependent Distortion: Objects near the poles are stretched horizontally compared to those at the equator.
  2. The Artificial Seam: The left and right edges of the image are physically adjacent in the real world, but standard neural networks treat them as disconnected boundaries.
  3. Sampling Bias: Because of the projection, a pixel at the pole represents a much smaller physical area than a pixel at the equator. Standard uniform sampling over-represents the poles, wasting computation on distorted, less-informative regions.

Methods

DEFDVO builds upon the DPVO (Deep Patch Visual Odometry) architecture but re-engineers it from the ground up to be "sphere-aware." Its core innovations include:

  • Spherically Initialized Deformable Features: Instead of using standard square convolutions, the authors use Modulated Deformable Convolutions (DCNv2). Crucially, they initialize these convolutions with a "SphereNet" prior—a mathematical calculation of how a kernel should stretch to match ERP distortion. The network then learns small "residuals" to fine-tune these offsets based on the actual image content.
  • End-to-End Seam Awareness: To treat the image as a continuous cylinder, the authors implemented circular padding in the encoder, circular indexing for correlation lookups, and "wrapped" residuals in the bundle adjustment step. This ensures that if a feature moves off the right edge of the frame, the system correctly tracks it reappearing on the left.
  • Spherical-Gradient Patch Selection: Rather than picking points to track randomly on the 2D grid, DEFDVO samples patches based on their solid angle (the actual area they occupy on a sphere). It combines this with image gradient information to ensure it tracks high-contrast features while maintaining a uniform distribution across the entire viewing sphere.

Datasets

The model was trained exclusively on synthetic data and then tested for its ability to generalize to the real world:

  • TartanAirV2: A large-scale synthetic dataset of ERP imagery with ground-truth camera poses and depth.
  • 360DVO Dataset: A real-world benchmark containing 20 sequences of $360^\circ$ video, divided into "Easy" and "Hard" categories based on motion complexity.

Key Insights and Ideas

  • Geometry + Learning is the Winning Combo: The authors found that neither pure geometric fixes (like SphereNet) nor pure data-driven learning (like standard DCNv2) performed as well as combining them. Starting with a geometric prior and letting the network learn the rest provided the best of both worlds.
  • Consistency is Key: Many previous methods only fixed one part of the pipeline (e.g., just the features). DEFDVO shows that for high-precision tracking, the entire pipeline—from the encoder to the optimization backend—must respect the spherical topology.
  • Synthetic-to-Real Generalization: Despite never seeing a real-world image during training, DEFDVO outperformed models specifically tuned for real-world data. This suggests that correctly modeling the geometry of the sensor is more important for VO than having perfectly realistic textures in the training set.

Key Results

  • Massive Accuracy Gains: On the TartanAirV2 benchmark, DEFDVO reduced the Average Absolute Trajectory Error (ATE) from 0.201m to 0.042m—a $4.8\times$ improvement over the previous best method (360DVO).
  • Real-World Dominance: It achieved the lowest average ATE on both the Easy and Hard splits of the real-world 360DVO benchmark, proving its robustness to complex, non-synthetic environments.
  • Efficiency: The system runs at approximately 7.1 FPS on an NVIDIA RTX A5000. While not quite "real-time" (usually defined as 30 FPS), it is fast enough for many robotic applications.

Limitations and Future Work

The primary bottleneck is the deformable encoder, which accounts for over 80% of the per-frame runtime at full resolution. The authors suggest that exploring more efficient deformable operators could significantly boost speed.

Additionally, DEFDVO is currently a "Visual Odometry" system, meaning it tracks local motion. It does not yet include "Loop Closure"—the ability to recognize a previously visited location and correct long-term drift. The authors point toward integrating DEFDVO into a full 360° SLAM (Simultaneous Localization and Mapping) system as the next logical step.

Summary updated

Abstract

Omnidirectional ($360^\circ$) cameras capture the full surrounding scene in every frame, providing a strong advantage for visual odometry (VO) over pinhole sensors with a limited field of view. However, their equirectangular projection (ERP) introduces latitude-dependent distortion, an artificial seam at the image border, and a sampling bias toward the poles. Existing $360^\circ$ VO methods address these challenges only partially. DEFDVO closes this gap with a patch-based deep VO method that operates directly on ERP imagery, combining spherically initialized deformable feature extraction, end-to-end seam-aware estimation, and spherical-gradient patch selection. The deformable feature extractor combines spherical geometric priors with learned sampling offsets, while seam-aware processing preserves ERP continuity, and spherical-gradient selection reduces polar sampling bias. Trained exclusively on synthetic TartanAirV2 ERP imagery, DEFDVO improves the best prior average absolute trajectory error (ATE) by $4.8\times$ (0.201 m to 0.042 m) on the TartanAirV2 evaluation set defined by 360DVO and achieves the lowest average ATE on both splits of the real-world 360DVO benchmark. Code is available at https://github.com/BurakSevsay/DEFDVO
BibTeX
@misc{sevsay2026defdvo,
  author = {Burak Sevsay and Egor Bondarev and Erkut Akdag},
  title = {{DEFDVO: Deformable Deep Visual Odometry for Equirectangular Images}},
  year = {2026},
  eprint = {2611.19095},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.19095},
}
Figure