NIF-VIO: Visual--Inertial Odometry Across Inertial and Non-Inertial Reference Frames

Zichao Shu, Lijun Li, Zhuochen Li, Zetao Chen, Jianyu WangarXiv 2026 · arXiv

TL;DR

NIF-VIO is a novel visual-inertial odometry framework that maintains accurate localization inside moving "carriers" like cars, subways, and elevators. By adaptively switching between full and selective sensor fusion based on real-time consistency checks, it overcomes the classic VIO failure mode where carrier acceleration conflicts with visual observations of a stationary interior.

What Problem Does This Paper Address?

Conventional Visual-Inertial Odometry (VIO) relies on a fundamental assumption: the camera and the Inertial Measurement Unit (IMU) are both operating in a stationary, inertial reference frame. This works great for a robot walking down a hallway, but it breaks the moment you step into a moving vehicle.

Inside an accelerating car or a rising elevator, a device’s camera sees a stationary interior (the dashboard or the elevator walls), suggesting zero motion. Simultaneously, the IMU feels the carrier's acceleration and rotation. This "non-inertial" conflict leads to massive estimation errors or total system failure in standard VIO. While you could switch to Visual Odometry (VO) alone, you’d lose the robustness and high-frequency tracking benefits that an IMU provides. NIF-VIO aims to bridge this gap using only device-side sensors, without needing any data from the vehicle itself.

Methods

The authors introduce a multi-stage approach to handle the transition between inertial and non-inertial states:

  1. Consistency-Guided Fusion Management: The system constantly compares image-based motion estimates with IMU preintegration. Using a statistical test based on Student-t distributions, it detects inconsistencies.
  2. Transition Handling: To prevent "poisoning" the local map during the time it takes to confirm a state change, NIF-VIO uses early constraint isolation. It temporarily ignores IMU factors at the first sign of trouble. Once the non-inertial state is confirmed, it performs a local reoptimization to repair any geometric distortions caused by the initial lag in detection.
  3. Selective Inertial Constraints: Instead of discarding the IMU entirely during carrier motion, NIF-VIO uses a compact carrier-motion model. It assumes that vehicle motion is locally regular (e.g., rotating around a dominant axis or accelerating linearly). By modeling these carrier-induced effects as auxiliary variables, the system can still extract useful device-relative motion data from the IMU to complement the vision system.
  4. Recovery: When the carrier stops, the system fits IMU data to visual poses to re-estimate gravity and velocity, eventually restoring full VIO fusion.
Fig. 2. System overview of NIF-VIO. Building on a tracking and local mapping architecture, the framework uses visual–inertial consistency to guide IMU fusion. Transition management supports reliable estimation as inertial conditions change, while selective inertial constraints preserve useful IMU information for relative-motion estimation under non-inertial conditions.

Datasets

The researchers evaluated NIF-VIO on:

  • Standard Benchmarks: EuRoC and TUM-VI were used to ensure the system performs as well as state-of-the-art VIO in traditional stationary environments.
  • New NIF Dataset: The authors collected a dedicated dataset using an Intel RealSense T265 across car, subway, and elevator scenarios. To get ground truth inside these moving platforms, they used ArUco markers and offline global bundle adjustment. This dataset is particularly valuable as it captures the specific transitions between stationary and moving carrier phases.

Key Insights and Ideas

  • Proactive Isolation is Better than Reactive Correction: Because evidence of inconsistency takes time to accumulate, the "early warning" system is critical. By isolating the IMU before confirmation, the system avoids the "drift" that usually occurs the moment a car starts moving.
  • The IMU is Still Useful in a Car: Even when a car is turning, the IMU provides high-frequency information about how the handheld device is moving relative to the car. By modeling the car's rotation as a "background" effect, NIF-VIO retains this precision.
  • Unified Factor Graph: The transition isn't just a binary switch; it's an evolution of the underlying mathematical graph. As shown in Figure 3, the system shifts which variables are "fixed" and which are "optimized" in real-time.
Fig. 3. Evolution of the local factor graph through a representative sequence of fusion stages. Each configuration shows the active constraints and optimized variables, illustrating how visual–inertial fusion adapts to changing inertial conditions.

Key Results

NIF-VIO demonstrates significant improvements in "Correct Localization Rate" (CR)—the percentage of time the system stays within a tight error bound (0.2m / 5°).

  • Non-Inertial Robustness: In car and subway tests, standard VIO systems (like ORB-SLAM3 or VINS-Fusion) often saw their CR drop to between 20% and 40%. NIF-VIO maintained a CR above 98% across all tested scenarios.
  • Accuracy: In the car scenario, NIF-VIO achieved a conditional Absolute Trajectory Error (C-ATE) of just 1.2 cm, significantly outperforming visual-only modes which lacked the IMU's high-frequency support.
  • Efficiency: Despite the added complexity of the carrier-motion model, the system runs in real-time. The tracking thread averages around 20.5 ms per frame, well within the limits for 30Hz camera input.
Comparison of trajectory segments showing NIF-VIO following the reference much more closely than standard VIO during carrier motion onset.

Limitations and Future Work

The current version of NIF-VIO assumes the camera's view is dominated by the carrier interior. If a user points the camera out the window of a moving car, the visual features would reflect world-relative motion while the IMU reflects carrier-relative motion, potentially confusing the current model.

The authors suggest that future work will involve integrating "external" visual observations to better constrain carrier motion and further strengthen the fusion for more complex, mixed-view environments. Additionally, the system currently requires an initial stationary period to calibrate; making this initialization more robust to immediate motion would be a logical next step.

Summary updated

Abstract

Conventional visual--inertial odometry (VIO) assumes that observations from cameras and an inertial measurement unit (IMU) are jointly consistent in an inertial reference frame. However, this assumption breaks down under non-inertial conditions. Inside accelerating or rotating carriers, visual observations of the interior reflect carrier-relative motion, while IMU measurements contain additional carrier-induced components, which can compromise conventional fusion. We therefore present NIF-VIO for localization relative to the local environment across inertial and non-inertial reference frames. It adapts visual--inertial fusion based on measurement consistency. As inertial conditions change, early constraint isolation is combined with local reoptimization to limit error propagation before inconsistency confirmation and repair the affected local geometry. Under non-inertial conditions, a compact model with temporally shared parameters is introduced to account for carrier motion while preserving inertial information that complements visual constraints. Experiments on public benchmarks and a self-collected dataset demonstrate that NIF-VIO achieves competitive performance under inertial conditions and improved localization accuracy and availability under non-inertial conditions.
BibTeX
@misc{shu2026nif,
  author = {Zichao Shu and Lijun Li and Zhuochen Li and Zetao Chen and Jianyu Wang},
  title = {{NIF-VIO: Visual--Inertial Odometry Across Inertial and Non-Inertial Reference Frames}},
  year = {2026},
  eprint = {2611.13662},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.13662},
}
Figure