Reflection-Robust Stereo Visual-Inertial Odometry via Kalman-Filtered Ground-Height Estimation

Uihyun Hwang, Sungjae Shin, Dongjae Kim, Hyun MyungarXiv 2026 · arXiv

TL;DR

This paper introduces a reflection-robust front-end for stereo visual-inertial odometry (VIO) that tackles the problem of "ghost" features appearing beneath reflective indoor floors. By combining real-time semantic floor segmentation with a Kalman-filtered ground height estimate, the system selectively rejects virtual features while preserving valid on-floor texture, significantly improving localization accuracy in challenging indoor environments.

What Problem Does This Paper Address?

Visual-Inertial Odometry (VIO) is a cornerstone of robot navigation, relying on tracking 3D points across camera frames. However, modern indoor environments are full of "optical traps": polished tiles, waxed concrete, and glass. These surfaces act as mirrors, creating high-contrast reflections of ceiling lights or windows.

Standard feature extractors treat these reflections as real objects. When a stereo camera triangulates these points, they appear as "virtual features" located below the physical floor. Because these reflections move relative to the floor as the robot moves, they violate the multi-view consistency required for stable VIO, leading to significant trajectory drift or even total filter divergence.

Existing solutions are often too blunt—like "naive floor rejection," which discards all features in the floor region—or too computationally heavy, like dense RANSAC-based plane fitting. The authors aim for a "Goldilocks" solution: a lightweight method that keeps the good features (tile edges, floor texture) and tosses the bad ones (reflections).

Methods

The proposed pipeline, illustrated in Figure 2, adds a geometry-aware filtering stage to the VIO front-end:

  1. Floor Segmentation: A real-time neural network (GCNet) generates a 2D mask of the floor region in the image.
  2. Mask-Aware Partitioning: Features are split into "background" (safe) and "floor-region" (potentially reflective) sets.
  3. Kalman-Filtered Height Estimation: Instead of fitting a complex 3D plane, the system maintains a scalar Kalman filter to estimate the ground height ($h_k$) along the gravity vector.
  4. Measurement: It takes a "support set" of floor features and calculates a weighted average of their heights.
  5. Uncertainty Modeling: Crucially, the filter accounts for stereo triangulation noise, translation error, and rotation error to weight these measurements.
  6. NIS Gating: A Normalized Innovation Squared (NIS) check acts as a gatekeeper, preventing "ghost" features from pulling the ground estimate downward.
  7. Selective Rejection: Each floor feature is tested against the estimated ground height. If a feature is significantly below the floor (beyond an uncertainty-aware safety margin), it is rejected.
  8. Recovery Mechanism: If the robot encounters a real change in floor height (like a step), the filter detects a sequence of consistent NIS-rejected measurements and resets to the new height.
Overview of the proposed reflection-robust VIO front-end pipeline.

Datasets

Because no public stereo-inertial datasets specifically targeted reflective indoor floors, the authors collected their own:

  • KAIST & Yonsei Sequences: 8 real-world sequences collected in university buildings (E4, E11, N3, N7, and Yonsei).
  • Hardware: An Intel RealSense D455 stereo camera, an Xsens IMU, and a Livox LiDAR (used only to generate ground-truth trajectories and floor heights).
  • Training: The GCNet segmentation model was trained on a mix of their own collected data and the Open-LORIS robotic vision dataset.

Key Insights and Ideas

  • Selective Rejection is Superior: As shown in Figure 1, naive rejection throws away 100% of floor features. The proposed method rejected ~79% of features in a test case, successfully keeping the valid on-floor textures that help stabilize the odometry.
  • Gravity is the Anchor: By projecting features onto the gravity vector (estimated during VIO initialization), the problem simplifies from 3D plane fitting to 1D height estimation, making it extremely efficient.
  • Uncertainty-Aware Margins: The rejection threshold isn't a fixed number of centimeters. It expands when the robot's pose estimate is shaky or when features are far away, preventing the accidental rejection of valid features due to noise.
Fig. 1. Comparison of feature extraction results on a re- flective floor in the Yonsei environment. Of the 71 fea- tures extracted within the floor-mask region, ours rejects 56 (78.9%) below-ground virtual candidates while pre- serving valid on-floor features.

Key Results

The authors integrated their front-end with two popular VIO frameworks: VINS-Fusion and DynaVINS.

  • Trajectory Accuracy: The method improved Absolute Trajectory Error (ATE) in 6 out of 8 sequences for VINS-Fusion. For DynaVINS, it restored convergence in four sequences where the baseline system completely failed.
  • Ground Height Stability: The mean error for ground height estimation remained at or below 0.05 meters across most sequences.
  • Real-Time Performance: The geometric filtering adds only 4.93 ms per update. The overall processing rate for VINS-Fusion dropped negligibly from 15.0 Hz to 14.7 Hz, proving it is suitable for low-power mobile robots.
Fig. 4. VINS-Fusion trajectories on the E4 and N3 se- quences (original, naive floor rejection, and ours vs. the FAST-LIO2 reference).

Limitations and Future Work

  • Dark Environments: In the E11 sequence (dark corridors), the floor mask was less reliable, and sparse features led to slightly higher errors. The system relies on the quality of the semantic segmentation.
  • Mirror-Like Constraints: The authors noted that in extremely specular environments, mirror-like reflections can sometimes provide useful geometric constraints if handled correctly. Future work will explore how to selectively use these virtual features rather than just rejecting them.
  • Textureless Floors: The method still requires some valid floor texture or edges to estimate the height; perfectly uniform, clean floors remain a challenge for any feature-based VIO.

Summary updated

Abstract

Visual-inertial odometry relies on visual features that correspond to 3D points with consistent projections across sequential frames. In reflective indoor-floor environments, however, specular floor patterns can be detected and triangulated as virtual features below the physical ground plane, producing unreliable feature associations and inconsistent visual residuals. This paper proposes a lightweight reflection-aware front-end for feature-based stereo VIO. Rather than suppressing all floor-region features, the proposed method estimates the ground height in real time and rejects only feature candidates whose inferred 3D positions are below the estimated ground height. A floor-region mask restricts the validation process to reflection-prone regions, while stereo height uncertainty and pose uncertainty are used to construct frame-level ground-height measurements. A scalar Kalman filter with Normalized Innovation Squared (NIS) gating then provides a stable ground-height estimate, and an uncertainty-aware safety margin is used for selective below-ground virtual feature rejection. Experiments on real reflective-floor indoor sequences show that the proposed front-end generally improves trajectory accuracy in reflection-dominant environments while maintaining accurate ground-height estimates and recovering from actual ground-height changes.
BibTeX
@misc{hwang2026reflection,
  author = {Uihyun Hwang and Sungjae Shin and Dongjae Kim and Hyun Myung},
  title = {{Reflection-Robust Stereo Visual-Inertial Odometry via Kalman-Filtered Ground-Height Estimation}},
  year = {2026},
  eprint = {2611.05938},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.05938},
}
Figure