Filtering Based Adaptive Visual Odometry Sensor Framework Robust to Blurred Images

Haiying Zhao, Yong Liu, Xiaojia Xie, Yiyi Liao, Xixi Liu

TL;DR

This paper introduces a novel visual odometry (VO) framework that's robust to blurred images, a common problem in real-world robotics. By adaptively identifying and handling blurred frames using a new "small image gradient distribution" (SIGD) metric and an intelligent key-frame selection strategy, the system significantly improves odometry accuracy without a substantial computational overhead.

What Problem Does This Paper Address?

Visual odometry (VO) systems estimate a robot's pose (position and orientation) by analyzing sequences of camera images. A critical assumption for most VO algorithms is that the input images are clear. However, in practical robotic applications, especially those involving movement in dynamic or challenging environments, images can become blurred due to camera shake, motion, or poor lighting. This blurring severely degrades the accuracy of feature detection and matching, leading to significant errors or complete failure in pose estimation. While simply discarding blurred frames might work for occasional blurs, it's not a viable solution when blurred images are frequent, as it can lead to a loss of tracking. Existing methods either struggle with the computational cost of deblurring or are limited in their applicability.

Methods

The proposed framework tackles the blurred image problem in a three-pronged approach:

  1. Small Image Gradient Distribution (SIGD): This is a novel objective measure to quantify the degree of blur in an image. Unlike traditional gradient methods that only consider pixel differences in two directions, SIGD calculates gradients in eight directions within a 3x3 window and uses the maximum absolute difference. It focuses on the Y channel of the YUV color space to emphasize brightness. The "blurred degree" is then calculated based on the number of pixels with gradients below a certain threshold, normalized to a [0, 10] range. This method is computationally efficient.
  2. Adaptive Blurred Image Classification: To handle varying environmental conditions, a constant blur threshold is insufficient. This paper proposes an adaptive classification algorithm that dynamically adjusts the blur threshold. It combines aspects of inertial and Finite Impulse Response (FIR) filters to create a threshold that can follow gradual changes in image quality while still reacting to sudden blurs. This algorithm classifies frames into two categories: clear (C1) and blurred (C2).
  3. Anti-Blurred Key-Frame Selection: Instead of processing every frame, the system selects "key-frames" to feed into the VO pipeline. This anti-blurred key-frame selection algorithm prioritizes clear frames from category C1. If C1 is empty, it selects the "clearest" frame from C2 (the one with the smallest blurred degree). This strategy ensures that the VO system primarily uses high-quality frames, minimizing the impact of blurred images on motion estimation.

Datasets

The paper evaluates its methods using:

  • NewCollege Dataset: An existing benchmark dataset captured by a stereo camera system, containing mostly clear images with some naturally occurring blurred frames.
  • Real Captured Datasets (dataset1 and dataset2): Datasets captured by the authors' mobile platform on campus, specifically designed to include numerous blurred images due to rough roads and varying light conditions. These datasets were collected while the platform walked in closed loops.

Key Insights and Ideas

  • Blur is a pervasive problem: The paper highlights that blurred images are not an edge case but a common occurrence in real-world robotics, significantly impacting VO performance.
  • SIGD offers a good trade-off: The proposed SIGD metric effectively quantifies blur and is computationally much cheaper than other state-of-the-art metrics, making it suitable for real-time VO.
  • Adaptability is crucial: A static blur threshold is insufficient; an adaptive threshold that can adjust to changing conditions is essential for robust classification.
  • Key-frame selection is key: By intelligently selecting frames, the system can avoid processing problematic blurred images, thereby improving VO accuracy without needing to deblur every frame.
  • Anti-blurring benefits standard VO: The proposed framework can be integrated with existing VO algorithms (like libviso2 and SVO) to enhance their performance in blurred conditions.

Key Results

  • SIGD Performance: SIGD was shown to be directly monotone to linear motion blur, Gaussian blur, and rotation motion blur, indicating it reliably reflects the degree of blur. It also demonstrated significantly lower computational time compared to other metrics like Marziliano, JNBM, and CPBD.
  • Adaptive Classification: Experiments on both benchmark and real datasets showed that the adaptive classification algorithm could effectively distinguish between clear and blurred frames, even in environments with varying conditions.
  • VO Improvement: When integrated with standard VO algorithms (libviso2 and SVO), the anti-blurred key-frame selection approach significantly reduced the closed-loop error. For instance, on dataset2, the error for libviso2 dropped from 21.2164m (frame-by-frame) to 6.5788m (anti-blurred key-frame), and for SVO, it dropped from 29.9466m to 8.1435m.
  • Comparison with AP: The proposed method (SVO + A-B) outperformed Alberto Pretto's (AP) algorithm, which relies on a linear motion blur assumption, demonstrating broader applicability.
  • Computational Cost: The addition of the anti-blurred framework only slightly increased the average computation time per frame (e.g., SVO + A-B took 58.5ms compared to SVO's 55ms), making it practical for real-time applications.

Limitations and Future Work

  • Limitations: The paper acknowledges that the AP algorithm's assumption of linear blurred motions might not always hold in practice, especially on rough roads with non-linear situations. While their method doesn't require this assumption, the paper doesn't explicitly state limitations of their own approach beyond the need for experimental parameter tuning (e.g., for Dmin/Dmax thresholds).
  • Future Work: The authors suggest incorporating additional sensors, such as inertial sensors, into the anti-blurred VO framework to further mitigate the impact of blurred images. They also mention the possibility of including illustration situations (likely referring to lighting conditions) into their framework.

Summary updated

Abstract

Visual odometry (VO) estimation from blurred image is a challenging problem in practical robot applications, and the blurred images will severely reduce the estimation accuracy of the VO. In this paper, we address the problem of visual odometry estimation from blurred images, and present an adaptive visual odometry estimation framework robust to blurred images. Our approach employs an objective measure of images, named small image gradient distribution (SIGD), to evaluate the blurring degree of the image, then an adaptive blurred image classification algorithm is proposed to recognize the blurred images, finally we propose an anti-blurred key-frame selection algorithm to enable the VO robust to blurred images. We also carried out varied comparable experiments to evaluate the performance of the VO algorithms with our anti-blur framework under varied blurred images, and the experimental results show that our approach can achieve superior performance comparing to the state-of-the-art methods under the condition with blurred images while not increasing too much computation cost to the original VO algorithms.
BibTeX
@inproceedings{zhao2016filtering,
  author = {Haiying Zhao and Yong Liu and Xiaojia Xie and Yiyi Liao and Xixi Liu},
  title = {{Filtering Based Adaptive Visual Odometry Sensor Framework Robust to Blurred Images}},
  booktitle = {Sensors},
  year = {2016},
  url = {https://openreview.net/forum?id=6gncDiZfydw2},
}
Figure