Fast Adaptive Object Detection towards a Smart Environment by a Mobile Robot

Shigeru Takano, Ilya Loshchilov, David Meunier, Michèle Sebag, Einoshin SuzukiAmI 2013 · DOI

TL;DR

This paper introduces a novel method for fast, adaptive object detection for mobile robots in smart environments. By leveraging a specially designed Lifting Complex Wavelet Transform (LCWT) that can be tuned to specific environments, the robot can efficiently recognize objects and their attributes, paving the way for more intelligent and responsive robotic systems.

What Problem Does This Paper Address?

Designing "smart environments" often relies on accurately recognizing objects and their states (e.g., moving, stationary) within that environment. Traditionally, this has been a manual process for designers, which is burdensome and doesn't scale. While autonomous mobile robots offer a promising solution for this recognition task, their effectiveness is heavily dependent on their ability to adapt to the specific environment they are operating in. Existing object detection methods often require significant computational resources, making real-time performance on a mobile robot challenging. This paper aims to bridge this gap by developing a fast and adaptive object detection system that can be deployed on-board a mobile robot, enabling it to understand and interact with its surroundings more intelligently.

Methods

The core of this work lies in the Lifting Complex Wavelet Transform (LCWT). Unlike standard wavelet transforms, LCWTs have controllable "lifting parameters" that allow them to be customized. The authors propose to tune these parameters to extract local image features that are robust to shifts, scales, and rotations, and importantly, are optimized for the specific environment.

The process involves:

  • Multi-scale Feature Extraction: Using the tunable LCWT, the robot extracts multi-scale local features from images. These features are designed to be efficient to compute and robust.
  • Keypoint Detection and Description: A keypoint detection algorithm identifies salient points in the image. For each keypoint, a 128-dimensional feature vector is computed based on oriented gradients. This vector is then normalized for illumination invariance.
  • Visual Vocabulary and Clustering: The extracted feature vectors from training images are clustered using an e-means clustering algorithm. Each cluster represents a "visual word," forming a visual vocabulary.
  • Adaptive LCWT Training: The crucial adaptive step involves training the LCWT parameters. This is achieved using a Genetic Algorithm (GA). The GA aims to maximize the entropy of the distribution of feature vectors after clustering. This maximization is hypothesized to lead to a more discriminative set of visual words, thus improving object detection. The GA iteratively selects, breeds, and mutates lifting parameters to find the optimal set for a given environment.
  • Object Detection: Once the LCWTs are trained and a visual vocabulary is built, the system performs object detection by comparing the visual word histogram of a query image against a database of images using histogram intersection.

Datasets

The paper utilizes training images captured by the mobile robot in an experimental room. Specifically, 132 training images were used to build the visual vocabulary. The authors do not mention using any publicly available benchmark datasets for evaluation, but rather rely on experimental results from their own robot in their controlled environment.

Key Insights and Ideas

  • Adaptation is Key: The central idea is that a mobile robot's ability to "smartly" perceive its environment hinges on its capacity to adapt its feature extraction mechanisms to that specific environment.
  • LCWT as a Tunable Feature Extractor: The LCWT provides a flexible framework for feature extraction, allowing for optimization beyond generic feature detectors.
  • Entropy as an Optimization Metric: Using the entropy of the visual word distribution as a fitness function for the Genetic Algorithm is an interesting approach to guide the learning of LCWT parameters, aiming for a more discriminative feature space.
  • Bridging On-board Vision and Environment Understanding: The work directly addresses the challenge of performing complex vision tasks like object detection in real-time on resource-constrained mobile robots.

Key Results

  • The proposed method can design LCWTs that adapt to the environment.
  • The adaptive LCWTs lead to improved object detection performance, with target images being ranked higher in retrieval tasks compared to using initial, non-adapted LCWTs.
  • The experimental results demonstrate the effectiveness of the method in accurately detecting existing objects in the room.
  • The computational time for object detection using the trained LCWTs is reported to be within one second.

Limitations and Future Work

  • Highly Detailed Images: The paper notes that highly detailed images with many keypoints can sometimes be ranked higher than expected target images, suggesting a need to consider precision and recall more carefully in future parameter tuning.
  • Offline Training: The current method involves offline training of LCWT parameters. The authors explicitly state their future work will focus on developing an online algorithm for learning the LCWT. This would allow the robot to adapt its perception capabilities continuously as it explores new environments.

Summary updated

BibTeX
@inproceedings{takano2013fast,
  author = {Shigeru Takano and Ilya Loshchilov and David Meunier and Michèle Sebag and Einoshin Suzuki},
  title = {{Fast Adaptive Object Detection towards a Smart Environment by a Mobile Robot}},
  booktitle = {AmI},
  year = {2013},
  doi = {10.1007/978-3-319-03647-2\_13},
}
Figure