Learning an Image-based Obstacle Detector With Automatic Acquisition of Training Data

Stefano Toniolo, Jérôme Guzzi, Luca Maria Gambardella, Alessandro GiustiAAAI 2018 · DOI

Abstract

This paper presents a self-supervised approach for training a vision-based obstacle detector on a mobile robot. By leveraging low-cost infrared (IR) proximity sensors to automatically label camera images during an autonomous exploration phase, the system bypasses the need for manual data annotation. A convolutional neural network (CNN) is trained to predict the state of these proximity sensors from monocular images, enabling the robot to navigate complex environments using vision alone.

Introduction

Autonomous navigation in unknown environments typically relies on expensive sensors like LiDAR or computationally intensive SLAM algorithms. While deep learning offers a robust alternative for vision-based navigation, the "data bottleneck" — the requirement for massive, manually labeled datasets — remains a significant hurdle. This research addresses this challenge by enabling a robot to learn to "see" obstacles by "touching" them with its own physical sensors. The core concept is to use a low-bandwidth, short-range sensor (IR) to train a high-bandwidth, long-range sensor (camera).

Methodology

The proposed system utilizes a Thymio robot platform equipped with a standard webcam and five forward-facing IR proximity sensors. The learning process consists of three main stages:

1. Autonomous Data Collection

The robot explores its environment using a "smart controller." It moves forward until an IR sensor detects an obstacle, at which point it rotates to capture the obstacle from multiple viewpoints before making a random turn to continue exploration. This ensures the dataset covers various angles and positions within the camera's field of view.

2. Self-Labeling Mechanism

At each timestep $t$, the robot records a camera image $I(t)$ and the binary state of its five IR sensors $y(t)$ (1 if an obstacle is detected, 0 otherwise). The sensor data serves as an automatic ground truth label for the corresponding image.

Figure 3: An example of the self-labeling process where a Rubik's cube triggers the central IR sensors, generating a ground truth vector for the CNN.

3. Neural Network Architecture

A CNN maps 60x60 RGB images to a 5-element vector representing the probability of each proximity sensor being active. The architecture includes three pairs of convolutional and pooling layers followed by a dense layer. A sigmoid activation function in the output layer allows for independent binary classification of each sensor, enabling the detection of multiple obstacles across the field of view.

Experimental Setup and Dataset

The researchers collected a dataset of 53,560 samples over 106 minutes of recording across eight different scenarios. These scenarios varied in floor texture, obstacle types, and lighting conditions to ensure robustness. A "leave-one-scenario-out" cross-validation scheme was used to verify that the model could generalize to environments it had never encountered during training.

Figure 6: The experimental setup featuring Thymio robots navigating an environment with obstacles like traffic cones.

Results and Discussion

The model demonstrated high accuracy, particularly for the central sensors, achieving Area Under the Curve (AUC) scores near 0.95. The vision-based detector operates in real-time, allowing for smooth reactive steering and speed control.

A key finding was the "peripheral blind spot," where the network struggled more with the outermost sensors ($y_{LL}$ and $y_{RR}$). This is attributed to obstacles being at the very edge or slightly outside the camera's field of view while still triggering the wide-angle IR sensors. Furthermore, the vision system's effective range is currently limited by the short range of the IR "teacher" sensors (approximately 10cm); the camera learns to be "worried" only when objects are very close.

Conclusion

This work demonstrates that robots can effectively teach themselves to detect obstacles by correlating visual patterns with physical sensor feedback. This self-supervised paradigm reduces the reliance on manual labeling and allows robots to adapt to specific environments autonomously. Future work will explore using longer-range sensors, such as ultrasound, to extend the vision system's look-ahead distance and applying the method to larger, faster robotic platforms.

Summary updated

BibTeX
@inproceedings{toniolo2018learning,
  author = {Stefano Toniolo and Jérôme Guzzi and Luca Maria Gambardella and Alessandro Giusti},
  title = {{Learning an Image-based Obstacle Detector With Automatic Acquisition of Training Data}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.1609/aaai.v32i1.11368},
}
Figure