Abstract
This paper presents a self-supervised learning approach that enables a mobile robot to perceive obstacles at long range using only a standard camera and short-range proximity sensors. By utilizing future sensor readings as training labels for current camera views, the robot learns to predict obstacles up to 30cm away—effectively tripling the range of its physical sensors without manual data labeling.
Introduction
Small, low-cost robots frequently rely on infrared (IR) or ultrasonic proximity sensors for obstacle avoidance. While reliable, these sensors typically have a very short range (often under 10cm), forcing the robot to react only immediately before a collision. This constraint leads to jerky movements and inefficient navigation.
To achieve smoother, proactive navigation, robots generally require expensive LIDAR or complex computer vision systems that depend on massive, manually labeled datasets. This work addresses the challenge of long-range perception on a budget: how can a robot autonomously learn to interpret visual data to predict distant obstacles using its existing, limited hardware?
Methodology
The researchers employ a temporal self-supervision strategy that allows the robot to learn from its own experiences. The process involves four key stages:
- Exploration: A "Mighty Thymio" robot autonomously explores various environments, recording synchronized video, odometry, and proximity sensor data.
- Automated Labeling: For any camera frame at time $t$, the system looks ahead in the recorded trajectory. If the robot later encounters an obstacle at a distance $d$ centimeters in front of its position at time $t$, that sensor "hit" becomes the ground-truth label for that specific distance in the earlier visual frame.
- Model Architecture: A Convolutional Neural Network (CNN) is trained to map single camera frames to a $31 \times 5$ grid of binary labels, representing predicted sensor outputs for 5 sensors across 31 distance increments (0 to 30 cm).
- Masked Loss: Because the robot's path is not always linear, future data is not available for every distance in every frame. A masked loss function is used to ensure the network learns only from distances where ground truth was successfully collected.
Experimental Setup
The system was validated using a custom dataset collected by a Mighty Thymio robot equipped with five forward-facing IR sensors and a 720p webcam. The dataset includes 90 minutes of autonomous exploration across 10 diverse environments featuring various floor textures and obstacle types.
Results and Discussion
The core insight is that a robot's future experience serves as a perfect label for its current visual input, bypassing the need for human intervention. By associating physical sensor readings with visual features seen from a distance, the robot extends its effective perception range from ~8cm to 30cm.
Key results include:
- Predictive Navigation: The multi-distance depth map allows for smoother steering, as the robot can anticipate and avoid obstacles much earlier than reactive IR-based systems.
- Performance Decay: As shown in the performance heatmaps, prediction accuracy is highest at medium ranges and naturally decreases as the distance approaches the 30cm limit.
- Blind Spots: Obstacles very close to the robot or at the extreme periphery proved harder to detect. This is attributed to the camera's field of view (FOV) being narrower than the proximity sensors' coverage in certain areas.
Conclusion
This self-supervised approach effectively triples a robot's perception range without requiring manual labeling. While successful, limitations remain regarding FOV mismatches and odometry drift on slippery surfaces. Future work will explore more complex geometric projections to incorporate data from turning maneuvers and improve the robustness of the predictive model.
Summary updated