Abstract
Autonomous robots operating in dynamic indoor environments require the ability to identify interesting or "salient" objects on-the-fly. This paper presents a novel framework that combines self-supervised incremental learning with a Reinforcement Learning-based Intelligent Adaptive Curiosity (RL-IAC) exploration strategy. This approach allows a robot to learn a visual saliency model without human intervention while optimizing its movement to prioritize high-learning-progress areas.
Introduction
Most state-of-the-art object detection and saliency models rely on massive, pre-labeled datasets and intensive offline training. However, mobile robots in home or office settings must be flexible, adapting to new environments without human intervention. This research addresses two primary challenges: how a robot can learn to recognize salient objects (like chairs or boxes) without human-provided labels, and how it should move through a building to improve its internal model as efficiently as possible.
Methods
The proposed architecture consists of two intertwined systems: Incremental Saliency Learning (ISL) and RL-IAC Exploration.
Incremental Saliency Learning
The robot learns what is "salient" by using its depth sensor as a "teacher" for its RGB camera. This cross-modal transfer allows the robot to eventually predict saliency using only RGB images, even for objects beyond the depth sensor's range.
- Feature Extraction: The system uses a pre-trained GoogLeNet CNN, extracting 1024 high-level feature maps from the Class Activation Mapping (CAM) layer.
- Self-Supervision: A depth-based segmentation algorithm identifies objects sitting on flat surfaces (floors or tables). These are used as "pseudo-labels" (Salient vs. Not Salient).
- Online Classifier: An Online Random Forest is trained on these features and labels, upscaled using SEEDS superpixels to maintain high resolution.
RL-IAC Exploration
The core innovation is Reinforcement Learning - Intelligent Adaptive Curiosity (RL-IAC). Traditional curiosity-based exploration is often "greedy," moving to the most interesting spot next without considering travel costs. RL-IAC uses Q-Learning to plan a path that maximizes total Learning Progress (LP)—the rate at which prediction error drops—while minimizing the time and battery spent traveling between distant regions.
Datasets
The system was evaluated on three distinct datasets:
- ENSTA Dataset: A 15-minute video sequence (4,000 frames) recorded by a Pioneer 3DX robot in an office building.
- Washington RGB-D Scenes: A public dataset of indoor tabletop scenes.
- Ciptadi Dataset: A public dataset specifically for evaluating RGB-D saliency.
Key Results
- Saliency Accuracy: The robot’s self-taught model (ISL) outperformed several state-of-the-art offline models (such as SALGAN and DSS) on the robot-specific datasets. This demonstrates that a model specialized for a specific environment can beat a general model trained on generic images.
- Learning Efficiency: In simulations, the RL-IAC strategy reduced the model's error rate significantly faster than random exploration or uniform coverage.
- Behavioral Adaptation: Qualitative results showed the robot spent early time in "easy" areas (like corridors) to learn the background, then moved to "complex" areas (like labs) to learn objects, and finally ignored areas once they were mastered.
Limitations and Future Work
The current "teacher" (depth segmentation) is limited to recognizing objects on flat planes; objects hanging from walls or ceilings are currently ignored. Future work may involve fine-tuning the CNN feature extractor online or moving toward more computationally efficient binary CNN architectures to reduce the onboard resource requirements for mobile platforms.
Summary updated