Exploring to learn visual saliency: The RL-IAC approach

Céline Craye, Timothée Lesort, David Filliat, Jean-François Goudou

Abstract

Autonomous robots operating in dynamic indoor environments require the ability to identify interesting or "salient" objects on-the-fly. This paper presents a novel framework that combines self-supervised incremental learning with a Reinforcement Learning-based Intelligent Adaptive Curiosity (RL-IAC) exploration strategy. This approach allows a robot to learn a visual saliency model without human intervention while optimizing its movement to prioritize high-learning-progress areas.

Introduction

Most state-of-the-art object detection and saliency models rely on massive, pre-labeled datasets and intensive offline training. However, mobile robots in home or office settings must be flexible, adapting to new environments without human intervention. This research addresses two primary challenges: how a robot can learn to recognize salient objects (like chairs or boxes) without human-provided labels, and how it should move through a building to improve its internal model as efficiently as possible.

Methods

The proposed architecture consists of two intertwined systems: Incremental Saliency Learning (ISL) and RL-IAC Exploration.

The proposed architecture integrates Incremental Saliency Learning (ISL) with a Reinforcement Learning-based Intelligent Adaptive Curiosity (RL-IAC) exploration strategy.

Incremental Saliency Learning

The robot learns what is "salient" by using its depth sensor as a "teacher" for its RGB camera. This cross-modal transfer allows the robot to eventually predict saliency using only RGB images, even for objects beyond the depth sensor's range.

  • Feature Extraction: The system uses a pre-trained GoogLeNet CNN, extracting 1024 high-level feature maps from the Class Activation Mapping (CAM) layer.
  • Self-Supervision: A depth-based segmentation algorithm identifies objects sitting on flat surfaces (floors or tables). These are used as "pseudo-labels" (Salient vs. Not Salient).
  • Online Classifier: An Online Random Forest is trained on these features and labels, upscaled using SEEDS superpixels to maintain high resolution.
Depth-based segmentation identifies planes (walls, floors) and objects to provide 'salient' and 'not salient' labels for the RGB classifier.

RL-IAC Exploration

The core innovation is Reinforcement Learning - Intelligent Adaptive Curiosity (RL-IAC). Traditional curiosity-based exploration is often "greedy," moving to the most interesting spot next without considering travel costs. RL-IAC uses Q-Learning to plan a path that maximizes total Learning Progress (LP)—the rate at which prediction error drops—while minimizing the time and battery spent traveling between distant regions.

Datasets

The system was evaluated on three distinct datasets:

  • ENSTA Dataset: A 15-minute video sequence (4,000 frames) recorded by a Pioneer 3DX robot in an office building.
  • Washington RGB-D Scenes: A public dataset of indoor tabletop scenes.
  • Ciptadi Dataset: A public dataset specifically for evaluating RGB-D saliency.

Key Results

  • Saliency Accuracy: The robot’s self-taught model (ISL) outperformed several state-of-the-art offline models (such as SALGAN and DSS) on the robot-specific datasets. This demonstrates that a model specialized for a specific environment can beat a general model trained on generic images.
  • Learning Efficiency: In simulations, the RL-IAC strategy reduced the model's error rate significantly faster than random exploration or uniform coverage.
  • Behavioral Adaptation: Qualitative results showed the robot spent early time in "easy" areas (like corridors) to learn the background, then moved to "complex" areas (like labs) to learn objects, and finally ignored areas once they were mastered.

Limitations and Future Work

The current "teacher" (depth segmentation) is limited to recognizing objects on flat planes; objects hanging from walls or ceilings are currently ignored. Future work may involve fine-tuning the CNN feature extractor online or moving toward more computationally efficient binary CNN architectures to reduce the onboard resource requirements for mobile platforms.

Summary updated

Abstract

The problem of object localization and recognition on autonomous mobile robots is still an active topic. In this context, we tackle the problem of learning a model of visual saliency directly on a robot. This model, learned and improved on-the-fly during the robot’s exploration provides an efficient tool for localizing relevant objects within their environment. The proposed approach includes two intertwined components. On the one hand, we describe a method for learning and incrementally updating a model of visual saliency from a depth-based object detector. This model of saliency can also be exploited to produce bounding box proposals around objects of interest. On the other hand, we investigate an autonomous exploration technique to efficiently learn such a saliency model. The proposed exploration, called Reinforcement Learning-Intelligent Adaptive Curiosity (RL-IAC) is able to drive the robot’s exploration so that samples selected by the robot are likely to improve the current model of saliency. We then demonstrate that such a saliency model learned directly on a robot outperforms several state-of-the-art saliency techniques, and that RL-IAC can drastically decrease the required time for learning a reliable saliency model.
BibTeX
@article{craye2019exploring,
  author = {Céline Craye and Timothée Lesort and David Filliat and Jean-François Goudou},
  title = {{Exploring to learn visual saliency: The RL-IAC approach}},
  journal = {Robotics Auton. Syst.},
  year = {2019},
  doi = {10.1016/j.robot.2018.11.012},
}
Figure