Learning to Navigate by Growing Deep Networks

Thushan Ganegedara, Lionel Ott, Fabio RamosarXiv 2017 · arXiv

Abstract

Traditional autonomous navigation systems often struggle with the "Goldilocks" problem of fixed-structure neural networks: architectures that are either too simple to capture environmental complexity or too large for efficient training. This paper presents Reinforced Adaptive Denoising Autoencoders (RA-DAE), a self-supervised framework that enables robots to learn navigation from scratch. By employing reinforcement learning to dynamically adjust its own network structure—growing, merging, or pooling neurons—the RA-DAE system adapts to environmental complexity in real-time. Our results show that this approach significantly improves learning speed and reduces computational overhead by nearly 50% compared to fixed-structure models, while effectively mitigating catastrophic forgetting.

Introduction

Autonomous navigation is a cornerstone of robotics, yet it typically relies on two suboptimal approaches: hand-coded heuristics that fail in novel environments, or deep learning models with static architectures. Fixed-structure models present a significant challenge; if a network is too small, it lacks the capacity to model complex environments, whereas an oversized network wastes computational resources and requires excessive training time. Furthermore, most deep learning approaches are brittle when faced with "covariate shift"—performance degradation caused by changes in environmental conditions like lighting or new obstacle types.

To overcome these limitations, we propose a system that starts with a "blank slate" and builds its internal representation through exploration. By utilizing self-generated labels from physical interactions—where collisions serve as negative signals and successful movements as positive ones—the robot autonomously develops a vision system tailored to its specific operational context.

Methodology: Reinforced Adaptive Denoising Autoencoders

The core of our approach is the Reinforced Adaptive Denoising Autoencoder (RA-DAE). The system operates through a continuous self-supervision loop where visual data is paired with labels derived from a laser "bump" sensor.

System Architecture and Self-Supervision

The navigation pipeline integrates sensor data, perception, and a learning module. As the robot moves, it captures images and records movement outcomes. These are stored in dual data pools: one dedicated to training newly added neurons and another for fine-tuning the entire network on instances it previously misclassified. This dual-pool strategy ensures that the model remains robust to difficult edge cases while quickly incorporating new features.

The RA-DAE system pipeline, illustrating the flow from sensors to the learning and classification module.

Dynamic Network Growth via Reinforcement Learning

Unlike standard networks where the architecture is predefined by a human designer, RA-DAE uses a Q-learning agent to manage its own structure. Based on the generative error (reconstruction accuracy) and discriminative error (collision prediction accuracy), the agent selects one of three actions to modify the network:

  • Increment: Adding neurons to capture novel environmental features when errors are high.
  • Merge: Combining redundant neurons to streamline the model and reduce overfitting.
  • Pool: Optimizing existing weights without structural changes when the architecture is sufficient.

Furthermore, the output layer is modified to use $K$ individual softmax nodes, one for each discrete action (Left, Straight, Right). This allows the model to represent scenarios where multiple actions are equally valid for a given visual input, leading to smoother navigation trajectories.

The RA-DAE neural network structure, showing hidden layers and the multiple softmax output nodes.

Experimental Evaluation

The system was evaluated across three distinct environments to test its adaptability:

  1. Simulation: A virtual indoor environment using the MORSE simulator, providing a baseline for rapid iteration.
  2. Real-World Office: A controlled setting with static lighting but various furniture obstacles.
  3. Real-World Outdoor: A complex environment characterized by variable lighting, shadows, and diverse ground textures.

The hardware platform consisted of a robot equipped with a Firefly MV monocular camera for vision and a Hokuyo laser scanner. Crucially, the laser was used strictly as a binary "bump" sensor for automated labeling, rather than for active obstacle avoidance.

Results and Discussion

Experimental results demonstrate that RA-DAE achieves a steeper reduction in collisions during early learning phases compared to fixed-structure Stacked Denoising Autoencoders (SDAE) and Logistic Regression. By starting with a minimal network and growing it only as needed, RA-DAE reduced training and prediction time by approximately 50%.

Qualitatively, the robot demonstrated an ability to learn meaningful environmental features. In outdoor settings, it successfully learned to associate "dark blobs" with obstacles and "light areas" with traversable paths. The adaptive mechanism also mitigated "catastrophic forgetting" by allocating new neurons for novel features while preserving the weights associated with previously learned concepts.

Limitations and Future Work

While effective for discrete action spaces, scaling to continuous control may require more advanced RL techniques like Deep Double Q-Networks (DDQN). Additionally, integrating convolutional layers remains a challenge because CNN filters are shared across nodes, making the dynamic resizing of layers mathematically more complex. Future research will explore using RA-DAE for transfer learning, adapting pre-trained models to novel environments through structural growth.

Summary updated

Abstract

Adaptability is central to autonomy. Intuitively, for high-dimensional learning problems such as navigating based on vision, internal models with higher complexity allow to accurately encode the information available. However, most learning methods rely on models with a fixed structure and complexity. In this paper, we present a self-supervised framework for robots to learn to navigate, without any prior knowledge of the environment, by incrementally building the structure of a deep network as new data becomes available. Our framework captures images from a monocular camera and self labels the images to continuously train and predict actions from a computationally efficient adaptive deep architecture based on Autoencoders (AE), in a self-supervised fashion. The deep architecture, named Reinforced Adaptive Denoising Autoencoders (RA-DAE), uses reinforcement learning to dynamically change the network structure by adding or removing neurons. Experiments were conducted in simulation and real-world indoor and outdoor environments to assess the potential of self-supervised navigation. RA-DAE demonstrates better performance than equivalent non-adaptive deep learning alternatives and can continue to expand its knowledge, trading-off past and present information.
BibTeX
@misc{ganegedara2017learning,
  author = {Thushan Ganegedara and Lionel Ott and Fabio Ramos},
  title = {{Learning to Navigate by Growing Deep Networks}},
  year = {2017},
  eprint = {1712.05084},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1712.05084},
}
Figure