Curiosity based exploration for learning terrain models

Yogesh A. Girdhar, David Whitney, Gregory Dudek

TL;DR

This paper introduces a "curiosity-driven" robotic exploration strategy designed to efficiently learn visual terrain models in unknown environments. By using a realtime spatiotemporal topic model to evaluate the semantic novelty (topic perplexity) of neighboring areas, the robot plans paths that maximize information gain. This approach significantly outperforms standard coverage or random walk strategies in building accurate terrain maps, especially when exploration time is strictly limited.

What Problem Does This Paper Address?

Traditional robotic exploration usually focuses on spatial navigation—finding the boundaries between free space and obstacles to build a geometric map. However, in open environments like aerial surveying or underwater reef monitoring, obstacle avoidance isn't the primary goal. Instead, the robot needs to understand what it is looking at (e.g., distinguishing between sand, coral, urban areas, or forests).

The challenge is twofold: First, low-level sensor data (like raw pixel colors or edges) is too noisy to represent abstract terrain concepts. Second, without a pre-existing map, how does a robot decide where to move next to learn a generalized, accurate model of the environment as quickly as possible? The authors tackle this by framing exploration as an unsupervised learning problem driven by a mathematical definition of "curiosity."

Methods

The authors adapt topic modeling—a technique originally used in Natural Language Processing (like Latent Dirichlet Allocation, or LDA) to find hidden themes in text—for visual data.

  • Visual Words: Instead of text, the system extracts "visual words" from images using ORB (for local features) and textons (for texture/color).
  • ROST (Realtime Online Spatiotemporal Topic Modeling): The robot builds a topic model on the fly. To ensure topics cluster logically in physical space, the model uses Dirichlet priors that factor in the topic distribution of neighboring spatial cells.
  • Realtime Gibbs Sampling: To keep compute times constant as the robot gathers more data, the algorithm only updates the topic label for the current observation and one randomly sampled past observation per time step.
  • Curiosity-Based Routing: The robot decides its next move by calculating a weight for adjacent cells. The authors tested four weighting strategies:
  • Random Walk
    1. Stochastic Coverage (repulsed by previously visited locations)
    2. Word Perplexity (drawn to areas with novel low-level visual features)
    3. Topic Perplexity (drawn to areas with novel high-level semantic topics)

Datasets

The authors evaluated their algorithms by simulating robot paths over three large, static maps:

  • Montreal1 & Montreal2: Two aerial satellite maps (1024x1024 pixels, ~1.6M to 3.2M visual words).
  • SouthBellairs: An underwater coral reef map (2500x2500 pixels, ~1.6M visual words).

(Note: The visual word dictionary was pre-computed by extracting features from the visually diverse 1992 documentary film Baraka).

Key Insights and Ideas

  • Semantic Novelty beats Visual Noise: The most surprising insight is the stark difference between Word Perplexity and Topic Perplexity. Word perplexity performed terribly because it gets distracted by high-frequency visual noise (e.g., a highly complex but irrelevant texture). Topic perplexity succeeds because it abstracts the noise into a semantic category, allowing the robot to seek out genuinely new types of terrain rather than just visually messy areas.
  • Efficiency in Short Paths: Curiosity-driven exploration shines brightest when time is limited. If a robot has infinite time in a bounded environment, standard coverage algorithms will eventually map everything. But when the path length is short (roughly the diameter of the map), topic perplexity learns a highly accurate model much faster than competing methods.
  • Spatiotemporal Clustering: By forcing the generative model to consider the neighborhood of a cell, the resulting terrain maps are cohesive and smooth, closely matching human intuition of how terrains are laid out in the real world.

Key Results

  • Short-term superiority: For short exploration paths (80 steps, roughly the width of the maps), the Topic Perplexity strategy achieved 1.51x, 1.20x, and 1.05x higher mutual information with human-labeled ground truth across the three datasets compared to the next best performing technique.
  • Long-term convergence: For long paths (320+ steps), the Stochastic Coverage algorithm eventually matches the mean performance of the Topic Perplexity algorithm, as the bounded nature of the maps forces the coverage algorithm to eventually stumble across all terrain types.
  • Failure of Random Walk: Brownian random motion performed poorly across the board, as a random walk in 2D tends to stay localized (traveling a distance of $\sqrt{n}$ from the start), preventing the robot from discovering distinct terrains.

Limitations and Future Work

  • Dictionary Domain Shift: The visual word dictionary was pre-computed from a movie rather than the target environments. While it worked well here, relying on a static, pre-computed dictionary could limit adaptability in highly alien environments.
  • Grid Dependency: The experiments rely on a 2D grid decomposition of the world. While the authors note the math applies to any topological decomposition of spacetime, this wasn't explicitly tested on 3D or dynamic environments.
  • Word Perplexity Pitfalls: The authors openly acknowledge the failure of word-level perplexity, noting that algorithms drawn to complex word distributions will get "stuck" in complex terrains and fail to explore. Future work could potentially look at hybridizing these approaches or applying this framework to active 3D mapping.

Summary updated

Abstract

We present a robotic exploration technique in which the goal is to learn a visual model that can be used to distinguish between different terrains and other visual components in an unknown environment. We use ROST, a realtime online spatiotemporal topic modeling framework to model these terrains using the observations made by the robot, and then use an information theoretic path planning technique to define the exploration path. We conduct experiments with aerial view and underwater datasets with millions of observations and varying path lengths, and find that paths that are biased towards locations with high topic perplexity produce better terrain models with high discriminative power.
BibTeX
@inproceedings{girdhar2014curiosity,
  author = {Yogesh A. Girdhar and David Whitney and Gregory Dudek},
  title = {{Curiosity based exploration for learning terrain models}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2014},
  url = {https://openreview.net/forum?id=5hvvKRQ5BE},
}

Loading related content...

Figure