FrontierNet: Learning Visual Cues to Explore

Boyang Sun, Hanzhi Chen, Stefan Leutenegger, Cesar Cadena, Marc Pollefeys, Hermann Blum

Abstract

Autonomous exploration of unknown environments is a fundamental challenge in robotics. Traditional methods rely heavily on dense 3D occupancy maps to identify "frontiers"—the boundaries between known and unknown space. However, these approaches are computationally expensive and ignore rich semantic visual cues. This paper introduces FrontierNet, a learning-based system that identifies frontiers and predicts potential information gain directly from 2D RGB images. By leveraging visual context, FrontierNet achieves a 15% improvement in early-stage exploration efficiency over traditional geometric methods.

Introduction

To explore an unknown building, a robot must continuously determine which viewpoints will yield the most new information. Conventional "frontier-based" exploration algorithms typically construct a dense 3D occupancy grid and search for edges between free and unknown voxels. This approach suffers from two primary limitations: high computational overhead for maintaining dense maps and a lack of visual context. A geometric map treats all openings equally, whereas a human observer can distinguish between a promising hallway and a dead-end gap behind furniture. FrontierNet addresses these issues by learning to recognize exploration frontiers directly from visual appearance cues like texture, color, and semantics.

Methodology

The FrontierNet architecture is a deep learning model designed for joint frontier proposal and information gain prediction from a single RGB image and a monocular depth prior.

Architecture and Pipeline

The model utilizes a UNet-like structure with two specialized output heads:

  1. Frontier Proposal Head: This head regresses a distance field on the 2D image, identifying pixels that constitute the boundary of the robot's current field of view.
  2. Information Gain Head: This head performs multi-class classification to estimate the volume of unmapped space lying beyond specific frontier pixels.
The FrontierNet system pipeline: from RGB input to 3D exploration goals.

3D Goal Generation

To translate 2D predictions into actionable navigation, the system employs a multi-step "lifting" process. It uses depth gradients to determine optimal robot orientations and applies the HDBSCAN algorithm to cluster dense 2D frontier pixels into a sparse set of 3D candidate viewpoints. These points are projected into 3D space by averaging foreground and background depths at the detected edges.

Experimental Results

FrontierNet was trained and evaluated using the HM3D (Habitat-Matterport 3D) dataset, featuring high-fidelity scans of diverse indoor environments.

Efficiency and Success Rate

The system demonstrated a 15% improvement in exploration efficiency during the critical early stages (the first 25-50% of the map) compared to state-of-the-art baselines like SEER and NBVP. It consistently achieved the highest success rates across various validation scenes, including complex multi-floor layouts.

Comparison of exploration paths between FrontierNet and the Classic Frontier baseline.

Robustness and Real-world Deployment

A key finding is FrontierNet's robustness to depth noise. Even when using estimated monocular depth, the system outperformed traditional methods provided with perfect simulator depth. This is attributed to the model's reliance on relative visual cues rather than absolute geometric precision. The system's real-time capability (~5 Hz inference) was validated through deployment on a Boston Dynamics Spot robot in a cluttered indoor environment.

Conclusion and Future Work

FrontierNet represents a significant shift toward visually-guided autonomous exploration. By integrating semantic context into frontier detection, it enables more efficient and robust navigation. Future research will focus on addressing scale ambiguity in monocular depth, handling dynamic environments, and incorporating explicit object-level semantics to guide task-specific exploration.

Summary updated

BibTeX
@article{sun2025frontiernet,
  author = {Boyang Sun and Hanzhi Chen and Stefan Leutenegger and Cesar Cadena and Marc Pollefeys and Hermann Blum},
  title = {{FrontierNet: Learning Visual Cues to Explore}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2025},
  url = {https://openreview.net/forum?id=9OckDp3ToI},
}

Loading related content...

Figure