Watch Your STEPP: Semantic Traversability Estimation Using Pose Projected Features

Sebastian Ægidius, Dennis Hadjivelichkov, Jianhao Jiao, Jonathan Embley-Riches, Dimitrios Kanoulas

Abstract

Autonomous navigation in unstructured environments remains a significant challenge for legged robots. Traditional geometric methods often fail to capture semantic nuances, while supervised learning requires labor-intensive manual labeling. We present STEPP (Semantic Traversability Estimation using Pose Projected Features), a robot-agnostic framework that learns traversability by observing human walking paths. By leveraging pre-trained foundation models (DINOv2) and a self-supervised anomaly detection scheme, STEPP identifies safe terrain based on reconstruction error. Our approach demonstrates high accuracy in diverse environments and successful real-world deployment on an ANYmal-D quadruped.

Introduction

For an autonomous robot, "traversability" is a hardware-specific property. A legged robot might easily traverse tall grass that would obstruct a small wheeled platform, yet both must avoid hazards like hidden ditches or large boulders. Existing solutions typically rely on geometric mapping (slopes and heights) or supervised semantic segmentation. Geometric approaches lack semantic context (e.g., distinguishing between soft grass and a deep pond), while supervised methods require thousands of hours of manual pixel-level labeling, which does not scale to the infinite variety of natural terrains.

Figure 1: The STEPP framework enables a quadruped robot to navigate complex terrain by predicting traversability costs based on visual features.

The STEPP framework addresses these issues by asking: Can we learn what is "safe" simply by watching where humans or other agents have successfully walked before?

Methodology: Self-Supervised Anomaly Detection

The STEPP pipeline utilizes a self-supervised "anomaly detection" approach, where hazardous or unfamiliar terrain is identified by high reconstruction error in a vision model.

Figure 3: The STEPP pipeline architecture, showing feature extraction via DINOv2, path projection, and the autoencoder-based traversability estimation.
  1. Feature Extraction: RGB images are processed through DINOv2, a pre-trained Vision Transformer, to generate dense, high-dimensional feature embeddings for every pixel.
  2. Pose Projection: Using odometry and SLAM, the path an agent actually traversed is projected onto the camera images. SLIC superpixels are used to group similar-looking pixels.
  3. Autoencoder Training: A Multi-Layer Perceptron (MLP) autoencoder is trained exclusively on the features of the traversed ground. The autoencoder learns to compress and reconstruct these "safe" features.
  4. Traversability Estimation: During inference, the model attempts to reconstruct features for all terrain segments. Low reconstruction error indicates terrain similar to previously traversed paths (Traversable), while high error signals "anomalous" or unfamiliar terrain (Untraversable).

Datasets and Evaluation

The researchers utilized a combination of real-world and synthetic data:

  • Real-World Forest: ~55,000 images from a custom camera-LiDAR rig.
  • Indoor Lab: ~5,000 images of controlled settings.
  • Synthetic Data: ~27,000 images from a custom Unreal Engine plugin, simulating diverse human-like trajectories.
  • Benchmark: 500 manually annotated forest images served as the ground truth for evaluation.

Key Results and Insights

The model achieved an accuracy of 83.5% when trained on a mixed dataset. Key findings include:

  • Human-to-Robot Transfer: Human walking paths serve as an effective proxy for quadruped traversability.
  • Foundation Model Utility: Leveraging DINOv2 provides sufficient latent knowledge to distinguish terrain types without training a vision model from scratch.
  • Simulation Benefits: While synthetic data alone yielded 68.4% accuracy, its inclusion significantly boosted the model's ability to generalize in real-world scenarios.

The system was successfully integrated into an ANYmal-D quadruped robot. In field tests, the robot navigated tall grass and avoided obstacles like rope fences and trees that traditional height-map-based planners often missed.

Limitations and Future Work

Despite its success, several areas for improvement remain. The current inference speed of 2.5 Hz introduces synchronization lag, which may be problematic at higher velocities. Additionally, the model exhibits an "egocentric bias" toward the bottom-center of images, reflecting typical human gaze patterns. Future work will focus on improving LiDAR-camera calibration for better depth precision and exploring data augmentation to mitigate perspective biases.

Summary updated

BibTeX
@inproceedings{gidius2025watch,
  author = {Sebastian Ægidius and Dennis Hadjivelichkov and Jianhao Jiao and Jonathan Embley-Riches and Dimitrios Kanoulas},
  title = {{Watch Your STEPP: Semantic Traversability Estimation Using Pose Projected Features}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2025},
  doi = {10.1109/icra55743.2025.11127781},
}
Figure