From Tabula Rasa to Emergent Abilities: Discovering Robot Skills via Real-World Unsupervised Quality-Diversity

Luca Grillotti, Lisa Coiffard, Oscar Pang, Maxence Faldor, Antoine Cully

Abstract

Autonomous skill discovery aims to enable robots to acquire diverse behaviors without explicit supervision. Learning such behaviors directly on physical hardware remains challenging due to safety and data efficiency constraints. Existing methods, including Quality-Diversity Actor-Critic (QDAC), require manually defined skill spaces and carefully tuned heuristics, limiting real-world applicability. We propose Unsupervised Real-world Skill Acquisition (URSA), an extension of QDAC that enables robots to autonomously discover and master diverse, high-performing skills directly in the real world. We demonstrate that URSA successfully discovers diverse locomotion skills on a Unitree A1 quadruped in both simulation and the real world. Our approach supports both heuristic-driven skill discovery and fully unsupervised settings. We also show that the learned skill repertoire can be reused for downstream tasks such as real-world damage adaptation, where URSA outperforms all baselines in 5 out of 9 simulated and 3 out of 5 real-world damage scenarios. Our results establish a new framework for real-world robot learning that enables continuous skill discovery with limited human intervention, representing a significant step toward more autonomous and adaptable robotic systems.

Introduction

In traditional robotics, agents are typically trained to optimize a single objective, such as walking forward. While effective in controlled settings, these "optimal" gaits often fail when the robot encounters novel environments or mechanical damage. To build resilient robotic systems, researchers utilize Quality-Diversity (QD) algorithms, which aim to discover a "repertoire" of diverse, high-performing behaviors. However, traditional QD methods face significant hurdles for real-world deployment: they often require manually defined behavior spaces, massive amounts of data, and involve high-risk exploration that can damage physical hardware.

The proposed URSA (Unsupervised Real-world Skill Acquisition) framework addresses these challenges, enabling safe, sample-efficient, and unsupervised skill discovery directly on physical hardware.

Methodology: The URSA Framework

URSA integrates three core components to facilitate real-world learning:

  1. World Models (DayDreamer): To overcome the data inefficiency of physical trials, URSA employs a world model. This allows the robot to "imagine" the consequences of its actions, practicing thousands of skill variations in a latent simulation before executing them in the real world.
  2. Unsupervised Skill Discovery: Instead of relying on human-defined heuristics for diversity, URSA utilizes a Variational Autoencoder (VAE). The VAE compresses raw sensor data—such as joint angles and body orientation—into a low-dimensional latent space. The robot then explores this space to discover a wide variety of behaviors.
  3. Constrained Quality-Diversity (QDAC): URSA optimizes for both performance (quality) and novelty (diversity) within the latent map. A critical addition is the Safety Critic, which acts as a predictive guardrail, preventing the robot from attempting behaviors that the world model forecasts will lead to falls or hardware failure.
The URSA framework architecture, showing the integration of world models, VAE-based skill discovery, and safety constraints.

Experimental Setup and Evaluation

The framework was evaluated using a Unitree A1 quadruped robot. Data was collected through:

  • Simulation: 1 million timesteps in PyBullet.
  • Real-World: 5 hours of continuous interaction.

To test adaptability, the authors benchmarked the robot across 9 simulated and 5 real-world "damage" scenarios, such as locking joints or disabling motors.

Results and Key Insights

URSA demonstrated a significant improvement over standard reinforcement learning baselines, achieving 4x greater coverage of the joint-angle space. The robot successfully learned a "menu" of movement styles, ranging from crouching shuffles to agile trots, in just five hours of real-world training.

Visualization of the learned skill repertoire, showing the 2D latent space and corresponding physical behaviors discovered by the robot.

Resilience and Adaptation

When subjected to mechanical damage, URSA utilized Iterative Trial and Error (ITE) to search its learned repertoire for a viable gait. It outperformed all baselines in the majority of damage scenarios (5/9 simulated, 3/5 real-world).

Safety as an Enabler

A key insight from this work is that safety constraints do not necessarily hinder exploration; rather, they enable it. By providing a safety critic, URSA allows the robot to aggressively explore diverse behaviors without the risk of catastrophic failure, which is essential for real-world learning.

Discussion and Limitations

While URSA represents a major step forward, several limitations remain:

  • Limb Dependency: Because skills are learned via a single neural network, damage to one leg can sometimes disable large portions of the repertoire simultaneously.
  • Autonomy Gaps: The system still requires human intervention for "resets" if the robot wanders out of its designated testing area.
  • Reward Shaping: The current reward function can lead to "crab-walking" behaviors as it does not strictly penalize lateral drift.

Future work will focus on learning safety boundaries directly from interaction data and improving the autonomy of the reset process to enable truly continuous, long-term skill discovery.

Summary updated

Abstract

Autonomous skill discovery aims to enable robots to acquire diverse behaviors without explicit supervision. Learning such behaviors directly on physical hardware remains challenging due to safety and data efficiency constraints. Existing methods, including Quality-Diversity Actor-Critic (QDAC), require manually defined skill spaces and carefully tuned heuristics, limiting real-world applicability. We propose Unsupervised Real-world Skill Acquisition (URSA), an extension of QDAC that enables robots to autonomously discover and master diverse, high-performing skills directly in the real world. We demonstrate that URSA successfully discovers diverse locomotion skills on a Unitree A1 quadruped in both simulation and the real world. Our approach supports both heuristic-driven skill discovery and fully unsupervised settings. We also show that the learned skill repertoire can be reused for downstream tasks such as real-world damage adaptation, where URSA outperforms all baselines in 5 out of 9 simulated and 3 out of 5 real-world damage scenarios. Our results establish a new framework for real-world robot learning that enables continuous skill discovery with limited human intervention, representing a significant step toward more autonomous and adaptable robotic systems. Demonstration videos are available at https://adaptive-intelligent-robotics.github.io/URSA.
BibTeX
@inproceedings{grillotti2025from,
  author = {Luca Grillotti and Lisa Coiffard and Oscar Pang and Maxence Faldor and Antoine Cully},
  title = {{From Tabula Rasa to Emergent Abilities: Discovering Robot Skills via Real-World Unsupervised Quality-Diversity}},
  booktitle = {Conference on Robot Learning},
  year = {2025},
  url = {https://openreview.net/forum?id=b8RqTaDyb3},
}
Figure