TamedPUMA: safe and stable imitation learning with geometric fabrics

Saray Bakker, Rodrigo Pérez-Dattari, Cosimo Della Santina, Wendelin Böhmer, Javier Alonso-Mora

Abstract

TamedPUMA addresses the challenge of deploying imitation learning (IL) in unstructured environments by integrating the adaptability of neural motion policies with the formal safety guarantees of Geometric Fabrics. By formulating learned demonstrations as second-order dynamical systems, the framework enables robots to perform complex tasks while reactively avoiding obstacles and respecting joint limits in real-time. Experimental results demonstrate a 100% success rate in obstacle-rich environments with millisecond-level computation, bridging the gap between flexible learning and robust safety.

Introduction

Teaching robots to operate in unstructured environments—like homes or farms—is notoriously difficult. Imitation Learning (IL) is a popular solution because it allows non-experts to "show" a robot how to perform a task. However, IL has a major safety gap: a robot might learn to perform a task perfectly in isolation but lacks inherent mechanisms to avoid dynamic obstacles or respect its own joint limits.

Existing solutions usually fall into two camps: optimization-based methods (like MPC), which are safe but computationally expensive, and modulation-based methods, which can avoid obstacles but often "break" the stability of the learned motion, causing the robot to fail to reach its goal. TamedPUMA aims to bridge this gap, providing a method that is stable (always reaches the goal), safe (avoids collisions), and fast (runs in milliseconds).

Methods

The TamedPUMA framework integrates two sophisticated components: PUMA (Policy via neUral Metric leArning) and Geometric Fabrics. PUMA represents motion as a second-order dynamical system at the acceleration level, using a specific stability loss to ensure convergence. Geometric Fabrics use differential geometry to define robot behavior, treating avoidance tasks as energy-conserving systems and navigation tasks as energy-dissipating systems.

Overview of the TamedPUMA architecture, illustrating the mapping between configuration space and task space, the integration of learned policies with geometric fabrics, and the final behavior combination.

The authors propose two ways to "tame" these components:

  1. Forcing Policy Method (FPM): The learned PUMA policy is treated as an external force pushing the geometric fabric toward the goal.
  2. Compatible Potential Method (CPM): This mathematically rigorous approach derives a potential field directly from the PUMA model's internal latent space. Because this potential is compatible with the fabric's geometry, it provides strong guarantees that the robot will reach its goal without getting stuck in local minima.

Experiments and Results

The researchers evaluated TamedPUMA using human-provided demonstrations for tasks like tomato picking and pouring. Testing included 30 simulated scenarios in PyBullet with dynamic obstacles and real-world validation on a 7-DOF KUKA iiwa manipulator.

Key results include:

  • Success Rate: TamedPUMA achieved a 100% success rate in obstacle-rich environments, significantly outperforming vanilla fabrics (83%) and standard PUMA (40%).
  • Demonstration Fidelity: The system stayed much closer to the original human demonstration than vanilla fabrics, with a path difference of only 0.02–0.04m.
  • Real-Time Performance: The algorithm runs in 4–7 milliseconds. In simulations with 1,000 obstacles, TamedPUMA maintained its millisecond-level performance, while optimization-based methods became unusable.

Conclusion and Future Work

TamedPUMA successfully combines the strengths of deep imitation learning with the safety of geometric control. By operating at the acceleration level and leveraging latent space potentials, it provides a robust solution for reactive robot control. While highly effective, future work will focus on extending formal convergence proofs to dynamic obstacles and optimizing redundancy resolution to ensure more natural, human-like postures.

Summary updated

BibTeX
@inproceedings{bakker2025tamedpuma,
  author = {Saray Bakker and Rodrigo Pérez-Dattari and Cosimo Della Santina and Wendelin Böhmer and Javier Alonso-Mora},
  title = {{TamedPUMA: safe and stable imitation learning with geometric fabrics}},
  booktitle = {L4DC},
  year = {2025},
}
Figure