Multi-CONDOR: Learning Multistable Motion Primitives from Demonstration

Gabriele Giuseppe Di Marzo, Mariano Ramírez Montero, Maximilian Stölzle, Rodrigo Pérez-Dattari, Cosimo Della SantinaarXiv 2026 · arXiv

TL;DR

Multi-CONDOR is a new framework for teaching robots motion skills that can reach multiple valid goals or even a continuous set of outcomes (like any point on a cup's rim) from a single learned policy. By aligning robot motions with a structured latent space, it provides strong stability guarantees while significantly outperforming existing methods in both accuracy and reliability.

What Problem Does This Paper Address?

Most robot "motion primitives"—the building blocks of complex actions—are designed to go from point A to point B. But real life is rarely that simple. Imagine a box with two handles: a robot should be able to grab either one depending on which is closer or if a human bumps into it. Or consider a circular rim: any point on that rim is a valid place to grasp.

Current methods often force a choice of a single goal before the motion even starts, which kills flexibility. While some "multistable" systems exist, they often struggle to provide mathematical guarantees that the robot won't get stuck in "spurious attractors" (dead zones where the robot just stops) or they can't handle continuous goal sets like circles and curves within a unified framework.

Methods

The authors introduce Multi-CONDOR, which builds on the previous CONDOR (Contrastive Learning of Dynamical Systems) framework. The core idea is to learn a task-space vector field—a map of where to move from any given position—by projecting it into a "latent space."

Overview of the Multi-CONDOR architecture: an encoder maps task states to a latent space where reference dynamics define the goal structure.

Key technical components include:

  1. Latent Reference Dynamics: Instead of a single goal, the latent space is equipped with a "reference system" that has the desired structure. For discrete goals, they use a Gaussian potential (multiple peaks). For continuous goals, they use a radially attracting circle.
  2. Dynamics Matching: A special loss function encourages the robot's actual motion to "match" these latent dynamics. If the latent system is stable, the robot's motion inherits that stability.
  3. Separation Loss: To prevent the neural network from "cheating" by mapping all inputs to the same point, a separation loss ensures that distinct states in the real world remain distinct in the latent space.
  4. Lyapunov Analysis: The authors provide a formal proof showing that if the matching is successful, the robot is guaranteed to converge to the desired goal set.
Visualization of the two types of latent reference dynamics: discrete attractors (left) and a continuous attracting circle (right).

Datasets

The framework was evaluated on:

  • 2D Benchmarks: Seven custom-drawn 2D tasks featuring two or three distinct goals with complex, curved demonstration paths.
  • Continuous Contours: Qualitative tests on star-shaped and circular goal sets.
  • Real Robot Grasping: A Franka Emika FR3 robot arm was taught to grasp a box from either of its two handles using 10 kinesthetic demonstrations (5 per handle).

Key Insights and Ideas

  • Inherited Stability: The "aha" moment is that you don't need to bake complex stability proofs into the high-dimensional neural network directly. Instead, you define a simple, stable system in a low-dimensional latent space and force the network to mimic its topology.
  • Unified Goal Structures: The same mathematical framework can handle "pick one of these three spots" and "pick anywhere on this line" just by swapping the latent reference function.
  • Robustness to Disturbances: Because the system is an autonomous vector field (it only cares about where it is now, not what time it is), if you push the robot, it naturally flows toward the most sensible remaining goal rather than trying to "catch up" to a pre-planned trajectory.

Key Results

Multi-CONDOR showed significant improvements over baselines like StableNODE and state-machine-based CONDOR:

  • Accuracy: It reduced trajectory Root Mean Square Error (RMSE) by 72% compared to using separate models for each goal.
  • Reliability: In tasks with three goals, the baseline (StableNODE) failed to reach a goal from 7.29% of initial positions. Multi-CONDOR achieved a 0% failure rate, meaning it reached a valid goal from every tested starting point.
  • Real-World Success: On the Franka robot, the system successfully generalized from demonstrations to reach either handle of a box based on its starting position, following the demonstrated "approach" behavior (coming from the side rather than the top).
Comparison of learned dynamics: Multi-CONDOR (left) cleanly converges to goals, while StableNODE (right) creates "spurious attractors" where the robot might get stuck.

Limitations and Future Work

While powerful, Multi-CONDOR has a few areas for improvement:

  • Approximate Guarantees: The theoretical convergence depends on "exact" dynamics matching. In practice, training only encourages this, so the guarantees are empirical rather than absolute in the final neural network.
  • Separated Formulations: Currently, the user must choose between the "discrete" or "continuous" reference system. Future work aims to unify these, allowing a single policy to handle a mix of points and curves.
  • Equilibrium Shifting: In the discrete case, if Gaussian attractors are placed too close together in latent space, they can merge or shift. The authors suggest future versions should use latent dynamics where equilibria are fixed by design.
The Franka robot successfully reaching one of two handles based on its initial position.

Summary updated

Abstract

Dynamical-system-based motion policies provide a framework for learning reactive robot motions with stability and convergence properties. Many manipulation tasks, however, admit several distinct goals or a continuous set of acceptable outcomes, requiring motion primitives that preserve these alternatives. Accommodating both goal structures within a unified framework with continuous, autonomous dynamics remains an open challenge. We propose a CONDOR-based framework, multi-CONDOR, for learning motion primitives with either multiple discrete attractors or a continuous attracting goal contour. The approach combines imitation learning with losses that encourage agreement between the learned task dynamics and latent reference systems encoding the desired attraction structure. We provide a Lyapunov-based analysis of the convergence properties of the learned motion dynamics. We evaluate the framework on simulated benchmarks with discrete goals, qualitative examples with continuous goal contours, and robotic grasping experiments. In simulation, multi-CONDOR reduces trajectory RMSE by approximately 72% compared with separately learned single-goal motion primitives selected by a state machine. For tasks with two and three goals, respectively, it reduces the fraction of evaluated initial states ending outside the goal tolerance from StableNODE's 0.31% and 7.29% to 0% in both settings. Experiments with a Franka Emika robot arm demonstrate successful grasping of a container by either of its two handles using motions learned from demonstrations. Together, these results show multi-CONDOR combines motion reproduction with the flexibility to converge to alternative valid outcomes.
BibTeX
@misc{marzo2026multi,
  author = {Gabriele Giuseppe Di Marzo and Mariano Ramírez Montero and Maximilian Stölzle and Rodrigo Pérez-Dattari and Cosimo Della Santina},
  title = {{Multi-CONDOR: Learning Multistable Motion Primitives from Demonstration}},
  year = {2026},
  eprint = {2611.21059},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21059},
}
Figure