Object Centric Trajectory Warping

30 concepts27 papers1 proposal

When a human demonstrates a manipulation task—like pouring a glass of water or tightening a bolt with a wrench—the core of the skill is not the absolute path of their hand in 3D space. Instead, the skill is defined by the relative motion between the tool and the target object. If the target object moves ten centimeters to the left, a rigid replay of the absolute coordinates fails completely.

To bridge this gap, a powerful paradigm has emerged: object-centric trajectory warping. Instead of training a neural network to learn a complex mapping from raw pixels to motor commands across millions of variations, we can mathematically adapt a single reference trajectory to fit new spatial configurations. By shifting our perspective from global coordinates to local, object-aligned coordinate frames, we can achieve robust generalization with minimal data.


The Core Mechanism: Coordinate Frame Transformations

Why does trajectory warping work so effectively? At its simplest level, robotic manipulation tasks obey geometric symmetries. If an object's initial or goal pose changes, the entire end-effector path can be mapped to the new configuration using rigid-body transformations (rotations and translations in 3D space).

The system maps the end-effector path from a source coordinate frame to a target frame. For instance, MimicGen demonstrates how a small set of human demonstrations can be scaled up by transforming reference robot trajectories to align with the pose of target objects in a new environment, allowing a single demonstration to be repurposed across diverse spatial arrangements.

This geometric insight allows researchers to frame imitation learning as a spatial alignment problem rather than a complex behavioral learning problem. Vitiello in One-Shot Imitation Learning: A Pose Estimation Perspective re-frames one-shot imitation learning as a geometric pose estimation problem. By calculating the rigid transformation between an object's pose in a demonstration and its pose in a deployment scene, the robot can directly transfer the recorded trajectory to a new location without any retraining.

[Global Frame] ──(Identify Object Pose)──> [Local Object Frame]
                                                   │
                                        (Apply Rigid Transformation)
                                                   │
[Warped Trajectory] <──────────────────────────────┘

This concept extends naturally to video-based imitation. Li et al. introduce OKAMI, which processes a single video of a human demonstration and transforms the motion trajectory from a global coordinate system to a local frame defined by the target object. By focusing on the object's coordinate frame rather than absolute coordinates, the robot can execute the functional movement even when the environment changes. This insight is shared by DITTO, which focuses on the trajectory of the manipulated object itself, warping the recorded motions to fit new scene layouts.


Handling Variations in Grasps and Geometries

The real world presents challenges beyond simple translations of identical objects. What happens if the robot grasps a tool differently, or if the target object belongs to a completely novel category with a different shape? Trajectory warping handles these variations by calculating corrective or correspondence-driven transformations.

When a robot holds a tool at an unusual angle, Papagiannis et al. show in Adapting Skills to Novel Grasps: A Self-Supervised Approach that the system can calculate a corrective transformation and apply it to the entire pre-defined end-effector trajectory. This preserves the functional path of the tool relative to the environment, regardless of the novel grasp. When the tool itself changes, the framework presented in Modeling and Learning Constraints for Creative Tool Use adapts existing skills by incorporating sparse human corrections to align a novel tool's geometry with the task requirements.

To generalize across entirely different object categories, researchers use semantic correspondences. Liang et al. introduce Autonomous Functional Play with Correspondence-Driven Trajectory Warping, which uses vision models to identify functional keypoints between a demonstration object and a new instance. By warping the 3D trajectory based on these semantic keypoints, the robot generalizes a single demonstration to novel object categories.

Similarly, Wu et al. propose AffordDP, which maps 3D contact points and trajectories from known objects to new geometries by calculating a 6D transformation between functional parts, ensuring precise interaction with novel object instances.


Scaling Robot Learning via Counterfactual Augmentation

Beyond direct execution, trajectory warping serves as a powerful engine for data augmentation. Collecting real-world robot data is notoriously expensive. By leveraging geometric transformations, a handful of real demonstrations can be warped into thousands of valid, counterfactual training examples.

For example, Ameperosa et al. introduce RoCoDA, a framework that applies rigid body transformations to object poses while simultaneously transforming the robot's recorded actions. This counterfactual data augmentation ensures geometric consistency across diverse workspace configurations, drastically improving data efficiency.

In the context of error recovery, Hoque et al. develop IntervenGen, which treats human corrective interventions as object-centric. By warping a single recovery trajectory based on the delta between perceived and actual object coordinates, they scale a small set of interventions into a broad library of recovery maneuvers.

[Original Demo] ───(Vary Object Poses)───> [Apply Inverse Warping] ───> [Thousands of Valid Synthetic Demos]

This approach also bridges the gap between reality and simulation. Yu et al. present Real2Render2Real, adapting demonstrated trajectories to new initial and goal poses to synthesize thousands of realistic executions without physical hardware. In a similar vein, Barcellona et al. introduce Dream to Manipulate, decomposing a single real-world demonstration into object-centric assets and applying physically-verified transformations in a simulator to enable robust one-shot imitation learning.

By focusing on the geometric invariants of a task, object-centric trajectory warping bypasses the need for massive, exhaustive datasets, allowing robots to generalize skills to new positions, orientations, and geometries from just a single example.

Key Foundational Work

Accelerating Integral Histograms Using an Adaptive Approach (2008) — contributed 1 concept elements to this direction.
An Experimental System for Robotic Heart Surgery (2005) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_object_centric_trajectory_warping_696060cd,
  title = {object-centric trajectory warping},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/object-centric-trajectory-warping/rdr_696060cda29d4a14b594a7964fcb06d7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_object_centric_trajectory_warping_696060cd_v1,
  title = {object-centric trajectory warping},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/object-centric-trajectory-warping/rdr_696060cda29d4a14b594a7964fcb06d7/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake