TL;DR
EquiContact introduces a hierarchical robotic policy that achieves full $SE(3)$ vision-to-force equivariance for contact-rich manipulation tasks like peg-in-hole insertion. By anchoring a localized, compliant low-level policy onto a globally estimated reference frame, the system seamlessly generalizes to entirely unseen spatial translations and rotations using only a handful of demonstrations.
What Problem Does This Paper Address?
Standard imitation learning methods (like Action Chunking Transformers or Diffusion Policy) excel at learning complex visuomotor skills from demonstrations. However, they suffer from a major flaw: they struggle to generalize to new spatial positions or orientations (out-of-distribution configurations) without massive, diverse datasets.
For precision, contact-rich tasks—such as inserting a peg into a hole with 1mm clearance—this limitation is magnified. High-level vision planners can estimate where an object is, but their outputs are typically too noisy for sub-millimeter precision. Conversely, low-level force controllers can handle the contact dynamics but are blind to global spatial shifts. The authors bridge this gap by creating a sample-efficient framework that is mathematically guaranteed to generalize across arbitrary 3D translations and rotations.
Methods
EquiContact utilizes a "divide-and-conquer" hierarchical architecture to achieve end-to-end $SE(3)$ equivariance (meaning if the task setup rotates or translates in 3D space, the robot's movements and forces adjust accordingly):
- High-Level Vision Planner: A Diffusion Equivariant Descriptor Field (Diff-EDF) processes a global point cloud of the scene to estimate a coarse local reference frame for the target object.
- Low-Level Visuomotor Policy (G-CompACT): Once the robot moves near the target, the Geometric Compliant Action Chunking Transformer takes over. Crucially, it operates only on localized observations defined in the end-effector frame: a Geometrically Consistent Error Vector (GCEV), force-torque sensor readings, and wrist-camera RGB images. It outputs relative poses and task-adaptive admittance gains.
- Compliant Control Layer: A Geometric Admittance Control (GAC) module translates the low-level policy's outputs into smooth, force-reactive motion commands.
To ensure the wrist-camera features are invariant to global spatial shifts, the authors use language grounding. They modulate a pretrained CLIP-ResNet50 vision backbone using Feature-wise Linear Modulation (FiLM) conditioned on text prompts (e.g., "putting a peg into the hole"). This forces the visual encoder to focus on object identity and local geometry rather than the global background.
Datasets
The low-level policy (G-CompACT) was trained on a small dataset of expert demonstrations collected via teleoperation (using a SpaceMouse) on a single, fixed platform pose. The training data included 13–18 language prompts per task and incorporated arbitrary lab objects as visual distractors to prevent background overfitting. The high-level planner (Diff-EDF) was trained separately on just 20 randomized point cloud demonstrations.
Key Insights and Ideas
- Induced Equivariance: Instead of building massive, computationally expensive equivariant neural networks for the entire pipeline, EquiContact induces equivariance. By structuring the low-level policy's inputs and actions to be strictly local to the end-effector frame, the policy inherently inherits $SE(3)$ equivariance.
- Anchoring Local Policies: The core philosophy is "anchoring a localized policy on a globally estimated reference frame." This decouples macro-spatial understanding from micro-precision handling, making the system highly sample-efficient.
- Language as a Geometric Stabilizer: Language tokens describe what an object is, not where it is. Leveraging language grounding helps the vision encoder ignore global coordinate transformations and focus purely on task-relevant local features.
Key Results
- Peg-in-Hole Generalization: When evaluated on translationally and rotationally unseen (OOD) configurations, EquiContact achieved a 20/20 success rate on flat OOD platforms and 19/20 on tilted (30°) OOD platforms.
- Baseline Comparison: Standard CompACT (which defines observations in the global frame) dropped to a 0% success rate (0/10) when evaluated on the same OOD configurations.
- Multi-Task Feasibility: Beyond peg-in-hole insertion, EquiContact successfully generalized to screwing and surface wiping tasks without changing the underlying architecture.
- Extreme Transformations: The system maintained high success rates even when subjected to extreme spatial transformations, such as pitching the entire setup by 90° or 135°.
Limitations and Future Work
- Symmetry Breaking via Singularities: If a spatial transformation forces the robot arm close to a kinematic singularity, pose tracking degrades. This introduces a distributional shift that can cause the policy to fail.
- Visual Distractor Sensitivity: In extreme rotation scenarios for the surface wiping task, the visual encoder occasionally tracked background structures (like aluminum frames) instead of the lines on the whiteboard, leading to failures.
- Advanced Vision Backbones: The authors note that incorporating newer vision-language models (like SigLIP) or explicit object-centric representations (like slot attention) could improve the robustness of the left-invariant visual features in future iterations.
Summary updated