Raising Body Ownership in End-to-End Visuomotor Policy Learning via Robot-Centric Pooling

Zheyu Zhuang, Ville Kyrki, Danica Kragic

Abstract

In end-to-end visuomotor learning, robots often struggle to distinguish their own physical components from external objects or similar-looking distractors. This lack of "body ownership" leads to brittle policies that fail in the presence of visual ambiguity or camera shifts. This paper introduces Robot-centric Pooling (RcP), a novel mechanism that utilizes internal proprioceptive states to filter visual inputs. By aligning what the robot "feels" with what it "sees," RcP enables robust policy learning that ignores distractors and maintains performance across spatial perturbations. Experimental results in both simulation and real-world environments demonstrate that RcP significantly outperforms standard pooling methods, achieving a 92% success rate in distractor-heavy scenarios where baselines fall below 15%.

1. Introduction

Standard end-to-end visuomotor policies map raw pixels directly to motor commands. While powerful, these models frequently lack a fundamental cognitive trait: the ability to differentiate the self from the environment. In complex scenes—such as those containing multiple robot arms or dynamic backgrounds—traditional pooling layers like Spatial-Softmax rely on visual saliency alone. This often results in the policy inadvertently "attaching" its logic to external objects or "self-distractors" (other robots of the same model). When the camera angle shifts or a distractor enters the frame, the model's spatial bias is disrupted, leading to catastrophic failure. To address this, we propose Robot-centric Pooling (RcP), which treats the robot's proprioception as a primary filter for visual perception.

2. Methodology

2.1 Robot-centric Pooling (RcP)

RcP is designed as a drop-in replacement for standard pooling layers in convolutional neural network (CNN) architectures. The core innovation is the Image-Proprioception Alignment (IPA) mechanism. Unlike traditional methods that seek the most visually prominent features, IPA uses the robot’s proprioceptive state—specifically joint angles and end-effector poses—as a query in a cross-attention framework.

In this setup, image features extracted from the CNN serve as keys and values. The model calculates alignment scores between the proprioceptive query and the visual keys, effectively asking: Which pixels in this image correlate with my current physical configuration? This process generates a feature map gated by the robot's internal state, ensuring the policy only processes information relevant to its own body.

2.2 Self-Supervised Contrastive Learning

To train the RcP layer without the need for manual pixel-level labeling, we employ a self-supervised approach inspired by Momentum Contrast (MoCo). We utilize a "cut-and-paste" data augmentation pipeline where the robot's arm is cropped from training images and overlaid onto diverse, random backgrounds. By knowing which pixels represent the robot during this augmentation, the encoder learns to recognize the robot as a distinct entity. This allows the system to develop a robust concept of "self" that is invariant to environmental changes.

2.3 Policy Integration

The resulting robot-centric feature map is passed to a standard policy regressor. Because the pooling mechanism is inherently tied to the robot's internal state, the policy naturally ignores visual features that do not move in synchronization with the robot’s proprioception.

3. Experimental Setup

3.1 Simulation

We evaluated RcP in a reaching task using a UR5 robot in the CoppeliaSim environment. The robot was tasked with reaching target cans while surrounded by various distractors, including household objects (bananas, mugs) and "self-distractors" (additional UR5 arms). A dataset of 3,000 trajectories was used for training.

3.2 Real-World Implementation

Real-world validation was conducted using a Franka Panda robot. The setup included a movable divider that could reveal a second, identical distractor robot. We collected 200 real-world trajectories, which were augmented with 1,000 simulated trajectories for fine-tuning the model.

4. Results and Discussion

4.1 Robustness to Distractors

In real-world tests featuring a self-distractor, RcP maintained a high success rate of 92%. In contrast, baseline models such as Spatial-Softmax and R3M saw performance plummet to below 15%. Saliency map visualizations confirmed that RcP consistently focused on the ego-robot, while baselines often split their attention between the robot and the distractor, frequently attempting to "control" the wrong arm.

4.2 Resilience to Camera Shifts

RcP demonstrated significant resilience to camera movement. When input images were artificially shifted to simulate a bumped camera, RcP showed a graceful decline in performance. Conversely, baseline models typically failed entirely after a shift of only 7% of the image width. This suggests that by being self-referential rather than relying on absolute spatial coordinates, RcP effectively breaks the spatial bias common in end-to-end models.

5. Limitations and Future Work

While effective, the current implementation of RcP is limited by its 7x7 feature map resolution. In scenarios where the robot and a distractor overlap significantly in the 2D camera view, this coarse resolution can lead to disentanglement difficulties. Furthermore, this work focuses primarily on spatial alignment. Future research could incorporate temporal alignment—matching pixel velocity and acceleration to the robot's movements—to further solidify the sense of body ownership. Finally, expanding the task scope beyond reaching to complex, long-horizon manipulation remains a key objective.

6. Conclusion

Robot-centric Pooling provides a scalable and effective method for instilling a sense of body ownership in visuomotor policies. By leveraging proprioception as a visual filter, RcP allows robots to remain focused on their own physical state, ensuring robust performance in dynamic, cluttered, and unpredictable environments.

Summary updated

Abstract

We present Robot-centric Pooling (RcP), a novel pooling method designed to enhance end-to-end visuomotor policies by enabling differentiation between the robots and similar entities or their surroundings. Given an image-proprioception pair, RcP guides the aggregation of image features by highlighting image regions correlating with the robot's proprioceptive states, thereby extracting robot-centric image representations for policy learning. Leveraging contrastive learning techniques, RcP integrates seamlessly with existing visuomotor policy learning frameworks and is trained jointly with the policy using the same dataset, requiring no extra data collection involving self-distractors. We evaluate the proposed method with reaching tasks in both simulated and real-world settings. The results demonstrate that RcP significantly enhances the policies' robustness against various unseen distractors, including self-distractors, positioned at different locations. Additionally, the inherent robot-centric characteristic of RcP enables the learnt policy to be far more resilient to aggressive pixel shifts compared to the baselines.
BibTeX
@inproceedings{zhuang2024raising,
  author = {Zheyu Zhuang and Ville Kyrki and Danica Kragic},
  title = {{Raising Body Ownership in End-to-End Visuomotor Policy Learning via Robot-Centric Pooling}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2024},
  doi = {10.1109/iros58592.2024.10802462},
}
Figure