Contact Localization for Robot Arms in Motion without Torque Sensing

Jacky Liang, Oliver Kroemer

TL;DR

This paper introduces a novel neural network approach for robots to detect and pinpoint where they are making contact with objects, even while moving and without needing specialized torque sensors. By using a unique cylindrical projection of the robot arm's surface and training with simulated data that accounts for real-world variations, they achieve high accuracy in contact localization, paving the way for more robust robotic interaction.

What Problem Does This Paper Address?

For robots to interact safely and effectively with their environment, especially in unstructured settings, they need to know precisely when and where they are touching things. Traditional methods often rely on expensive and delicate robot skins or assume the robot is stationary and only making a single contact at a time. Existing approaches using proprioceptive data (like joint angles and velocities) often require accurate robot dynamics models or torque sensing, which aren't always available or reliable. This paper tackles the challenge of localizing contacts on a moving robot arm using only readily available proprioceptive data, without needing torque sensors or precise dynamics models, and while allowing for multiple simultaneous contacts.

Methods

The core of this work is a neural network trained to predict contact locations. Key innovations include:

  • Domain Randomization: To bridge the "sim-to-real" gap, the network is trained on simulated data where the robot's inertial parameters and impedance controller gains are randomized. This makes the model more robust to variations encountered in the real world.
  • Cylindrical Projection: A novel representation is used to map the 3D surface of robot links into a 2D cylindrical coordinate system. This allows the network to efficiently process features and predict contact locations using convolutional and transposed convolutional layers, treating the robot surface like an "image."
  • Proprioceptive Features: The network uses readily available robot data such as joint velocities, and differences between current and "collision-free" joint angles and velocities. This avoids the need for torque sensors.
  • Contact Distance Fields: Instead of directly classifying contact points, the network predicts a "contact distance field" for each link. This field represents the distance to the nearest contact point, providing a smoother and denser supervision signal for training.

Datasets

The authors generated a large dataset for training and evaluation using the Nvidia Isaac Gym simulator. This dataset consists of approximately 2800 simulated trajectories where the robot arm interacts with randomly generated obstacles. The data includes various proprioceptive observations and ground truth contact information. No new benchmark datasets were introduced, but the generated data is designed to be more realistic by simulating dynamic interactions and using domain randomization.

Key Insights and Ideas

  • Moving Robots are Different: The paper argues that assuming a static robot during contact is less realistic for "active" manipulation tasks where the robot "goes" to the obstacle. Their method is designed for dynamic contact scenarios.
  • Torque Sensors Aren't Always Necessary: By leveraging proprioceptive data and domain randomization, the need for expensive and potentially fragile torque sensors is eliminated, making the approach more broadly applicable.
  • Cylindrical Projection as a Game Changer: This novel representation allows standard image processing techniques (like convolutions) to be applied to robot surfaces, enabling efficient feature extraction and prediction.
  • Distance Fields for Smoother Learning: Predicting contact distance fields, rather than direct contact points, provides a more robust training signal, especially for areas with sparse true positives.

Key Results

The trained neural network achieved:

  • Contact Detection Accuracy (ACC): 91.5%
  • Mean Contact Localization Error (AMCD): 3.0 cm
  • The proposed method (CDF) outperformed a direct classification approach (CLS) in terms of false negative rates for detection and achieved more spatially correlated contact predictions.
  • The system was successfully demonstrated in an obstacle mapping task in both simulation and the real world.

Limitations and Future Work

  • Stiff, Rigid, Stationary Obstacles: The current method assumes obstacles are stiff, rigid, and do not move. Extending this to deformable or dynamic obstacles is a natural next step.
  • Limited Obstacle Complexity: The training environments and obstacle configurations were relatively simple. More complex and varied environments would likely improve generalization.
  • Real-World Ground Truth: While the system was tested in the real world, ground truth contact localization data was not available for direct comparison. Future work could involve collecting such data.
  • Exploration Trajectories: The paper mentions both random and informed exploration policies. Further investigation into how different exploration strategies impact learning could be beneficial.

Summary updated

Abstract

Detecting and localizing contacts is essential for robot manipulators to perform contact-rich tasks in unstructured environments. While robot skins can localize contacts on the surface of robot arms, these sensors are not yet robust or easily accessible. As such, prior works have explored using proprioceptive observations, such as joint velocities and torques, to perform contact localization. Many past approaches assume the robot is static during contact incident, a single contact is made at a time, or having access to accurate dynamics models and joint torque sensing. In this work, we relax these assumptions and propose using Domain Randomization to train a neural network to localize contacts of robot arms in motion without joint torque observations. Our method uses a novel cylindrical projection encoding of the robot arm surface, which allows the network to use convolution layers to process input features and transposed convolution layers to predict contacts. The trained network achieves a contact detection accuracy of 91.5% and a mean contact localization error of 3.0cm. We further demonstrate an application of the contact localization model in an obstacle mapping task, evaluated in both simulation and the real world.
BibTeX
@inproceedings{liang2021contact,
  author = {Jacky Liang and Oliver Kroemer},
  title = {{Contact Localization for Robot Arms in Motion without Torque Sensing}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2021},
  doi = {10.1109/icra48506.2021.9562058},
}
Figure