Improving safety in physical human-robot collaboration via deep metric learning

Maryam Rezayati, Grammatiki Zanni, Ying Zaoshi, Davide Scaramuzza, Hans Wernher van de Venn

TL;DR

Researchers have developed a Deep Metric Learning (DML) approach that allows robots to distinguish between intentional human touch and accidental collisions with 98.6% accuracy. By mapping robot sensor data into a specialized "similarity space," the system achieves a 4% performance boost over standard deep learning methods and can generalize to new robot hardware without needing to be retrained.

What Problem Does This Paper Address?

In modern "flexible" factories, the physical barriers between humans and robots are disappearing. However, current safety standards are often blunt: if a robot senses any contact, it simply stops. While safe, this "stop-on-impact" approach kills productivity and prevents true collaboration, such as a human physically guiding a robot arm to a precise location.

The challenge lies in contact perception. A robot needs to use its internal sensors (proprioception) to determine if a touch is:

  1. Non-contact: Just the robot moving through the air.
  2. Intentional contact: A human operator guiding the robot.
  3. Incidental contact: An accidental collision that requires an emergency stop.

Traditional model-based methods require perfect mathematical descriptions of robot physics, which are difficult to obtain. Standard data-driven AI models often struggle to generalize when moved from one specific robot unit to another due to subtle differences in sensor noise and calibration.

Methods

The authors propose using Deep Metric Learning (DML). Unlike standard classification, which tries to map inputs directly to labels, DML learns an "embedding space." In this space, similar types of contact are forced close together, while different types are pushed far apart.

The pipeline consists of two parts:

  1. Embedding Network: Takes a 140ms window of robot data (joint torques, position errors, and velocity errors) and maps it to a 64-dimensional vector.
  2. Classifier: A simple network that looks at that vector and decides the contact type.

The researchers compared three different DML loss functions:

  • Pairwise (Siamese): Compares pairs of samples to see if they belong to the same class.
  • Triplet Loss: Compares an "anchor" sample against a "positive" (same class) and a "negative" (different class) simultaneously.
  • Magnet Loss: Sculpting the representation space by identifying clusters and respecting intra-class variations.

They tested these across three architectures: a 2D-CNN, a 1D-CNN, and a standard Multi-Layer Perceptron (FaceNet).

Datasets

The study used a publicly available collision detection dataset collected from a Franka Emika Panda robot arm.

  • Scenario: A real-world pick-and-place task involving pen components.
  • Data Points: 2,146 samples.
  • Features: 28 features (7 joints × 4 sensor types: torque, external torque, position error, velocity error).
  • Labels: Non-contact, Intentional (Link 5 or 6), and Incidental (Link 5 or 6).

Key Insights and Ideas

  • Explainability through Distance: By using DML, the "black box" of AI becomes slightly more transparent. You can visualize the embedding space (using PCA) and literally see clusters of "collisions" separating from "intentional touches."
  • Generalization is the "Killer App": One of the biggest headaches in robotics is that a model trained on "Robot A" often fails on "Robot B" due to hardware variance. This paper shows that DML creates a robust enough feature representation that the model works on a "target robot" without any retraining.
  • Speed Matters: The system achieves a detection delay of only 50ms. For context, human reaction time to touch is roughly 155ms. This means the robot can "realize" it has been hit and react faster than the human who hit it.

Key Results

  • Top Accuracy: The Conv1DNet trained with Triplet Loss was the winner, achieving 98.6% accuracy.
  • Baseline Comparison: This was a 4% improvement over a standard deep learning baseline (trained with sparse categorical cross-entropy).
  • Robustness: The Triplet Loss model showed significantly fewer false positives and false negatives compared to the Pairwise approach.
  • Transferability: When deployed on a second "target" robot, the system maintained high performance in detecting contact, though distinguishing between types of contact (intentional vs. accidental) on new hardware remains a slightly harder challenge.

Limitations and Future Work

  • Speed Constraints: The study only investigated robot joint velocities under 0.7 rad/s. High-speed industrial movements might introduce different vibration patterns that could confuse the model.
  • Hardware Diversity: While the model transferred between two robots of the same type, it hasn't been tested across different brands or configurations (e.g., moving from a Panda to a UR5).
  • Clustering Algorithms: The authors suggest that future work could explore more advanced clustering techniques like DBSCAN to further refine the embedding space.
  • Domain Adaptation: To improve the "target robot" accuracy for specific contact types, the authors plan to investigate domain adaptation techniques to bridge the gap between different hardware units.

Summary updated

BibTeX
@inproceedings{rezayati2022improving,
  author = {Maryam Rezayati and Grammatiki Zanni and Ying Zaoshi and Davide Scaramuzza and Hans Wernher van de Venn},
  title = {{Improving safety in physical human-robot collaboration via deep metric learning}},
  booktitle = {ETFA},
  year = {2022},
  doi = {10.1109/etfa52439.2022.9921623},
}
Figure