A Learning-based Adaptive Compliance Method for Symmetric Bi-manual Manipulation

Yuxue Cao, Shengjie Wang, Xiang Zheng, Wenke Ma, Tao ZhangarXiv 2023 · OpenReview

TL;DR

This paper introduces a novel Learning-based Adaptive Compliance (LAC) algorithm that unifies motion planning and adaptive impedance control for symmetric bi-manual manipulation. By using a centralized Actor-Critic reinforcement learning framework with LSTM networks to process force states, LAC significantly improves efficiency, robustness, and synchronization in complex tasks like dual-arm peg-in-hole assembly, outperforming existing methods in both simulation and real-world experiments.

What Problem Does This Paper Address?

On-orbit operations, such as assembling large space structures or maintaining spacecraft, heavily rely on symmetric bi-manual manipulation due to its inherent stability and load capacity. However, traditional approaches often treat motion planning and compliant control as separate, sequential modules. This separation can lead to conflicts when desired trajectories and impedance parameters need to change simultaneously in response to external forces or disturbances. Furthermore, these traditional methods often require manual tuning of parameters by experts, limiting their adaptability and efficiency. The paper aims to bridge this gap by developing a unified framework that can adaptively adjust both motion planning and compliance control in real-time, enhancing performance and reducing reliance on manual parameter tuning.

Methods

The core of the proposed solution is the Learning-based Adaptive Compliance (LAC) algorithm, built upon a reinforcement learning (RL) foundation. Specifically, it employs the Soft Actor-Critic (SAC) algorithm, known for its ability to balance reward maximization with policy entropy, promoting better exploration.

Key methodological innovations include:

  • Unified Framework: LAC integrates the generation of desired trajectories and the adjustment of impedance parameters within a single RL framework. This addresses the contradictions that arise when these two aspects are handled separately.
  • Centralized Actor-Critic with LSTM: A centralized Actor-Critic architecture is used to manage the synchronized control of dual arms. Crucially, Long Short-Term Memory (LSTM) networks are employed to preprocess the force states. This is important because force states are highly volatile and can benefit from temporal context to predict trends, leading to better synchronization and compliance.
  • Adaptive Impedance Control: The algorithm adaptively tunes impedance parameters (damping and stiffness) in real-time. The desired inertia matrix is kept constant for stability.
  • Hybrid Policy for Peg-in-Hole: For the specific peg-in-hole task, a hybrid policy is introduced, combining the RL output with a traditional PD controller's output for the object's desired trajectory. This hybrid approach accelerates the learning process, especially before contact is made.

Datasets

The paper does not explicitly mention or introduce specific datasets for training. Instead, it relies on simulation environments and real-world experiments for training and evaluation. The primary benchmark task is the dual-arm cooperative peg-in-hole assembly.

Key Insights and Ideas

  • Unifying Planning and Control is Crucial: The paper strongly suggests that treating motion planning and compliant control as a single, integrated problem is essential for achieving high performance in complex manipulation tasks, especially those involving contact.
  • LSTM for Force Trend Prediction: Preprocessing force states with LSTMs is a significant contribution. It allows the RL agent to learn from the trend of forces, not just instantaneous values, leading to more proactive and robust compliance.
  • Centralization for Synchronization: For tasks requiring tight coordination between multiple manipulators (like symmetric bi-manual manipulation), a centralized RL approach is more effective than decentralized methods.
  • Adaptive Impedance Outperforms Fixed Impedance: The ability to dynamically adjust impedance parameters is shown to be critical for handling varying contact forces and ensuring successful assembly, especially in the presence of uncertainties.

Key Results

The LAC algorithm demonstrated superior performance in dual-arm cooperative peg-in-hole assembly experiments, both in simulation and on a physical robot:

  • Improved Reward: In simulation, LAC achieved a more stable and higher average reward during training compared to baseline algorithms like TD3, SAC, LAC w/ fixed Imp, and LAC w/o LSTM.
  • Robustness to Errors: When tested with a random error in the hole's position, LAC achieved the highest success rate (18 out of 20 attempts) compared to LAC w/ fixed Imp (13/20) and LAC w/o LSTM (14/20).
  • Adaptability to Stiffness/Damping: LAC showed excellent adaptability in environments with varying stiffness and damping, successfully completing tasks where LAC w/ fixed Imp failed due to its inability to adjust impedance.
  • Real-World Success: In real-world experiments, LAC achieved a high success rate (17 out of 20 attempts) with an average assembly time of 7.05 seconds and a maximum contact force of 19.718N. The results closely mirrored simulation outcomes, indicating good generalization.
  • Reduced Contact Force: LAC effectively minimized environmental contact forces, keeping them within safe ranges during insertion.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that the desired inertia matrix is kept constant for stability, implying that adapting inertia might be a future avenue.
  • While successful, the algorithm's performance is evaluated on a specific task (peg-in-hole). Generalizing to a wider range of symmetric bi-manual manipulation tasks might require further investigation.
  • The reliance on a pre-trained policy for real-world experiments suggests that online adaptation or further fine-tuning in the real world could be challenging.
  • Future Work:
    • The authors suggest exploring the adaptation of inertia parameters.
  • Extending the LAC algorithm to a broader spectrum of symmetric bi-manual manipulation tasks is a natural next step.
  • Investigating more complex scenarios with dynamic environments or uncertain object properties could further validate the algorithm's robustness.

Summary updated

Abstract

Symmetric bi-manual manipulation is an essential skill in on-orbit operations due to its potent load capacity. Previous works have applied compliant control to maintain the stability of manipulations. However, traditional methods have viewed motion planning and compliant control as two separate modules, which can lead to conflicts with the simultaneous change of the desired trajectory and impedance parameters in the presence of external forces and disturbances. Additionally, the joint usage of these two modules requires experts to manually adjust parameters. To achieve high efficiency while enhancing adaptability, we propose a novel Learning-based Adaptive Compliance algorithm (LAC) that improves the efficiency and robustness of symmetric bi-manual manipulation. Specifically, the algorithm framework integrates desired trajectory generation and impedance-parameter adjustment under a unified framework to mitigate contradictions and improve efficiency. Second, we introduce a centralized Actor-Critic framework with LSTM networks preprocessing the force states, enhancing the synchronization of bi-manual manipulation. When evaluated in dual-arm peg-in-hole assembly experiments, our method outperforms baseline algorithms in terms of optimality and robustness.
BibTeX
@misc{cao2023learning,
  author = {Yuxue Cao and Shengjie Wang and Xiang Zheng and Wenke Ma and Tao Zhang},
  title = {{A Learning-based Adaptive Compliance Method for Symmetric Bi-manual Manipulation}},
  year = {2023},
  eprint = {2303.15262},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2303.15262},
}
Figure