Human-Robot Interaction with Robust Prediction of Movement Intention Surpasses Manual Control

Sebastijan Veselič, Claudio Zito, Dario Farina

Abstract

Designing robotic assistance devices for manipulation tasks is challenging. This work aims at improving accuracy and usability of physical human-robot interaction (pHRI) where a user interacts with a physical robotic device (e.g., a human operated manipulator or exoskeleton) by transmitting signals which need to be interpreted by the machine. Typically these signals are used as an open-loop control, but this approach has several limitations such as low take-up and high cognitive burden for the user. In contrast, a control framework is proposed that can respond robustly and efficiently to intentions of a user by reacting proactively to their commands. The key insight is to include context- and user-awareness in the controller, improving decision making on how to assist the user. Context-awareness is achieved by creating a set of candidate grasp targets and reach-to grasp trajectories in a cluttered scene. User-awareness is implemented as a linear time-variant feedback controller (TV-LQR) over the generated trajectories to facilitate the motion towards the most likely intention of a user. The system also dynamically recovers from incorrect predictions. Experimental results in a virtual environment of two degrees of freedom control show the capability of this approach to outperform manual control. By robustly predicting the user’s intention, the proposed controller allows the subject to achieve superhuman performance in terms of accuracy and thereby usability.

Introduction

In traditional physical human-robot interaction (pHRI), such as operating robotic arms for hazardous waste disposal or using powered exoskeletons, control is typically "open-loop." The human provides a signal, and the machine executes it directly. This paradigm suffers from high cognitive burden, as users must constantly correct for noise and precision errors, and low accuracy due to the inherent noise in human motor control.

To address these issues, this research proposes a proactive control framework that understands both the environment (context) and the user's likely goal (intention) to provide smooth, intuitive assistance.

Conceptual overview of the human-robot interaction loop, highlighting the arbitration between human intent and robot feedback.

The "Predict-then-Blend" Framework

The proposed system is built on two pillars: context-awareness and user-awareness.

Context and User Awareness

Context-awareness is achieved by identifying potential grasp targets in a scene and pre-calculating optimal trajectories to reach them. User-awareness is implemented via a Time-Variant Linear Quadratic Regulator (TV-LQR). Unlike standard LQR, which uses a fixed rule, the TV-LQR adapts its feedback based on the user's current state along a specific trajectory, effectively acting as a "denoising" filter for human motion.

Arbitration Process

The final control signal ($u^*$) is a weighted combination of the user's raw input ($u_i$) and the controller's corrective feedback ($Kx$). A key parameter, $\alpha$, acts as a "confidence dial." If the user's movement aligns with a predicted trajectory, the system provides strong assistance. If the user deviates—indicating a change of intention—the system reduces its influence, allowing the user to steer toward a different target.

The arbitration framework blending human intention ($u_i$) with robot feedback control ($Kx$) to produce the final control signal ($u^*$).

Experimental Evaluation

The framework was tested in a 2D virtual simulation environment with 6 healthy subjects. The study included "characterization runs" to optimize parameters like $\alpha$ and "switch scenarios" where subjects were required to change targets mid-movement to test the system's robustness.

Key Results and Insights

  • Superhuman Performance: The assisted system allowed users to achieve significantly higher accuracy than manual control. In trials with 200 waypoints, the assisted condition reduced the Mean Euclidean Distance error from 27.48 (manual) to 24.26.
  • Robustness to Intention Shifts: Even when users changed targets mid-way, the assisted controller outperformed manual control (24.33 vs. 27.26 error). The TV-LQR can dynamically "hop" between candidate goals.
  • Consistency and Usability: Improvements were observed across all participants. The continuous blending approach ensures that assistance feels intuitive rather than restrictive, avoiding jarring transitions between manual and autonomous modes.

Conclusion and Future Work

This work demonstrates that robust intention prediction can lead to "superhuman" performance in pHRI. Future research will focus on extending this framework to 3D environments with complex physics, automating context generation using onboard sensors (e.g., RGB-D cameras), and personalizing the controller to individual user noise profiles.

Summary updated

Abstract

Designing robotic assistance devices for manipulation tasks is challenging. This work aims at improving accuracy and usability of physical human-robot interaction (pHRI) where a user interacts with a physical robotic device (e.g., a human operated manipulator or exoskeleton) by transmitting signals which need to be interpreted by the machine. Typically these signals are used as an open-loop control, but this approach has several limitations such as low take-up and high cognitive burden for the user. In contrast, a control framework is proposed that can respond robustly and efficiently to intentions of a user by reacting proactively to their commands. The key insight is to include context- and user-awareness in the controller, improving decision making on how to assist the user. Context-awareness is achieved by creating a set of candidate grasp targets and reach-to grasp trajectories in a cluttered scene. User-awareness is implemented as a linear time-variant feedback controller (TV-LQR) over the generated trajectories to facilitate the motion towards the most likely intention of a user. The system also dynamically recovers from incorrect predictions. Experimental results in a virtual environment of two degrees of freedom control show the capability of this approach to outperform manual control. By robustly predicting the user’s intention, the proposed controller allows the subject to achieve superhuman performance in terms of accuracy and thereby usability.
BibTeX
@inproceedings{veselic2021human,
  author = {Sebastijan Veselič and Claudio Zito and Dario Farina},
  title = {{Human-Robot Interaction with Robust Prediction of Movement Intention Surpasses Manual Control}},
  booktitle = {Frontiers in Neurorobotics},
  year = {2021},
  doi = {10.3389/fnbot.2021.695022},
}
Figure