Feedback Strategies for Telemanipulation with Shared Control of Object Handling Forces

Weston B. Griffin, William R. Provancher, Mark R. CutkoskyPRESENCE 2005 · DOI

TL;DR

This paper explores "shared control" for telemanipulation, where a robot can assist a human operator by taking over fine force control of an object. Experiments show that this shared control, combined with specific feedback strategies (like audio alarms and visual cues), significantly improves delicate object handling and reduces dropped objects compared to traditional teleoperation.

What Problem Does This Paper Address?

Telemanipulation, especially for delicate tasks like handling fragile objects, presents a challenge: human operators need precise control over forces and motions, but factors like time delays and imperfect haptic feedback can lead to errors like dropping objects or applying too much force. Traditional "bilateral control" gives the operator direct force feedback, but can be susceptible to these issues. "Supervisory control" allows the robot to automate tasks, but can reduce the operator's sense of presence and direct control. This paper addresses the gap by investigating "shared control," a middle ground where the robot can intelligently assist with low-level force regulation while the human operator retains high-level command and direct sensory access. The core problem is determining how to best implement this shared control, specifically focusing on the types and combinations of feedback that optimize performance and operator experience.

Methods

The researchers developed a shared control telemanipulation system featuring a master interface (instrumented glove, force feedback exoskeleton) and a slave system (industrial robot with a dexterous two-fingered hand). The core innovation lies in the shared control framework itself, which allows the robot to intervene and regulate the internal grasp force when it detects a risk of dropping the object. This intervention is based on estimating the minimum force required to prevent slippage.

They designed an experiment with seven different conditions (cases) varying the presence and type of feedback:

  • Direct bilateral control (baseline): Operator controls grasp force, receives direct force feedback.
  • Robot-assisted control: Robot intervenes to maintain a minimum grasp force (110% of minimum required) if the operator's command drops too low.
  • Feedback variations: Different combinations of:
  • Audio alarms: High-frequency for slip risk, low-frequency for excessive force.
  • Visual indicators: LEDs on the robot hand to signal intervention.
  • Force feedback: Direct feedback of actual grasp force vs. feedback proportional to the operator's commanded force.

The experiment involved 11 subjects performing a pick-carry-and-place task with a fragile object. Performance was evaluated objectively (measured internal force, number of failures) and subjectively (preference rankings).

Datasets

Not discussed in the paper. The experiment used a custom-built telemanipulation system and a 0.2 kg wood block as the object.

Key Insights and Ideas

  • Shared control is beneficial: The robot's ability to intervene and regulate grasp force significantly reduces the measured internal force applied to the object and the number of dropped objects.
  • Feedback is crucial for effective shared control: Simply having the robot intervene isn't enough; how the operator is informed about the intervention and potential risks dramatically impacts performance.
  • Audio alarms are powerful: High-frequency audio alarms indicating slip risk were particularly effective in reducing failures, especially when combined with robot intervention.
  • Informing the operator matters: Cases where the operator was explicitly informed about robot intervention (via LEDs and alarms) led to better performance than cases where intervention was silent.
  • Commanded force feedback is better than actual force feedback during intervention: When the robot was actively controlling the grasp force, feeding back a signal proportional to the operator's desired force (rather than the robot's actual force) led to fewer dropped objects, suggesting it provides a more intuitive cue for manipulation.

Key Results

  • Reduced Internal Force: Cases with robot intervention and informative feedback (Cases 4, 6, and 7) showed a statistically significant reduction in the average measured internal force (around 15%) compared to the baseline bilateral control (Case 1).
  • Fewer Failures: Case 6 (robot intervention with audio alarms, LEDs, and reduced force feedback) had the lowest number of failures (dropped objects). Case 7, identical to Case 6 but without audio alarms, had four times more failures, highlighting the importance of auditory cues.
  • No significant impact on task completion time: While performance improved in terms of force and failures, task completion times were not statistically different across the cases, suggesting the added complexity of shared control didn't slow operators down.
  • Subjective Preference: Cases 6 and 7 were most preferred by subjects, indicating a positive user experience with these shared control strategies. Cases 1 and 3 (baseline and silent intervention) were the least preferred.

Limitations and Future Work

  • Learning and Fatigue: The authors acknowledge the potential for learning and fatigue effects but conclude, based on their analysis of model residuals, that these were not significant in their study. However, a more in-depth investigation could be warranted.
  • Generalizability: The study focused on a specific pick-carry-and-place task with a simple object. Future work could explore more complex manipulation tasks and object geometries.
  • Sense of Presence: While the paper mentions the concern that shared control might reduce the operator's sense of presence, this was not directly measured or quantified in the study. Future research could explicitly investigate this aspect.
  • Optimizing Feedback: The paper identifies effective feedback strategies, but further research could explore even finer tuning of alarm thresholds, audio characteristics, and visual cues for optimal performance and minimal distraction.

Summary updated

Abstract

Shared control represents a middle ground between supervisory control and traditional bilateral control in which the remote system can exert control over some aspects of the task while the human operator maintains access to low-level forces and motions. In the case of dexterous telemanipulation, a natural approach is to share control of the object handling forces while giving the human operator direct access to remote tactile and force information at the slave fingertips. We describe a set of experiments designed to determine whether shared control can improve the ability of an operator to handle objects delicately and to determine what combinations of force, visual, and audio feedback provide the best level of performance and operator sense of presence. The results demonstrate the benefits of shared control and the need to carefully choose the types and methods of direct and indirect feedback.
BibTeX
@inproceedings{griffin2005feedback,
  author = {Weston B. Griffin and William R. Provancher and Mark R. Cutkosky},
  title = {{Feedback Strategies for Telemanipulation with Shared Control of Object Handling Forces}},
  booktitle = {PRESENCE},
  year = {2005},
  doi = {10.1162/105474605775196634},
}
Figure