Solving ambiguities with perspective taking

Raquel Ros, Emrah Akin Sisbot, Rachid Alami, Jasmin Steinwender, Katharina Hamann, Felix Warneken

TL;DR

This paper tackles the challenge of human-robot interaction by developing a strategy for robots to resolve ambiguous requests from humans. By combining visual perspective taking, spatial perspective taking, and feature-based descriptions, the proposed system can autonomously clarify human intentions, leading to more natural and less tedious interactions.

What Problem Does This Paper Address?

Human-robot interaction (HRI) often falters due to ambiguities in human communication. Humans naturally generate and resolve these ambiguities in everyday interactions, but robots struggle with this. When a robot cannot understand a human's request, it often has to repeatedly ask for clarification, which is frustrating and inefficient for the human. This paper addresses the need for robots to proactively and intelligently resolve these ambiguities to enable smoother and more acceptable HRI. The core problem is how a robot can infer the correct referent when a human's request is underspecified, especially when the ambiguity arises from differing perspectives.

Methods

The authors propose a clarification strategy that combines three distinct mechanisms:

  • Visual Perspective Taking (VPT): This involves the robot understanding what objects are visible to the human by projecting the 3D environment onto a 2D plane and accounting for occlusions. An object is considered visible if its projected visibility ratio exceeds a certain threshold.
  • Spatial Perspective Taking (SPT): This mechanism allows the robot to compute object locations relative to different reference frames, specifically the robot's own and the human's. The space around an object is divided into regions to model proximity.
  • Feature Based Description (FBD): This technique leverages object attributes (color, size, shape, dimensions, class) to differentiate between objects. The robot maintains a database of object descriptions and searches for a feature that has a unique value for each potential referent.

These techniques are integrated into a decision tree that first uses VPT to eliminate occluded objects, then employs SPT and FBD to find a discriminative feature. If ambiguity persists, the human provides further clues, and the process repeats.

Datasets

Not discussed in the paper. The authors mention designing six different scenarios for testing, implying a custom-built set of test cases rather than standard datasets.

Key Insights and Ideas

  • Proactive Ambiguity Resolution is Crucial for HRI: The paper strongly emphasizes that robots must be able to handle ambiguities internally rather than relying solely on human clarification to achieve fluent interaction.
  • Multi-faceted Approach to Ambiguity: Combining visual, spatial, and feature-based reasoning provides a more robust solution than relying on a single method, as different ambiguities may be best resolved by different techniques.
  • Inferring Human Perspective is Key: The ability to understand what the human can and cannot see (VPT) is fundamental to resolving many spatial ambiguities.
  • The "Invisible" Object Problem: The paper highlights scenarios where an object might be visible to the robot but occluded from the human's view, requiring the robot to infer that the human cannot be referring to that object.

Key Results

The proposed clarification strategy was tested in six custom-designed scenarios, both in simulation and with the HRP-2 robot. The system successfully resolved ambiguities in all tested situations. Two particularly challenging scenarios, involving multiple sources of ambiguity, were tested on the HRP-2 robot, and the system's components performed accurately, leading to successful ambiguity resolution.

Limitations and Future Work

The paper acknowledges that if there are no candidates left after the clarification process, the robot cannot solve the ambiguity. Future work could explore more sophisticated ways to handle situations where the robot is unable to resolve the ambiguity, perhaps by asking more targeted questions or learning from past interactions. The current system relies on a predefined set of scenarios and object descriptions, and extending this to more dynamic and complex environments would be a valuable next step.

Summary updated

Abstract

Humans constantly generate and solve ambiguities while interacting with each other in their every day activities. Hence, having a robot that is able to solve ambiguous situations is essential if we aim at achieving a fluent and acceptable human-robot interaction. We propose a strategy that combines three mechanisms to clarify ambiguous situations generated by the human partner. We implemented our approach and successfully performed validation tests in several different situations both, in simulation and with the HRP-2 robot.
BibTeX
@misc{ros2010solving,
  author = {Raquel Ros and Emrah Akin Sisbot and Rachid Alami and Jasmin Steinwender and Katharina Hamann and Felix Warneken},
  title = {{Solving ambiguities with perspective taking}},
  year = {2010},
  doi = {10.1109/hri.2010.5453204},
}
Figure