Towards Better Robot Learners: Leveraging Implicit and Explicit Human Feedback Together in Human Robot Interactions

Kate CandonAAAI 2025 · DOI

TL;DR

This paper proposes a novel approach for robots to learn human preferences more effectively by combining explicit feedback (like button presses) with implicit cues (like eye rolls or gestures). By integrating these two feedback streams, robots can better adapt to individual users, leading to more seamless and personalized human-robot interactions.

What Problem Does This Paper Address?

Designing robots that can truly collaborate with humans requires them to understand and adapt to individual user preferences. While robots can learn from explicit feedback (e.g., "good job" or "bad job" buttons, demonstrations), relying solely on this can be burdensome for humans, interrupt the interaction flow, and is often inconsistent. Humans naturally provide a wealth of implicit feedback through non-verbal cues, but interpreting these signals is challenging due to their individual and context-dependent nature. This paper addresses the gap in effectively leveraging both explicit and implicit human feedback simultaneously to create more adaptive and user-friendly robot learning systems.

Methods

The core of this research lies in developing robot learning paradigms that integrate explicit and implicit human feedback. Specifically, the authors are investigating the use of inverse reinforcement learning (IRL) to learn a reward function that models human preferences. This reward function can then guide the robot's behavior to better satisfy those preferences. The proposed approach aims to update the robot's belief about the human's reward function by observing both explicit feedback (e.g., evaluative button presses) and implicit feedback (e.g., human actions during a task). The authors are also exploring how to recognize and leverage other human nonverbal behaviors, such as facial reactions and gestures, to further enhance this learning process.

Datasets

Not discussed in the paper. The authors mention using a simulated task environment for prototyping and plan to conduct a user study on a real-world experimental setup.

Key Insights and Ideas

  • The power of combined feedback: Explicit feedback alone is insufficient and can be taxing. Implicit feedback, while rich, is ambiguous. Combining them offers a more robust and less burdensome learning signal for robots.
  • Personalization is key: As human-robot interactions become more common, robots need to move beyond objective task completion to adapt to subjective, individual preferences.
  • Implicit cues are valuable, but tricky: Non-verbal behaviors like eye rolls or gestures can signal preferences, but their interpretation is highly individualized and context-dependent, posing a significant challenge for machine learning.
  • Inverse Reinforcement Learning for preference modeling: IRL provides a powerful framework for robots to infer underlying reward functions (i.e., user preferences) from observed feedback.

Key Results

The paper is presented as work in progress, with preliminary results expected for a workshop. The authors describe their current work as investigating how to use explicit and implicit feedback to learn a reward function for an interaction. They have a simulated environment for prototyping and plan to conduct a user study. Future work will focus on how robots can use implicit feedback to decide when to ask for explicit feedback and how implicit feedback can qualify explicit feedback.

Limitations and Future Work

Limitations: * The overall amount of feedback is still limited by the use of real humans, necessitating efficient learning strategies. * Interpreting implicit feedback is inherently challenging due to its individualized and context-dependent nature.

Future Work: * Investigate how robots can use implicit feedback to decide when to query a human for explicit feedback. * Explore how robots can use implicit feedback to qualify explicit feedback. * Develop methods for robots to make better sense of explicit and implicit feedback in conjunction. * Train models to recognize and leverage a wider range of human nonverbal behaviors during interactions. * Analyze user study data to determine the relative usefulness of facial reactions versus gestures in predicting feedback.

Summary updated

BibTeX
@inproceedings{candon2025towards,
  author = {Kate Candon},
  title = {{Towards Better Robot Learners: Leveraging Implicit and Explicit Human Feedback Together in Human Robot Interactions}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i28.35202},
}
Figure