Explain yourself! Effects of Explanations in Human-Robot Interaction

Jakob Ambsdorf, Alina Munir, Yiyao Wei, Klaas Degkwitz, Harm Matthias Harms, Susanne Stannek, Kyra Ahrens, Dennis Becker, Erik Strahl, Tom Weber, Stefan Wermter

Abstract

As artificial intelligence becomes increasingly complex, the "black box" nature of machine decision-making poses a challenge for human trust and collaboration. While Explainable AI (XAI) research often focuses on algorithmic transparency, the psychological impact of these explanations on human observers remains under-explored. This study investigates how justifications for actions influence perceptions of a robot's intelligence, competence, and lifelikeness. Through a controlled experiment involving simulated NICO robots playing Ultimate Tic-Tac-Toe, we find that while explanations significantly enhance perceived animacy and anthropomorphism, they do not automatically translate to higher ratings of competence or intelligence in short-term interactions.

Introduction

In the field of Human-Robot Interaction (HRI), a common assumption is that a robot capable of explaining its logic will naturally foster greater trust and appear more competent to its human counterparts. However, empirical evidence for this "explanation effect" is surprisingly sparse. Most XAI research prioritizes the technical accuracy of explanations rather than the human receiver's psychological response. This paper addresses this gap by examining whether a robot that justifies its moves in a competitive game gains more respect and trust from an observer than a silent one.

Methodology

To test the effects of robotic explanations, researchers designed a study centered around Ultimate Tic-Tac-Toe (UTTT). Unlike standard Tic-Tac-Toe, UTTT offers a high degree of complexity, preventing human observers from immediately determining the optimal move. This complexity forces observers to rely on the robots' verbal cues to assess their skill level.

The AI and Explanation System

The study utilized two simulated NICO (Neuro-Inspired Companion) robots. Both robots were powered by the same underlying Deep Q-Network (DQN) reinforcement learning agent, which had been trained over 80,000 games to ensure a high level of play.

While the AI logic was identical for both robots, their communication styles differed. The "explainable" robot utilized a post-hoc, rule-based system to translate its internal state into one of 12 human-understandable justifications. For example, after making a move, the robot might state: "I blocked my opponent here so he won't be able to win this local board."

Study Design

The researchers recruited 92 participants, primarily with backgrounds in computer science, for an online survey. Participants were divided into three experimental conditions:

  1. XAI Red: The red robot provided explanations; the blue robot only announced its moves.
  2. XAI Blue: The blue robot provided explanations; the red robot only announced its moves.
  3. Control: Both robots only announced their moves without justification.

Participants watched video segments of the games, placed bets on which robot they believed would win, and evaluated the robots using the Godspeed series (measuring anthropomorphism, animacy, likeability, intelligence, and safety) and the Perceived Competence Scale (PCS).

Key Insights

Explanations as a Catalyst for Animacy

The most prominent finding was that robots providing explanations were perceived as significantly more "alive." The XAI robots received higher ratings for animacy ($p = 0.008$) and anthropomorphism ($p = 0.043$). By articulating the "why" behind an action, the robot transitioned in the observer's mind from a scripted machine to a more human-like entity with its own thought process.

The Competence Paradox

Contrary to the researchers' initial hypotheses, providing justifications did not significantly boost the robots' perceived intelligence or competence. Even when a robot explained a strategic maneuver, observers did not necessarily view it as a superior player compared to the silent robot. This suggests that in short-term interactions, the mere presence of an explanation is not a shortcut to establishing perceived expertise.

Mitigating Baseline Biases

An unexpected "Red Robot" bias emerged in the control group, where participants favored the red robot over the blue one (57 bets vs. 36, $p = 0.038$), despite identical behavior. This may have been due to subtle differences in the Text-to-Speech voices or simple color preference. Interestingly, when the blue robot—the baseline "underdog"—was given the ability to explain itself, its likeability improved, effectively neutralizing the bias toward the red robot. This suggests XAI can serve as a "leveler" to overcome superficial prejudices in HRI.

Discussion and Limitations

The study's results indicate that while XAI is a powerful tool for making robots appear more human-like and engaging, it does not immediately solve the problem of perceived competence. Several factors may have influenced these results:

  • Exposure Time: Participants only observed three moves per robot. It is possible that trust and perceived intelligence require a longer track record of successful, explained actions to fully manifest.
  • Online Environment: Due to the COVID-19 pandemic, the study was conducted online. Physical presence and the resulting social cues might yield stronger emotional responses or different trust dynamics.
  • Game Symmetry: The specific board states shown in the snippets might have influenced betting more than the robots' speech, as UTTT moves are not perfectly symmetric.

Conclusion and Future Work

This research highlights that the value of XAI in human-robot interaction may lie more in "humanizing" the agent than in proving its intelligence. Future work will involve transitioning these experiments to physical NICO robots and allowing humans to play directly against the AI. This direct interaction will provide a more robust measure of how XAI influences human reliance and trust in real-time decision-making scenarios.

Summary updated

Abstract

Recent developments in explainable artificial intelligence promise the potential to transform human-robot interaction: Explanations of robot decisions could affect user perceptions, justify their reliability, and increase trust. However, the effects on human perceptions of robots that explain their decisions have not been studied thoroughly. To analyze the effect of explainable robots, we conduct a study in which two simulated robots play a competitive board game. While one robot explains its moves, the other robot only announces them. Providing explanations for its actions was not sufficient to change the perceived competence, intelligence, likeability or safety ratings of the robot. However, the results show that the robot that explains its moves is perceived as more lively and human-like. This study demonstrates the need for and potential of explainable human-robot interaction and the wider assessment of its effects as a novel research direction.
BibTeX
@inproceedings{ambsdorf2022explain,
  author = {Jakob Ambsdorf and Alina Munir and Yiyao Wei and Klaas Degkwitz and Harm Matthias Harms and Susanne Stannek and Kyra Ahrens and Dennis Becker and Erik Strahl and Tom Weber and Stefan Wermter},
  title = {{Explain yourself! Effects of Explanations in Human-Robot Interaction}},
  booktitle = {RO-MAN},
  year = {2022},
  url = {https://openreview.net/forum?id=9OmtWKwY8},
}
Figure