Introduction
Modern robotics increasingly relies on Deep Neural Networks (DNNs) for complex tasks such as computer vision and tactile sensing. However, these models often function as "black boxes," providing little insight into their decision-making processes. In collaborative Human-Robot Interaction (HRI), this lack of transparency is a significant hurdle; if a robot fails a task, a human operator needs to understand the underlying cause to provide corrective feedback or maintain trust. This paper proposes a framework that bridges the gap between internal model states and external environmental feedback to foster more transparent collaboration.
The Dual Nature of Explainability
The authors categorize explainability into two distinct but complementary types:
- Intrinsic Explanations: These are derived from the agent's internal states. A primary example is Gradient-weighted Class Activation Mapping (Grad-CAM), which generates heatmaps highlighting the regions of an input image that most influenced the network's classification.
- Extrinsic Explanations: These involve information received from external sources, such as a human teacher, to fill knowledge gaps or resolve uncertainties within the robot's model.
The NICO Platform and Neuro-Symbolic Integration
The research utilizes the Neuro-Inspired COmpanion (NICO), an open-source humanoid robot designed for multimodal sensing (vision, audio, and haptics). NICO serves as a testbed for a neuro-symbolic approach, where sub-symbolic data—such as neural network weights and activations—are translated into symbolic representations (words or rules) that are intelligible to humans.
Key Insights: Repurposing XAI for Learning
A core contribution of this work is the repurposing of intrinsic explanations for functional robotic behaviors. By analyzing Grad-CAM heatmaps, the robot can perform weakly-supervised localization. For instance, if a classifier identifies an "apple," the corresponding heatmap can be converted into a binary mask or bounding box. This allows the robot to learn the physical location of objects and develop grasping behaviors more efficiently than traditional supervised methods, as it does not require explicit training on localized datasets.
However, the authors highlight the "Heatmap Gap": while heatmaps are useful for developers, they are often cryptic to non-expert users. To be effective in HRI, these signals must be converted into symbolic language—for example, explaining that an object was identified as an apple due to its "red circular shape." This transition from sub-symbolic to symbolic is essential for building a "Theory of Mind" where the robot can predict and align with human beliefs.
Conclusion and Future Work
The paper validates the PeCoX framework, which distinguishes between perceptive (sub-symbolic) and cognitive (symbolic) explainability. While the qualitative proof-of-concept demonstrates successful visual localization, challenges remain in the "translation problem"—automatically converting internal activations into robust verbal rules. Future research will focus on building symbolic knowledge graphs from DNNs and expanding explainability to include auditory and tactile modalities, creating a truly interactive framework where the robot shapes its explanations based on human feedback.
Summary updated