Introduction
As robots transition from controlled industrial environments to dynamic household settings, their ability to reason about semantic relationships becomes critical. Modern robotic systems often utilize Knowledge Graph Embeddings (KGEs) to perform link prediction—inferring missing facts such as "a sponge is likely found in a sink." While KGEs are highly effective at representing complex relational data, they operate as "black boxes." Their internal logic is buried within high-dimensional vectors that are unintelligible to human users.
This lack of transparency creates a significant hurdle when a robot fails. If a robot searches for a coffee mug in a shower, a user cannot easily discern if the error arises from a sensor failure, a planning glitch, or a fundamentally flawed internal belief. This paper introduces a framework for Inference Reconciliation, allowing robots to explain the "why" behind their knowledge inferences in natural language, enabling users to correct faulty beliefs and improve task performance.
The Problem: The Gap in Robotic Explainability
Existing research in Explainable AI (XAI) for robotics has primarily focused on planning—explaining the "how" of a robot's actions (e.g., "I am moving to the kitchen to get a cup"). However, there is a distinct gap in explaining the underlying knowledge that informs those plans. When a robot makes an incorrect inference about the world, it lacks the mechanism to communicate its reasoning to a human. Without this communication, the human cannot act as a "teacher" to rectify the robot's world model.
Methodology: Pedagogical XAI via Decision Trees
The authors propose a pedagogical XAI approach, where a simple, interpretable model is trained to mimic the behavior of a complex black-box model. The framework consists of four primary components:
- The Black Box Model: The system uses TuckER, a state-of-the-art KGE, to represent the robot's world knowledge.
- Subgraph Feature Extraction (SFE): To explain a specific inference, the system identifies "neighbor" facts—paths of relationships in the graph (e.g., Object A is in Cabinet B, and Cabinet B is in the Kitchen).
- Local Approximation with Decision Trees: Unlike previous attempts that used linear models, this framework employs a Decision Tree Classifier to locally approximate the KGE’s predictions. Decision trees were selected for their ability to handle non-linear combinations of knowledge paths while remaining human-interpretable.
- Natural Language Generation: The logic captured by the decision tree is converted into human-readable explanations using language templates (e.g., "I believe a cleaning rag is in the cabinet because cabinets often contain towels and rags").
By focusing on local approximation—explaining one specific inference at a time rather than the entire global embedding—the model achieves high fidelity to the original black-box logic.
Experimental Setup and Datasets
The framework was evaluated using the VirtualHome simulator, which provided a knowledge graph of 291 entities (rooms, objects, locations) and various semantic relations (e.g., ObjInLoc, HasEffect).
To test the "reconciliation" aspect, the researchers created a Corrupted VirtualHome dataset, intentionally injecting errors into 30% of the facts. This simulated a robot with noisy or incorrect perceptions. The goal was to determine if human users, provided with explanations, could identify these errors and provide the correct information to the robot.
Key Results
The study yielded three major findings regarding fidelity, user preference, and functional utility:
- Algorithmic Fidelity: The decision tree model achieved a 98.9% F1-Fidelity score, meaning it almost perfectly mirrored the predictions of the TuckER KGE. This significantly outperformed the baseline XKE model, which achieved only 87.9%.
- User Preference: In a study involving 50 participants, users expressed a statistically significant preference for knowledge-graph-based explanations over standard causal-link explanations when querying the robot's beliefs.
- Performance Improvement: The most impactful result was the functional benefit of explainability. When non-expert users used the explanations to correct the robot's "noisy" beliefs, the robot’s link-prediction performance (measured by Mean Reciprocal Rank) improved by 117%. Furthermore, the success rate for executing household tasks improved by 33.7%.
Discussion and Future Work
The results demonstrate that explanations are more than just a tool for building trust; they are a functional mechanism for "closing the loop" between human knowledge and robotic inference. By allowing a user to correct a robot's internal knowledge graph, the system directly improves downstream task success.
However, limitations remain. The current system relies on template-based language, which can result in repetitive or "robotic" dialogue. Future iterations could integrate Large Language Models (LLMs) to generate more fluid, natural explanations. Additionally, the computational cost of bi-directional path searching in massive, real-world knowledge graphs (containing millions of nodes) remains a challenge for scalability. Finally, the authors note that in specialized domains like surgical robotics, the effectiveness of this framework would depend on the user possessing the necessary expertise to identify and correct complex factual errors.
Summary updated