OpenDialKG: Explainable Conversational Reasoning with Attention-based Walks over Knowledge Graphs

Seungwhan Moon, Pararth Shah, Anuj Kumar, Rajen SubbaACL 2019 · OpenReview

Abstract

Seungwhan Moon, Pararth Shah, Anuj Kumar, Rajen Subba. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019.

OpenDialKG introduces a novel dataset and an attention-based "walking" model that allows conversational agents to navigate massive knowledge graphs (1M+ facts) strategically. By learning to prune irrelevant paths and follow natural conceptual threads, the model generates more engaging, human-like, and explainable reasoning for its recommendations.

Introduction

Building open-ended dialog systems capable of discussing diverse topics like movies or books is a significant challenge in AI. While Knowledge Graphs (KGs) provide structured factual grounding, their sheer scale—often containing millions of triples—presents a daunting "search space" problem. Most connected entities are technically correct but contextually irrelevant or socially unengaging. The core challenge lies in pruning: identifying which specific entity or relation to mention next to maintain a natural conversational flow. Previous approaches often struggled with scalability or failed to capture the logical "threads" humans use when jumping between related ideas.

The OpenDialKG Dataset

To address these challenges, the authors present OpenDialKG, a large-scale benchmark for grounded dialog. The dataset consists of 15,673 human-to-human dialog sessions (91,209 utterances) across four domains: Movies, Books, Sports, and Music. Crucially, every utterance is manually annotated with specific KG entities and the paths from a filtered version of Freebase (~1.2M triples) that connect them. This grounding allows models to learn not just what to say, but the underlying structural reasoning.

Example of a dialog grounded in a Knowledge Graph (KG) path.

Methodology: The DialKG Walker

The proposed DialKG Walker treats dialog transitions as structured traversals over a KG. The architecture consists of three primary components:

  1. Input Encoding: The model processes three modalities—entities from the current turn, the utterance text, and the dialog history. These are encoded using Bi-LSTMs and KG embeddings, then aggregated via a "modality attention" mechanism to prioritize the most relevant context.
  2. Attention-based Graph Decoder: Instead of a simple destination prediction, the model generates a walk path. An auto-regressive decoder applies attention over available relations (e.g., "directed by", "has genre") to navigate the graph step-by-step.
  3. Zero-shot Re-ranking: To handle unseen entities, the model uses a hinge rank loss to map dialog contexts into the KG embedding space, ranking candidates based on semantic proximity to the conversation's trajectory.
The DialKG Walker architecture, illustrating the attention-based path generation process.

Key Results and Discussion

The DialKG Walker emphasizes "walking" over "jumping." By following logical threads (e.g., Actor → Movie → Genre), the model mimics human conceptual transitions. This approach provides explainability by design, as the output is a literal path through the graph.

Quantitative Performance

In "All Domains" testing, the DialKG Walker achieved a Recall@1 of 13.2%, significantly outperforming standard seq2seq (3.1%) and Tri-LSTM (3.2%) baselines.

Human Evaluation

In blind tests, human raters preferred the DialKG Walker’s responses as the most natural 38.6% of the time, nearly doubling the performance of the strongest baseline (21.5%).

Transfer Learning

The model demonstrated robust "cold start" capabilities through adversarial transfer learning. It successfully adapted to new domains, such as Sports, using only a fraction of the target data by exploiting the semantic similarities between relations across different domains (e.g., "author" in books vs. "director" in movies).

Conclusion

OpenDialKG represents a step forward in creating conversational agents that can reason over large-scale knowledge. By framing dialog as a path-finding task on a graph, the DialKG Walker achieves superior performance and provides transparent, explainable reasoning. Future work includes integrating these path-finding capabilities with high-fidelity natural language generation and exploring deeper multi-hop reasoning.

Summary updated

BibTeX
@inproceedings{moon2019opendialkg,
  author = {Seungwhan Moon and Pararth Shah and Anuj Kumar and Rajen Subba},
  title = {{OpenDialKG: Explainable Conversational Reasoning with Attention-based Walks over Knowledge Graphs}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  url = {https://openreview.net/forum?id=9m57dqPY2M},
}
Figure