Learning Commonsense Knowledge Through Interactive Dialogue

Benjamin Wu, Alessandra Russo, Mark Law, Katsumi Inoue

TL;DR

This paper introduces a novel system that learns commonsense knowledge through interactive dialogue with a user. By translating natural language into logical formulas and using Answer Set Programming (ASP) for reasoning and learning, the system can acquire new concepts and facts with minimal background knowledge, demonstrating a more natural and flexible approach to AI learning.

What Problem Does This Paper Address?

A fundamental challenge in Artificial Intelligence is equipping machines with commonsense knowledge – the vast, implicit understanding of the world that humans possess. While significant progress has been made in Natural Language Processing (NLP) for linguistic analysis, the ability to truly comprehend and learn commonsense remains elusive. This paper tackles the problem of acquiring commonsense knowledge by enabling an AI system to learn through dialogue, mimicking how humans naturally acquire knowledge from interaction and explanation. This is crucial for building more robust and adaptable AI systems that can understand and reason about the world in a human-like manner.

Methods

The system employs a multi-stage approach:

  • Translation Module: Sentences are parsed using spaCy for dependency analysis and POS tagging. WordNet is used to find general synonyms for root verbs. Adpositions, adjectives, and auxiliary verbs are appended to the root verb to form a predicate symbol. Nouns are then added as arguments to create an n-ary predicate. This process translates natural language into logical representations.
  • Event Calculus and Answer Set Programming (ASP): The translated logical representations are then encoded using a variant of Event Calculus, a logic-based formalism for reasoning about actions and events. Answer Set Programming (ASP) is used for both reasoning and inductive learning.
  • Context-Dependent Learning from Answer Sets (ILASP): The system utilizes a simplified ILASP framework for learning. Examples are represented as context-dependent partial interpretations, allowing for background knowledge to be specific to certain scenarios. This enables the system to learn hypotheses that are consistent with different contexts.
  • Interactive Dialogue: The core of the learning process is an interactive dialogue. The system asks "yes/no" questions to the user. Based on the user's feedback (correct/incorrect answers), the system generates positive or negative examples. If an answer is incorrect, a learning task is initiated.

Datasets

The system is evaluated using stories inspired by the Facebook's bAbI question-answering tasks. These tasks provide a benchmark for evaluating commonsense reasoning and question answering capabilities. The paper does not introduce new datasets but adapts existing ones for its evaluation.

Key Insights and Ideas

  • Learning through Dialogue is Natural: The paper highlights that learning commonsense through dialogue is a natural process for humans and proposes that AI systems can benefit from this approach.
  • Minimal Background Knowledge: The system can learn effectively with very limited initial background knowledge, primarily relying on domain-independent axioms of Event Calculus.
  • Context-Dependent Learning: The use of context-dependent partial interpretations allows the system to learn more nuanced concepts by considering the specific circumstances of each example, preventing conflicts that might arise from a global background knowledge.
  • Automatic Mode Declaration Generation: A significant advantage is the system's ability to automatically define mode declarations for learning tasks, making it more flexible and adaptable compared to systems that require manual definition.
  • Handling of Negation and Disjunction: The system incorporates specific rules to handle negation and disjunctions in sentences, translating them into appropriate Event Calculus predicates.

Key Results

The system demonstrates the ability to learn concepts from simple sentences through interactive dialogue. It can learn concepts related to:

  • Location and Movement: Understanding that moving to a new location implies not being in the previous one.
  • Carrying Objects: Learning the concept of "carrying" through interactions involving picking up and dropping objects.
  • Two-Supporting Facts: The system can answer questions requiring two supporting facts, equivalent to task 2 of the bAbI dataset.
  • Coreference and Basic Deduction: The system shows promise in handling basic coreference and deduction tasks, as demonstrated by its ability to learn from stories inspired by various bAbI tasks.

The paper emphasizes that the results are more qualitative than quantitative, showcasing the system's capabilities through examples of dialogues. It is noted that the system can learn concepts in as few as two or three questions, but this can vary greatly depending on the complexity of the concept and the user's questioning strategy.

Limitations and Future Work

  • Translation Module Limitations: The current translation module struggles with complex sentence structures, particularly those involving multiple arguments and inconsistent parsing. The authors suggest using statistical machine learning for translation to improve accuracy.
  • Scalability: As stories become more complex and new predicates are introduced, the hypothesis space can grow exponentially, potentially making learning too slow. The authors suggest strategies like using the "save hypothesis" functionality to clear old mode declarations and breaking down complex concepts into manageable steps.
  • Variable Type Definition: Currently, users must define variable types. Ideally, the system should be able to accurately define these types itself without being too specific or too general.
  • Ontologies and Relational Databases: Future work could involve using ontologies or relational databases like ConceptNet to map variables to similar types, improving generalization.
  • Automated Dialogue: The current interactive approach is manual. Automating the dialogue process would enable quantitative analysis over larger datasets.

Summary updated

Abstract

One of the most difficult problems in Artificial Intelligence is related to acquiring commonsense knowledge - to create a collection of facts and information that an ordinary person should know. In this work, we present a system that, from a limited background knowledge, is able to learn to form simple concepts through interactive dialogue with a user. We approach the problem using a syntactic parser, along with a mechanism to check for synonymy, to translate sentences into logical formulas represented in Event Calculus using Answer Set Programming (ASP). Reasoning and learning tasks are then automatically generated for the translated text, with learning being initiated through question and answering. The system is capable of learning with no contextual knowledge prior to the dialogue. The system has been evaluated on stories inspired by the Facebook's bAbI's question-answering tasks, and through appropriate question and answering is able to respond accurately to these dialogues.
BibTeX
@inproceedings{wu2018learning,
  author = {Benjamin Wu and Alessandra Russo and Mark Law and Katsumi Inoue},
  title = {{Learning Commonsense Knowledge Through Interactive Dialogue}},
  booktitle = {ICLP (Technical Communications)},
  year = {2018},
  doi = {10.4230/oasics.iclp.2018.12},
}
Figure