Deep Natural Language Understanding of News Text

Jaya Shree, Emily Liu, Andrew Gordon, Jerry HobbsNAACL 2019

TL;DR

This paper revisits the "interpretation as abduction" approach for deep natural language understanding, demonstrating its continued relevance and advancements. The authors successfully applied a modern abductive reasoning engine to a challenging news paragraph, showing it can recover nuanced meaning, coherence, and resolve complex references. This work highlights the progress in abductive reasoning and its potential for practical NLP applications.

What Problem Does This Paper Address?

Traditional Natural Language Processing (NLP) often focuses on shallow tasks like sentiment analysis or basic translation. However, humans comprehend text by constructing a much richer, inferential model. This paper tackles the challenge of achieving this "deep understanding" by explicating and encoding the inferences people make when reading. Specifically, it aims to derive the logical form of a text, incorporate commonsense knowledge, and use abductive reasoning (inference to the best explanation) to find the most plausible interpretation of a complex news paragraph. The authors are interested in whether recent advances in abductive reasoning can handle the subtle meanings, implicit relations, and coherence structures present in real-world text.

Methods

The core methodology is interpretation as abduction, an approach that views language understanding as a search problem. The goal is to find a set of assumptions (logical entailments) that explain the observed words in a text, given a knowledge base of linguistic and commonsense axioms.

Key aspects of their approach include:

  • Logical Form Generation: While initially exploring automated parsing (using Boxer), they opted to hand-author the logical form of the target paragraph to focus on the semantic and pragmatic challenges.
  • Commonsense Knowledge Encoding: They built a knowledge base of 80 hand-authored axioms (definite clauses). These axioms were designed to be general but specifically tailored to the target paragraph's interpretation. They acknowledge the need for both lexical knowledge (e.g., meaning of "means") and core commonsense theories (e.g., psychology, composite entities).
  • Incremental Etcetera Abduction: They utilized a modified implementation of incremental Etcetera Abduction. This engine treats the entire paragraph as a single input sequence, processing it incrementally. This approach helps manage the combinatorial explosion inherent in abductive reasoning.
  • Heuristics for Merging: To prevent over-merging of assumptions (i.e., incorrectly identifying entities as coreferential), they implemented two heuristics: one based on contradictory properties and another based on self-reference constraints. These heuristics significantly sped up processing.

Datasets

Not discussed in the paper. The authors used a single paragraph from a New York Times article as their test case and manually constructed the knowledge base.

Key Insights and Ideas

  • Abduction's Enduring Relevance: Despite the rise of statistical and deep learning methods, interpretation as abduction remains a powerful framework for deep NLU, especially for capturing nuanced meaning and coherence.
  • The Power of Hand-Authored Axioms: For complex, subtle meanings, manually crafting axioms can provide the precision needed for deep understanding, though it's labor-intensive.
  • Incremental Processing is Crucial for Scale: Processing text incrementally, rather than sentence by sentence, is essential for managing the computational complexity of abductive reasoning on longer passages.
  • Commonsense is Key, but So is "Glue" Knowledge: The knowledge base required not only core commonsense theories but also "glue" axioms to bridge semantic gaps and coerce arguments, highlighting the multifaceted nature of knowledge engineering.
  • Heuristics Dramatically Improve Efficiency: Simple heuristics for managing coreference and merging assumptions can significantly speed up abductive inference and prevent errors.

Key Results

  • The system successfully produced the correct interpretation (proof graph) for the entire 75-word paragraph.
  • No incorrect identifications of entities were made.
  • The interpretation correctly captured the coherence structure of the text and resolved the definite noun phrase "this fundamental problem" to the concept of "lack of control."
  • The interpretation was achieved with a modest window parameter (four literals) and a small beam (two hypotheses), but required a significant depth parameter (seven backward-chaining steps) due to the complexity of the semantic knowledge.
  • The final proof graph involved 32 assumptions of prior probabilities and 71 assumptions of conditional probabilities.

Limitations and Future Work

  • Scalability to Longer Texts: While incremental processing helps, the combinatorial nature of the search problem still limits scalability for very long and complex texts. The authors question how large a knowledge base will be needed for unseen texts and how to construct it.
  • Generalizing "Glue" Axioms: The axioms used to coerce or shuffle arguments had an "ad hoc" feel. Developing a more general approach for this type of knowledge is a key area for future work.
  • Automated Knowledge Acquisition: The current work relies on hand-authored axioms. Future research should focus on the automatic acquisition of axioms required for deep understanding.
  • Utility of Current Technologies: The paper highlights the need for further improvement in technologies for identifying the logical form of text.
  • Further Testing: The authors acknowledge that this is a "partial proof of possibility" and encourage further research on longer passages to uncover and solve additional technical problems.

Summary updated

BibTeX
@inproceedings{shree2019deep,
  author = {Jaya Shree and Emily Liu and Andrew Gordon and Jerry Hobbs},
  title = {{Deep Natural Language Understanding of News Text}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2019},
}

Loading related content...

Figure