GKR: Bridging the Gap between Symbolic/structural and Distributional Meaning Representations

Aikaterini-Lida Kalouli, Richard Crouch, Valeria de PaivaACL 2019

TL;DR

This paper introduces extensions to the Graphical Knowledge Representation (GKR) framework to better integrate symbolic and distributional meaning representations. By separating conceptual meaning (handled by distributional methods) from contextual/Boolean phenomena (handled by symbolic methods), they propose a hybrid approach that achieves high accuracy on challenging Natural Language Inference (NLI) tasks.

What Problem Does This Paper Address?

Natural Language Understanding (NLU) systems often struggle to effectively represent the meaning of language. Two dominant paradigms exist: symbolic/structural approaches, which excel at capturing logical relationships, quantifiers, and negation, but falter with nuanced conceptual similarity; and distributional approaches, which leverage word embeddings to capture semantic similarity and conceptual meaning but struggle with complex logical structures. This paper addresses the challenge of bridging this gap, aiming to combine the strengths of both to create more robust and expressive meaning representations, particularly for tasks like Natural Language Inference (NLI).

Methods

The core of the paper is the extension of the Graphical Knowledge Representation (GKR) framework. GKR is a layered, directed graph representation that already separates semantic information into distinct sub-graphs. The authors propose two main extensions:

  • More Symbolic Approach: This approach primarily expands the lexical layer of GKR. It integrates contextualized word embeddings (like ELMO and BERT) by linking concept nodes to their corresponding embeddings. Inference is then performed by matching concepts between GKR graphs, leveraging specificity relations and incorporating distributional similarities for better matching.
  • More Distributional Approach: This approach leverages GKR's separation of "hard" compositional phenomena (contextual/Boolean) into the context graph. The conceptual graph, representing predicate-argument structure, is then enriched with distributional representations (e.g., InferSent embeddings). This allows for flexible compositionality of basic predicate-argument structures using various distributional methods, while the context graph handles the more complex logical aspects.

Datasets

The paper evaluates its "more distributional" proposal on two test sets from Dasgupta et al. (2018). These sets are specifically designed to contain NLI pairs that are difficult for standard distributional methods, involving phenomena like conjunction and negation. The authors selected these sets because they represent challenging compositional cases where GKR's proposed division of labor is expected to be beneficial.

Key Insights and Ideas

  • Division of Semantic Labor: The central idea is that distributional methods are best suited for conceptual similarity and word meaning, while symbolic methods excel at logical, contextual, and Boolean phenomena (like negation, modals, quantifiers). GKR's layered structure naturally supports this separation.
  • GKR as a Bridge: The existing GKR framework, with its distinct conceptual and contextual layers, provides a natural architecture for integrating these two types of representations.
  • Contextual Embeddings for Lexical Links: Integrating contextualized word embeddings into the lexical layer of GKR can improve the matching of concepts, especially for words with subtle semantic relationships not captured by traditional knowledge bases.
  • Factoring out "Hard" Compositionality: The "more distributional" approach demonstrates that by isolating complex logical structures in the context graph, distributional methods can be applied more effectively to the remaining conceptual structure, leading to improved performance.
  • Robustness through Hybridity: The hybrid approach offers robustness. Even if the symbolic (concept) graph has errors, the distributional representations can still enable correct matching and inference if the context graph is accurate.

Key Results

The "more distributional" proposal achieved a remarkable 99.5% accuracy on the two test sets from Dasgupta et al. (2018). The authors attribute the few misclassifications to errors in the Stanford Parser's output, which affected the initial GKR graph construction. This high accuracy is particularly significant given that these test sets were specifically designed to challenge distributional methods.

Limitations and Future Work

  • Parser Dependency: The performance of the "more distributional" approach is sensitive to the accuracy of the initial dependency parse, which impacts the construction of the GKR graphs.
  • Simplicity of Test Sets: The chosen test sets, while challenging for distributional methods, are considered "artificially simple" in some aspects (e.g., direct matching of representations). More complex datasets, like those with scrambled word order, might require a stronger symbolic/structural component.
  • Future Work: The authors suggest extending their proposals to larger-scale systems and exploring combined approaches. Further testing with diverse datasets is needed to fully understand the strengths and weaknesses of their hybrid method and to refine the balance between symbolic and distributional contributions.

Summary updated

BibTeX
@inproceedings{kalouli2019gkr,
  author = {Aikaterini-Lida Kalouli and Richard Crouch and Valeria de Paiva},
  title = {{GKR: Bridging the Gap between Symbolic/structural and Distributional Meaning Representations}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure