Distributional Semantics Meets Construction Grammar. towards a Unified Usage-Based Model of Grammar and Meaning

Giulia Rambelli, Emmanuele Chersoni, Philippe Blache, Chu-Ren Huang, Alessandro LenciACL 2019

TL;DR

This paper proposes a novel framework, Distributional Construction Grammar (DisCxG), that unifies Construction Grammar (CxG) and Distributional Semantics (DS). By integrating distributional vectors into CxG's semantic representations, DisCxG aims to create a usage-based model where grammar and meaning are learned from language use, addressing a key limitation of traditional CxG.

What Problem Does This Paper Address?

Traditional Construction Grammar (CxG) views language as a collection of form-meaning pairings called "constructions." While CxG acknowledges that these constructions emerge from language use, it has historically relied on hand-crafted, symbolic representations for their meaning, often drawing from Frame Semantics. This approach creates a disconnect: how can the semantic content of constructions be learned from empirical data, and how does it align with the usage-based nature of CxG? On the other hand, Distributional Semantics (DS) excels at learning word meanings from their distributional patterns in large corpora, representing them as dense vectors. However, DS typically focuses on word-level semantics and lacks a direct framework for representing grammatical structures like constructions. This paper bridges this gap by proposing a unified model that leverages the strengths of both approaches.

Methods

The core of the paper is the introduction of the Distributional Construction Grammar (DisCxG) framework. DisCxG integrates three main components:

  • Constructions: These are the fundamental form-meaning pairings of CxG, ranging from morphemes to abstract syntactic patterns.
  • Frames: These represent schematic semantic knowledge of scenes and situations, defining semantic roles.
  • Events: A novel component inspired by cognitive models, representing specific event instances with their participants.

The key innovation is that both Frames and Events are represented using distributional vectors derived from corpus co-occurrences. This means that instead of symbolic representations, semantic roles within frames and the details of events are captured by the statistical distribution of words in text. The framework uses recursive feature structures, similar to Sign-Based Construction Grammar (SBCG), to model these components.

Specifically, DisCxG adds a DS-VECTOR feature to the semantic layer of constructions. This allows for the integration of distributional representations of both lexical items (like the verb "read") and abstract syntactic constructions. For frames, the values of semantic roles are distributional vectors of their typical fillers. For events, they are represented as paths within a Distributional Event Graph (DEG), which captures semantic relationships and expectations derived from parsed corpora.

Datasets

The paper does not explicitly mention specific datasets used for training or evaluation. It refers to the use of "large textual corpora" for deriving distributional vectors and mentions FrameNet (Ruppenhofer et al., 2016) as a source for semantic frames. The authors also reference a Distributional Event Graph (DEG) built from "parsed corpora."

Key Insights and Ideas

  • Bridging the Gap: The most significant insight is the successful integration of distributional semantics into the formalisms of Construction Grammar, creating a unified, usage-based model.
  • Learning Meaning from Usage: By representing semantic roles and events with distributional vectors, DisCxG offers a principled way to learn constructional meaning directly from language use, addressing a long-standing challenge in CxG.
  • Complementary Strengths: The paper highlights how DS's ability to capture statistical regularities from data complements CxG's structured representation of grammar and meaning.
  • Predictive Power of Events: The inclusion of "Events" as a component, represented through distributional information, emphasizes the role of dynamic, context-specific knowledge in language comprehension and prediction.
  • Similarity Measurement: The paper points out that distributional vectors naturally allow for computing similarity between constructions and frames, which is difficult with purely symbolic representations.

Key Results

This paper is primarily theoretical, proposing a new framework. Therefore, it does not present quantitative results from experiments or evaluations. The "results" are conceptual: the establishment of the DisCxG framework and the outlined potential for it to address existing challenges in CxG.

Limitations and Future Work

The authors acknowledge several limitations and outline future directions:

  • Approximation of Frame Elements: Representing semantic roles as simple distributional vectors of their typical fillers is considered an approximation. The authors state that "how to satisfactorily characterize semantic frames and roles using DS is in fact still an open research question."
  • Balancing Components: Integrating different layers of meaning representation (semantic vectors and formal construction structures) is noted as a non-trivial problem, requiring finding optimal ways to balance them.
  • Future Work:
    • Automatic Building of Frames and Events: The goal is to automatically build distributional representations of frames and events.
  • Distributional Construction Treebank: The authors aim to create the first such treebank.
  • Predictive Model of Language Comprehension: Applying the framework to model sentence comprehension, defining how constructions are activated by syntactic, lexical, and distributional cues.
  • NLP Applications: Exploring applications in Knowledge Representation, Reasoning, Natural Language Understanding, and Generation.
  • Psycholinguistic Comparison: Using DisCxG as a basis for comparison with psycholinguistic models of human language comprehension.

Summary updated

BibTeX
@inproceedings{rambelli2019distributional,
  author = {Giulia Rambelli and Emmanuele Chersoni and Philippe Blache and Chu-Ren Huang and Alessandro Lenci},
  title = {{Distributional Semantics Meets Construction Grammar. towards a Unified Usage-Based Model of Grammar and Meaning}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure