TL;DR
This paper introduces a novel connectionist parser (CM parser) that aims to explain human sentence parsing phenomena like garden path and deeply nested sentences. By using Sigma-Pi units and a unique network architecture, it proposes a unified principle for parsing preferences based on activation levels, potentially bridging the gap between symbolic and connectionist approaches to natural language processing.
What Problem Does This Paper Address?
Traditional symbolic parsing models, while effective, struggle to explain certain cognitive aspects of human sentence processing. Specifically, they often fail to account for why humans find garden path sentences (e.g., "The astronomer married the star") and deeply nested sentences difficult to parse, or why we exhibit preferences for certain interpretations of ambiguous sentences. The authors argue that existing models, rooted in symbol manipulation, lack the flexibility to integrate syntactic, semantic, and contextual information uniformly and to naturally explain these "hard-to-recognize" phenomena. They propose a connectionist approach as a more promising avenue for understanding these aspects of human sentence parsing.
Methods
The core of the proposed CM parser lies in its connectionist architecture, specifically utilizing Sigma-Pi Units. These units are crucial for implementing the "trigger link" mechanism, which enforces the order of activation between units representing grammatical categories (e.g., NP must precede VP). This mechanism allows the parser to represent phrase structure rules like C → A B, where B follows A.
A key innovation is the copying sub-network mechanism. Instead of pre-defining an infinite number of sub-networks for recursive phrase structure rules, the CM parser uses a single central sub-network for each rule. When a sub-network is needed, it's dynamically copied and integrated into the parse tree. This copying is managed by a Connection Information Distribution (CID) mechanism and three types of connectors (A, B, and C-connectors). These connectors dynamically establish connections between sub-networks, allowing the parse tree to grow in a bottom-up fashion.
Furthermore, the paper introduces a mechanism to handle resource-bounded conditions. This is achieved through weak suppressive connection links between every pair of units. This limitation prevents the parser from maintaining all possible candidate parse trees simultaneously, forcing it to favor the most strongly activated parse tree, thus explaining preferences.
Datasets
Not discussed in the paper. The paper focuses on the theoretical framework and demonstrates its principles with example sentences rather than formal evaluation on datasets.
Key Insights and Ideas
- Unified Principle for Parsing Preferences: The CM parser proposes that parsing preferences (like minimal attachment or right association) can be uniformly explained by the activation levels of units within the connectionist network. The most strongly activated parse tree is inherently preferred.
- Dynamic Sub-network Copying: The ability to dynamically copy and integrate sub-networks for phrase structure rules, rather than pre-instantiating them, is a significant step towards a more biologically plausible and computationally efficient connectionist parser.
- Weak Suppression for Resource Bounding: The introduction of weak suppressive links between units elegantly models the limited capacity of human short-term memory, explaining why multiple parse trees cannot be maintained indefinitely and how the parser settles on a single interpretation.
- Explaining Cognitive Difficulties: The model provides a principled explanation for why deeply nested and garden path sentences are difficult. In deeply nested sentences, multiple units might have similar high activation levels, making selection difficult. For garden path sentences, a initially favored parse tree might be syntactically invalidated later, requiring a sudden shift in activation, which is cognitively demanding.
Key Results
The paper doesn't present quantitative results from experiments on large datasets. Instead, it offers qualitative explanations for several linguistic phenomena:
- Deeply Nested Sentences: The parser explains the difficulty by showing how multiple competing parse paths can achieve similar high activation levels, making definitive selection challenging.
- Garden Path Sentences: The model explains the difficulty as a consequence of a initially preferred parse tree being syntactically invalidated, forcing a sudden shift in activation to a less favored but ultimately correct path.
- Syntactic Ambiguity Preferences: The parser integrates principles like minimal attachment and right association by relying on the activation levels of units, demonstrating how these preferences can emerge from the network's dynamics. For example, in "John bought the book which I had selected for Mary," the model predicts a preference for connecting "for Mary" to the VP that was more strongly activated.
Limitations and Future Work
- Lack of Formal Evaluation: The paper presents a theoretical framework and illustrative examples rather than empirical validation on large corpora. The performance of the CM parser in terms of accuracy, speed, and robustness is not quantified.
- Semantic and Contextual Integration: While the authors mention the advantage of connectionist models in integrating semantic and contextual information, this paper primarily focuses on syntactic parsing. The paper explicitly states that unifying semantic and context information is a direction for future work.
- Learning Mechanism: The paper assumes that connection weights are already learned. The process of how these weights are acquired through learning is not detailed.
- Computational Cost of Copying: While dynamic copying is an improvement, the computational overhead of creating and managing these copies in real-time parsing is not discussed.
Future work suggested by the authors includes the unification of semantic and contextual information into the CM parser, which would make it a more comprehensive model of human language understanding.
Summary updated