TL;DR
This paper introduces a connectionist model that extends existing approaches to language processing, specifically focusing on how the brain resolves pronoun anaphora in connected discourse. By employing a network of interconnected units that mimic neural structures, the model demonstrates a novel way to track the "focus" of a conversation, enabling it to correctly interpret pronouns like "it" even when the antecedent isn't explicitly stated in the same sentence.
What Problem Does This Paper Address?
Understanding how humans process language, particularly connected discourse, is a significant challenge. A key aspect of this is anaphora resolution: figuring out what a pronoun (like "he," "she," "it") refers to. Traditional computational models often struggle with this, especially when the antecedent is not immediately present. This paper tackles the problem of anaphor resolution in discourse by proposing a connectionist model that aims to capture the brain's parallel processing and distributed nature, moving beyond traditional symbolic AI approaches. The authors highlight that the human nervous system's structure, characterized by massive parallelism and interconnectedness, is fundamentally different from traditional von Neumann architectures, suggesting that a connectionist approach is more biologically plausible and potentially more successful for modeling cognitive functions like language.
Methods
The paper describes a connectionist model built using an interactive simulator. This model is an extension of previous work on sentence-centered models and specifically addresses anaphoric resolution in discourse. The core of the model consists of interconnected, neuron-like units that communicate via weighted levels of excitation and inhibition.
Key methodological components include:
- Connectionist Architecture: The model employs a network of units, each with input sites, site functions, a potential function, and an output function. This mirrors the structure of biological neurons.
- Levels of Representation: The model is organized into several interacting levels:
- Lexical Level: Units representing individual words in the lexicon.
- Word Sense Level: Units representing the meaning of morphemes, handling word sense disambiguation through inhibitory links between competing senses.
- Focus Level: Units that track the "focus" of the discourse, distinguishing between "actor focus" (animate object in the agent case) and "discourse focus" (central theme). This level implements a parallel, on-line focus tracking mechanism.
- Case Level: An "exploded case" representation where specific cases (e.g., Kick-agent) are activated based on predicates and fillers.
- Schema Level: A hierarchical system of schemas representing knowledge about concepts (e.g., "WORK PLACE MEETING"). These schemas have slots that can be filled by specific units.
- Focus Tracking Mechanism: A crucial aspect is how the model tracks the discourse focus. The focus with the strongest activation inhibits competing focuses. A shift in focus occurs when a new focus's activation surpasses the old one. This is implemented using a parallel process, unlike some serial approaches.
- Anaphora Resolution: The model resolves anaphora by linking pronouns to the current discourse focus. For example, the pronoun "it" is linked to the focus unit representing the "meeting" in the provided discourse examples.
Datasets
Not discussed in the paper. The paper uses example discourse fragments (D1 and D2) to illustrate the model's performance, which are based on examples by Sidner (1983).
Key Insights and Ideas
- Discourse Focus as a Key to Anaphora: The paper strongly suggests that understanding the dynamic "focus" of a discourse is critical for resolving anaphora, especially for pronouns like "it." The model's ability to track this focus, even when it shifts, is a central contribution.
- Parallel Processing for Focus: Implementing focus tracking in a parallel, on-line manner is presented as an advantage over serial approaches, potentially mirroring cognitive processes more closely.
- Schema-Based Knowledge Representation: The use of hierarchical schemas with slots and fillers provides a structured way to represent knowledge and integrate it with the discourse processing. The model demonstrates how specific schemas (like "WORK PLACE MEETING") can be activated and contribute to focus determination.
- Limitations of Co-reference: The paper highlights why simple co-reference (where an anaphor and antecedent refer to the exact same object) is insufficient for discourse understanding, using the example of "they" referring to a class of dogs. The concept of "specification" and "co-specification" is introduced as a more robust alternative.
- Connectionist Strengths for Cognitive Modeling: The authors argue that the inherent parallelism and distributed nature of connectionist models make them well-suited for modeling cognitive functions, as they more closely resemble the brain's architecture than traditional symbolic systems.
Key Results
The paper demonstrates the model's ability to resolve simple "it" anaphora in two discourse examples (D1 and D2).
- Discourse D1: The model successfully tracks the discourse focus as the "meeting" throughout the discourse, correctly interpreting "it" in D1-2 as referring to the meeting.
- Discourse D2: The model shows how the discourse focus can shift. Initially, the focus is the "meeting," but by sentence D2-4 ("It's kind of small"), the focus shifts to "my office." The model correctly identifies this shift and resolves the subsequent "it" anaphora based on the new focus.
The paper emphasizes the qualitative success of the model in handling these examples and illustrating the principles of connectionist discourse processing, rather than presenting quantitative performance metrics.
Limitations and Future Work
The authors acknowledge several limitations:
- Specificity of Schema Representations: The current schema representations are highly specific, lacking a mechanism for variable binding (i.e., using a general schema and binding fillers to slots dynamically). This requires pre-defining all possible slot fillers.
- Schema Development and Modification: The model does not explain how new schemas are developed or how existing ones are modified. Knowledge incorporation relies on changing weights, implying that new and old elements must already be connected.
- "Brittleness" of Connectionist Models: Programming connectionist networks can lead to arbitrary fixes. The authors note the lack of structured programming techniques for networks.
- Need for Robust Sub-networks: The development of a reliable taxonomy of sub-networks (like winner-take-all and regulated networks) is suggested as a way to build more robust networks and avoid issues like saturation or silence.
Future work suggested includes developing a "Network Programming Language" (NPL) that would allow for more sophisticated network construction, selective activation of sub-networks, and better control flow, addressing some of the simulator's limitations.
Summary updated