Introduction
In the realm of natural language processing (NLP), tracking the "mental hooks" that speakers and listeners use to refer to objects is a fundamental challenge. These hooks, known as discourse entities, allow us to understand that "they" in one sentence refers to "seniors" in the previous one. While Bonnie Webber established the theoretical framework for these entities in the late 1970s, transitioning from descriptive rules to a functional computational system remained a significant hurdle. This paper details the first practical implementation of these theories within Janus, a multi-modal interface designed for complex command-and-control environments.
The Challenge of Discourse Tracking
For a computer to interact naturally, it needs a formal way to generate and track entities introduced during a conversation. The authors identify four primary obstacles in creating a robust discourse entity system:
- Context Dependence: Entities often change meaning based on temporal or situational contexts (e.g., "the ships ready yesterday" vs. "the ships ready today").
- Indeterminacy: Formally representing indefinite references (e.g., "a cat") when the specific individual is unknown.
- Complex Dependencies: Tracking entities that depend on others, such as "the peach each girl wanted," where the identity of the peach is tied to a specific girl.
- Multi-modality: Integrating non-verbal inputs, such as a user pointing at a map, into the linguistic discourse.
Methodological Innovations
The Janus system utilizes World Model Language (WML), a higher-order intensional logic, to represent these entities. The authors refined Webber’s original rules through several key technical advancements:
Intensional Core and Skolemization
By representing entities with an "intensional core" indexed by time and world state, Janus ensures that if a user refers to historical data, the system points to the correct state even if the current reality has changed. Furthermore, the authors replaced Webber’s somewhat vague EVOKE predicate with Skolem functions. This provides a mathematically rigorous way to pick an "arbitrary individual" from a set and treat it as a referential constant, providing a clearer framework for handling indefinite noun phrases.
Generalized Dependency Rules
The authors updated structural rules to handle nested quantifiers. This ensures that the "distributiveness chain" between entities is preserved. For example, in the sentence "Every boy gave a girl he knew a peach she wanted," the system must track that the peach is tied to a specific girl-boy pair; otherwise, it might incorrectly include peaches given to girls the boys did not actually know.
Pointing as a Communicative Act
A significant leap in this work is the treatment of gestures as first-class linguistic citizens. A mouse click on a display is categorized as a "pointing communicative act," generating a discourse entity just as a noun phrase would. This allows the system to seamlessly handle mixed-mode queries like "Which of them are within 200 miles of it? <point to Kennedy>."
Implementation and Domains
As a foundational work in computational linguistics, the system was demonstrated within two specific military command-and-control domains:
- FCCBMP: DARPA’s Fleet Command Center Battle Management Program (Navy).
- ALBM: The Air Land Battle Management Program (Army).
The implementation proved that logic-based discourse tracking could be modular. By operating on the semantic representation (WML), the discourse component remained functional even when the underlying syntactic parser was changed.
Limitations and Future Work
Despite its success in integrating text and gesture for real-time anaphora resolution, the authors noted several areas for improvement. The system did not yet handle "donkey sentences" (e.g., "Every farmer who owns a donkey beats it") or complex interactions with negation. Additionally, while the system tracked objects effectively, it did not yet generate discourse entities for events (actions described by verbs), which is necessary for resolving phrases like "That happened yesterday." Future work aimed to integrate "centering theory" to better manage shifts in conversational focus, particularly when triggered by non-linguistic events like menu selections.
Summary updated