TL;DR
This paper introduces the concept of "focus of attention" as a crucial constraint for natural language generation systems. By modeling how a system's focus shifts, it can make more reasoned choices about what information to present and how to structure it, leading to more coherent and natural-sounding generated text.
What Problem Does This Paper Address?
Generating natural language requires more than just accessing a large knowledge base; it demands making intelligent choices about what to say and how to say it. A key challenge is constraining the vast possibilities of information and linguistic expression to produce coherent and relevant output. This paper tackles the problem of how to guide these choices, specifically by investigating the role of "focus of attention" – what the system is currently attending to – in shaping the generated text. Without such constraints, generated text can be disjointed, irrelevant, or difficult to follow.
Methods
The paper proposes that "focus of attention" can be used computationally to constrain the generation process. This is achieved by:
- Global Focus: Initially restricting the knowledge base to information relevant to the overall discourse goal. This is a high-level filtering mechanism.
- Immediate Focus: Modeling how attention shifts between utterances. The system uses constraints developed by Sidner (1979) to manage focus. These constraints dictate preferences for shifting focus to newly introduced items, maintaining focus on the current topic, or returning to a previous topic.
- Discourse Schema: A "schema" or discourse plan guides the overall structure of the answer.
- Focus Record: For each proposition selected for inclusion, a "focus record" is maintained. This record includes the focused argument and a "potential focus list" of other arguments that could become the focus in subsequent utterances.
- Surface Choice: The focus record influences surface-level decisions like lexical choice, pronominalization, and syntactic structure (e.g., active vs. passive voice). For instance, if the protagonist is the focused argument, the active voice is preferred; if the goal is focused, the passive voice is used.
Datasets
Not discussed in the paper. The paper mentions that the TEXT system was implemented using an ONR database containing information about military vehicles and weapons, and examples are taken from this domain.
Key Insights and Ideas
- Focus as a Constraint: The core idea is that "focus of attention" isn't just about what information is available, but what information is currently relevant to the system and the user. This relevance acts as a powerful constraint on generation.
- Shifting Focus is Natural: Human conversation involves natural shifts in focus. The paper formalizes these shifts (e.g., to new information, back to old topics) and shows how they can be modeled computationally.
- Focus Guides Surface Realization: The paper demonstrates a direct link between the abstract concept of focus and concrete linguistic choices. For example, focus dictates whether to use a pronoun or a full noun phrase, and whether to use an active or passive sentence structure. This makes generated text more natural and easier to understand.
- Coherence Through Focus Management: By managing focus, the system can ensure that its output is not only factually correct but also logically connected and easy for the listener/reader to follow, avoiding abrupt topic shifts or confusing pronoun references.
Key Results
The paper doesn't present quantitative results in terms of metrics like accuracy or BLEU scores. Instead, it focuses on demonstrating the mechanism and qualitative benefits of using focus constraints. The key results are:
- A computational model for managing focus of attention in a generation system (TEXT).
- Demonstration of how focus influences decisions about:
- Lexical choice: Selecting words that best express intent.
- Pronominal reference: Deciding when to use pronouns versus full noun phrases.
- Syntactic structure: Choosing between active and passive voice, and the use of "there-insertion."
- Examples are provided to illustrate how inappropriate surface choices can be avoided by considering focus, leading to more coherent text.
Limitations and Future Work
- Limited Implementation: The current implementation of focus constraints is limited, particularly in its influence on lexical choice and the use of other surface-level structures like parallel sentence structures or subordinate clauses.
- Future Work: The authors suggest extending the implementation to include tests for additional types of surface choice, such as lexical choice and the use of parallel or subordinate sentence structures to signal focus shifts or maintain cohesiveness. Encoding these surface choice tests as part of the grammar is also proposed as a future direction.
Summary updated