Focus Constraints on Language Generation

Kathleen R. McKeown

Abstract

Computer generation of natural language requires the ability to make reasoned choices from a large number of possible things to say as well as from a large number of expressive possibilities. This paper examines in detail how one influence on a generated text, focus of attention, can be used to constrain the many possibilities that a generation system must consider. A computational treatment of focus of attention is presented that can be used to constrain what the system needs to consider when deciding what to say next. In this process, information is produced that provides constraints on which words and syntactic structures best express the system's intent, thus ensuring that its resulting text is coherent. This analysis has been used in the fully implemented TEXT system which generates paragraph-length responses to questions about database structure.

Introduction

In the early development of Natural Language Generation (NLG), the primary challenge shifted from basic grammatical correctness to the more complex problem of discourse coherence. When a computer generates text, it faces an overwhelming "choice problem." For any given set of data, there are thousands of ways to order information and even more ways to phrase it (e.g., "John gave Mary the book" vs. "Mary was given the book by John"). Without a governing strategy, early systems often produced "jumpy" text that was difficult for humans to follow. This paper addresses how to decide what to say next and which syntactic structures to use so that the reader’s attention is guided smoothly from one idea to the next.

The TEXT System: Methodology

The author presents the TEXT system, a fully implemented generator that uses a computational model of "focus" to make decisions. The methodology relies on two main pillars: Immediate Focus Constraints and Surface Choice Selection.

Immediate Focus Constraints

Drawing on Sidner’s (1979) work on discourse, the system tracks the "current focus" (the main object being discussed) and a "potential focus list" (other items mentioned). The TEXT system introduces a preference ordering for how focus should move to maintain coherence:

  1. Shift focus to a newly introduced item (if there is more information to convey about it).
  2. Maintain the current focus.
  3. Return to a previous topic.
  4. Select based on implicit links.

By prioritizing a shift to a "just mentioned" item, the system mimics the natural human tendency to follow a thread of conversation to its logical end before switching topics.

Surface Choice Selection

The system uses the focus state to decide on "surface" features of the language, ensuring the syntax reflects the discourse structure:

  • Pronominalization: Deciding when to use pronouns (e.g., "it" or "she") instead of a full noun to refer to the focused item.
  • Active vs. Passive Voice: Using the passive voice specifically to keep a focused item in the subject position, even if the underlying data structure suggests an active form.
  • Lexical Choice: Choosing between verbs like "buy" and "sell" depending on whether the focus is on the buyer or the seller.

Implementation and Data

The TEXT system was implemented and tested using an Office of Naval Research (ONR) database. This database contained structured information about military vehicles and weapons, such as missiles, torpedoes, and submarines. The system generated paragraph-length responses to questions about these database structures, providing a practical testbed for focus-based generation.

Key Insights

Focus as a Search Filter

Instead of evaluating every possible sentence the grammar could produce, the system uses focus to "circumscribe" a small subset of the knowledge base. This makes generation computationally efficient by limiting the search space to only those facts relevant to the current or potential focus.

The "New Info at the End" Rule

The paper incorporates linguistic theories suggesting that "new" or "unfocused" information should appear at the end of a sentence. The TEXT system uses syntactic transformations, such as "there-insertion" (e.g., "There are two types of guided projectiles..."), to introduce a new set of objects into the focus of the discourse while keeping the flow natural.

Results and Qualitative Success

The TEXT system demonstrated that coherence is a deliberate choice rather than a byproduct of grammar. Key results included:

  • Syntactic Precision: When asked about missiles, the system correctly chooses the passive voice ("Missiles are carried by...") because "missiles" is the current focus, even if the database relation is stored as "vehicles carry weapons."
  • Logical Transitions: By following focus preference rules, the system avoids "side-tracking" and ensures each sentence ties into the previous discourse.
  • Set Introduction: The system successfully uses specific syntactic structures to introduce new sets of objects, effectively managing the reader's attention as the topic shifts.

Limitations and Future Work

While the TEXT system proved the viability of focus constraints, several areas for improvement were identified. The implementation of lexical influence (e.g., choosing between "buy" and "sell") was not fully realized in the initial version. Furthermore, the model primarily handled simple sentence structures; future work could extend focus constraints to signal complex relations like parallel structures or subordinate clauses. Finally, the author suggests that these focus tests should eventually be encoded directly into functional grammar formalisms rather than existing as a separate processing layer.

Summary updated

Abstract

Computer generation of natural language requires the ability to make reasoned choices from a large number of possible things to say as well as from a large number of expressive possibilties. This paper examines in detail how one influence on a generated text, focus of attention, can be used to constrain the many possibilities that a generation system must consider. A computational treatment of focus of attention is presented that can be used to constrain what the system needs to consider when deciding what to say next. In this process, information is produced that provides constraints on which words and syntactic structures best express the system's intent, thus ensuring that its resulting text is coherent. This analysis has been used in the fully implemented TEXT system which generates paragraph length responses to questions about database structure.
BibTeX
@inproceedings{mckeown1983focus,
  author = {Kathleen R. McKeown},
  title = {{Focus Constraints on Language Generation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1983},
}
Figure