Automatically Extracting and Representing Collocations for Language Generation

Frank A. Smadja, Kathleen R. McKeownACL 1990 · OpenReview

Abstract

Collocational knowledge is necessary for language generation. The problem is that collocations come in a large variety of forms. They can involve two, three or more words, these words can be of different syntactic categories and they can be involved in more or less rigid ways. This leads to two main difficulties: collocational knowledge has to be acquired and it must be represented flexibly so that it can be used for language generation. We address both problems in this paper, focusing on the acquisition problem. We describe a program, Xtract, that automatically acquires a range of collocations from large textual corpora and we describe how they can be represented in a flexible lexicon using a unification based formalism.

Introduction

In Natural Language Generation (NLG), producing grammatically correct sentences is only half the battle. To be truly effective, a system must produce "natural" prose that adheres to the idiosyncratic word preferences of a language, known as collocations. For example, while "heavy traffic" and "weighty traffic" might be semantically similar, only the former sounds idiomatic to a native speaker.

Traditionally, NLG systems relied on hand-coded phrasal lexicons, which are rigid and difficult to scale, or word-based lexicons that lack the "linguistic glue" required for fluency. This paper introduces Xtract, a system designed to overcome the "acquisition bottleneck" by automatically harvesting collocations from large corpora, and a representation framework that allows these phrases to be used flexibly in generation.

The Xtract System: Statistical Acquisition

Xtract identifies collocations through a multi-stage statistical analysis of raw text. Unlike simpler systems that only look for adjacent words, Xtract evaluates the relationship between word pairs across varying distances.

The core of this analysis relies on two primary metrics: Strength and Spread. Strength measures the frequency of co-occurrence, while Spread measures the variance of the distance between the two words. A low spread indicates a rigid compound (e.g., "stock market"), whereas a high spread suggests a more flexible relationship, such as a verb and its object (e.g., "shares... rose").

Table 1: Statistical metrics for word pairs including distance, strength, and spread.

As shown in the table above, Xtract can identify various types of relationships, from noun-noun compounds (NN) like "stock market" to noun-verb (NV) relationships like "shares totaled." By analyzing the surrounding context of these pairs, the system can further build complex "phrasal templates" that capture long-range dependencies.

Lexical Representation and Generation

Once collocations are extracted, they must be stored in a way that a generator can use them. The authors propose a hybrid lexicon using Functional Unification Grammars (FUGs). This approach treats collocations as constraints on word choice rather than rigid, unchangeable strings.

The lexicon is organized into three zones: sentential (for whole-sentence templates), lexical (for collocations), and syntactic (for general grammar). Through "unification," the system can merge these structures, allowing a specific collocation to be "plugged" into a general grammatical frame.

Figure 1: A lexical entry for 'advancer' showing semantic context and subject-verb collocates.

The lexical entry for "advancer" illustrates this flexibility. The system stores potential subject-verb (SV) collocates like "outnumber," "lead," or "finish." Depending on the "degree" of the market lead specified in the input data, the generator can choose the most appropriate verb to maintain professional-sounding prose.

Datasets and Results

The system was evaluated using a 10-million-word corpus of Associated Press stock market reports. Xtract successfully identified a wide array of domain-specific collocations, ranging from simple titles ("vice president") to complex predicative relations.

The resulting "Cook" generator produced highly fluent financial summaries. By automating the acquisition process, the authors demonstrate that NLG systems can be rapidly adapted to new domains—such as weather or sports—simply by processing a relevant corpus, rather than requiring years of manual lexicography.

Conclusion

The Xtract system represents a significant step toward more natural-sounding machine-generated text. By combining robust statistical extraction with a flexible, unification-based representation, the authors address both the acquisition and the implementation of collocational knowledge. Future work will focus on integrating automated syntactic parsing to further refine the classification of extracted relationships.

Summary updated

Abstract

Collocational knowledge is necessary for language generation. The problem is that collocations come in a large variety of forms. They can involve two, three or more words, these words can be of different syntactic categories and they can be involved in more or less rigid ways. This leads to two main difficulties: collocational knowledge has to be acquired and it must be represented flexibly so that it can be used for language generation. We address both problems in this paper, focusing on the acquisition problem. We describe a program, Xtract, that automatically acquires a range of collocations from large textual corpora and we describe how they can be represented in a flexible lexicon using a unification based formalism.
BibTeX
@inproceedings{smadja1990automatically,
  author = {Frank A. Smadja and Kathleen R. McKeown},
  title = {{Automatically Extracting and Representing Collocations for Language Generation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1990},
  url = {https://openreview.net/forum?id=SJEdsogu-r},
}
Figure