Raisins, Sultanas, and Currants: Lexical Classification and Abstraction via Context Priming

David J. HutchesACL 1993 · DOI

Abstract

This paper introduces "context priming," a dynamic statistical framework for lexical classification. By building localized networks around specific "seed" contexts, the method prunes the search space of large text corpora to identify relevant lexical similarities. The author demonstrates how this approach can discover word classes and generate "meta-nodes" for abstract representation without relying on predefined linguistic rules, effectively addressing issues of computational complexity and polysemy in natural language processing.

Introduction

In natural language processing, determining word classes—such as recognizing that "paced" and "stepped" are semantically similar to "walked"—is traditionally handled through static syntactic or semantic assignments. However, language is inherently fluid; a word’s meaning is often contingent upon its "association neighborhood."

Traditional statistical methods face two primary hurdles:

  1. Computational Complexity: Exhaustive comparison of usage patterns across massive corpora is computationally prohibitive.
  2. Polysemy: Words like "mark" can represent a name or a unit of currency. Without context, global statistical models struggle to disambiguate these distinct senses.

Context priming seeks to mitigate these issues by using specific phrases to "prime" the system, focusing analysis on relevant neighborhoods and reducing the search space.

Methodology

The proposed approach utilizes a graph-based model centered on the principle that "a word is known by the company it keeps."

Network Construction and Pruning

Starting with a "seed" context (e.g., "John walked across the room"), the system constructs a network where nodes represent lexical items and arcs represent co-occurrences. To maintain tractability, Mutual Information (MI) is employed to prune statistically weak relationships. The search is further constrained by a defined "window" (the distance between words) and "depth" (how many steps away from the seed to explore).

Similarity Metric

To quantify lexical similarity, the author employs a modified Tanimoto coefficient:

$$S(x, y) = \frac{|A \cap B|^2}{|A \cup B|}$$

where $A$ and $B$ represent the sets of arcs (neighborhoods) for two words. Squaring the numerator prioritizes nodes with a high proportion of shared neighbors, enhancing the precision of the classification.

Abstraction via Meta-nodes

Highly similar nodes are collapsed into "meta-nodes." This abstraction allows the system to treat entire word classes as single entities, which helps the model generalize and overcome the "sparsity" problem—where valid word combinations may be absent from a specific training dataset.

Experimental Setup

The methodology was tested on two English-language corpora:

  • Lancaster-Oslo-Bergen (LOB) Corpus: Approximately 1 million words with ~49,000 unique items.
  • ACL/DCI Wall Street Journal (WSJ) Subset: A larger dataset of ~18 million words and ~160,000 unique items. Pre-processing included lowercasing all letters and collapsing numerical tokens into a single category.

Key Results and Discussion

The system demonstrated a robust ability to instantiate word classes dynamically. For instance, using the seed "John walked across the room," the system successfully clustered other male names into the same class as "John."

Figure 1: A set of male names identified as a lexical class using context priming.

Beyond names, the system identified high-confidence pairings such as (currants, sultanas), (months, weeks), and (lawsuit, complaint). Empirical testing suggested that a similarity threshold of $S > 1.0$ typically yields meaningful pairings, whereas lower scores introduce significant noise (e.g., pairing "father" with "mind").

Limitations and Future Work

Despite its strengths, the system faces challenges with data sparsity. When a word has too few occurrences in the corpus, the resulting neighborhood is insufficient for accurate classification. For example, an attempt to classify "wife" resulted in a nonsensical cluster including "mouth" and "finger."

Figure 2: An example of a noisy cluster produced when data is sparse or neighborhoods are too small.

Other limitations include the lack of a principled stopping criterion for merging meta-nodes and issues with case sensitivity in the WSJ corpus, which led to the conflation of proper nouns and common words (e.g., confusing "Mark" the name with "mark" the currency). Future work will focus on refining merging thresholds and using context priming for more sophisticated word-sense disambiguation.

Summary updated

BibTeX
@inproceedings{hutches1993raisins,
  author = {David J. Hutches},
  title = {{Raisins, Sultanas, and Currants: Lexical Classification and Abstraction via Context Priming}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1993},
  doi = {10.3115/981574.981618},
}
Figure