A System for Creating and Manipulating Generalized Wordclass Transition Matrices From Large Labelled Text-Corpora

Wilfried Bloemberg, Michael KesselheimCOLING 1988

TL;DR

This paper introduces EMMA, a system designed to help linguists create and optimize wordclass systems for probabilistic grammars. EMMA tackles the challenge of finding the right level of detail for syntactic categories by providing tools to define, analyze, and manipulate transition matrices derived from large text corpora.

What Problem Does This Paper Address?

Developing robust natural language processing (NLP) systems, particularly for tasks like text-to-speech and speech recognition, often relies on probabilistic linguistic models. A common approach is to use Markov-type grammars, which model language based on the probability of transitions between syntactic categories. However, a critical challenge in this approach is determining the optimal "granularity" or level of detail for these syntactic categories (wordclasses). Too few categories might oversimplify the language, leading to poor prediction, while too many can make the model computationally intractable and difficult to manage. This paper addresses the problem of systematically finding an "optimal" set of wordclasses that balances prediction accuracy, information content, and computational economy.

Methods

The core of the paper is the description of a software system called EMMA (Editor for Matrices from Markov Analysis). EMMA is designed to assist linguists in the complex task of optimizing wordclass systems. Its key features include:

  • Set-Theoretic Formalism for Cover Symbols: EMMA uses a formal language to define "cover symbols." These are essentially higher-level categories that group together more detailed wordclasses. This allows for a hierarchical definition of syntactic categories, enabling users to abstract away from fine-grained details when necessary.
  • Matrix Manipulation and Analysis: The system allows users to create and manipulate transition probability matrices. These matrices capture the likelihood of one syntactic category following another (or a sequence of others). EMMA provides tools to generate these matrices from labelled text corpora and to transform them based on defined cover symbols.
  • Optimization Criteria and Measures: The paper outlines three key criteria for an optimal wordclass system:
  • Economy: Minimizing the number of labels.
  • Prediction: Maximizing the predictive power of transitions.
  • Information: Maximizing the information content for syntactic analysis. EMMA provides measures derived from entropy and correlation to help users evaluate different wordclass sets against these criteria. These measures include branching factor, variance of transition probabilities, correlation between labels, and relative frequency.

  • Interactive System: EMMA is designed to be user-friendly, employing menu and windowing techniques with extensive help facilities. It also supports experienced users with command file creation and logging.

Datasets

The paper mentions that the data for the examples are derived from:

  • An office text in German (80,000 words).
  • The same text in Dutch (100,000 words).

These corpora were labelled using the ESPRIT-wordclass system, with approximately 250 wordclasses for German and 104 for Dutch being used in the experiments. The paper does not introduce new benchmarks but rather uses existing labelled corpora to demonstrate the functionality of EMMA.

Key Insights and Ideas

  • The "Cover Symbol" Abstraction: The concept of "cover symbols" is central. It provides a flexible way to manage the trade-off between a highly detailed, potentially unwieldy, wordclass system and a more abstract, economical one. This allows for a systematic reduction in complexity.
  • Quantitative Evaluation of Wordclasses: EMMA moves beyond subjective linguistic intuition by providing quantitative measures (entropy-based, correlation) to assess the "goodness" of a wordclass system. This allows for data-driven optimization.
  • Hierarchical System Design: The system supports a hierarchical approach to defining wordclasses, which mirrors how linguists often think about language structure, from broad categories to specific sub-classes.
  • Balancing Competing Criteria: The paper highlights that finding an optimal system involves balancing competing demands (economy vs. prediction vs. information), and EMMA provides the tools to explore this multi-objective optimization problem.

Key Results

The paper focuses on demonstrating the capabilities of the EMMA system rather than presenting definitive quantitative results on achieving an "optimal" wordclass system. The examples illustrate:

  • Example 1: How to define a cover symbol ('Z_CON') for conjunctions in a specific scope (first position in a Markov chain) using the set-theoretic notation.
  • Example 2: Identification of frequent wordclass triples (e.g., determiner-adjective-noun, preposition-determiner-noun) in the German corpus.
  • Example 3 (Statistics): Analysis of transition probabilities and standard deviations for symbols in the first position of a chain. This example suggests that a symbol with a very low standard deviation might be a candidate for inclusion in a cover symbol, while one with a high probability might warrant its own class.
  • Example 4 (Correlations): Demonstrates high correlation between 'M02' (date) and 'B02' (adjective), suggesting they could be candidates for a shared cover symbol, but emphasizes the need for further analysis using other criteria.
  • Example 5 (Entropy): Shows entropy values for cover symbols representing main word classes in the Dutch corpus, indicating that the chosen set of cover symbols might be too small to meet information requirements for certain tasks.

Limitations and Future Work

  • Defining "Optimal": The paper acknowledges that finding an "exact" solution to the optimization problem is difficult, if not impossible, due to the dimensionality of the problem and the dependence on the analysed corpus. The system assists in finding an "optimal" system, implying a heuristic or approximate solution.
  • Corpus Dependence: The quality and nature of the training corpus significantly influence the resulting wordclass system. The paper notes this dependence.
  • Future Work: The authors suggest that the system can be extended to handle more complex linguistic phenomena and that further research is needed to explore the full potential of the defined measures and the system's capabilities. The paper implies that the partners within the consortium were just beginning the development of optimal wordclass systems at the time of writing.

Summary updated

BibTeX
@inproceedings{bloemberg1988system,
  author = {Wilfried Bloemberg and Michael Kesselheim},
  title = {{A System for Creating and Manipulating Generalized Wordclass Transition Matrices From Large Labelled Text-Corpora}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1988},
}
Figure