Tagging French Without Lexical Probabilities - Combining Linguistic Knowledge And Statistical Learning

Evelyne Tzoukermann, Dragomir R. Radev, William A. GalearXiv 1997 · OpenReview

TL;DR

This paper tackles the challenge of part-of-speech (POS) tagging for French, a highly inflected language, by proposing a novel "genotype" approach. Instead of relying on word-specific probabilities (which are sparse in limited training data), they use a combination of linguistic rules and statistical learning on "genotypes" (sets of possible tags for a word) to achieve high accuracy.

What Problem Does This Paper Address?

Part-of-speech tagging is a fundamental task in Natural Language Processing (NLP), aiming to assign the correct grammatical category (like noun, verb, adjective) to each word in a text. For languages like French, which have rich morphology (words change form based on gender, number, tense, etc.), this becomes significantly more complex. A single word can have many possible analyses, making it difficult to learn reliable probabilities from limited training data. Traditional approaches often rely on lexical probabilities (how often a specific word appears with a specific tag), but this breaks down when training corpora are small, as many word forms might be missing. The authors aim to develop a POS tagger for French that handles this morphological complexity effectively, optimizes probability estimation, and offers user flexibility in tagset selection.

Methods

The core innovation of this paper is the introduction of the "genotype" concept. A genotype represents the set of all possible POS tags a given word can take, derived from its morphological analysis. Instead of learning probabilities for individual words, the system learns probabilities for these genotypes.

The tagging process involves several modules:

  • Tokenization: Preprocessing the text and breaking it into words.
  • Morphological Analysis: Using finite-state transducers (FSTs) to generate all possible analyses (and thus genotypes) for each word.
  • Linguistic Knowledge: Applying deterministic rules (negative constraints) to eliminate improbable tag sequences. These rules capture linguistic generalizations about tag co-occurrences.
  • Statistical Learning: Estimating n-gram probabilities (unigram, bigram, trigram) over genotypes. This allows the system to learn contextual patterns of tag sequences.
  • Tagset Reduction: Optionally reducing a large, detailed tagset to a smaller, more manageable one, offering user flexibility.

The key idea is that by grouping words with the same set of possible tags (their genotype), the system can leverage more robust statistical information, even for rare or unseen words.

Datasets

The authors used several corpora for training and testing:

  • Training Corpora: Three corpora of sizes 10,000, 30,000, and 36,000 words were manually tagged. An additional corpus of 10,000 words was used for initial experiments.
  • Test Corpora: Two corpora of 1,000 and 1,500 words were used for testing, reflecting different text styles.
  • Corpus for Ambiguity Analysis: Two corpora of 94,882 and 200,182 tokens were used to analyze the ambiguity of French words.

Key Insights and Ideas

  • Genotypes as a Solution to Data Sparsity: The central insight is that focusing on "genotypes" (sets of possible tags) rather than individual word forms significantly mitigates the problem of sparse training data. This allows for more reliable probability estimation, especially for less frequent words.
  • Linguistic Rules as Anchors: Deterministic linguistic rules are crucial for providing initial "anchors" by eliminating clearly incorrect tag sequences. This pre-processing step improves the efficiency and accuracy of subsequent statistical learning.
  • Contextual Probabilities Over Lexical: The paper strongly argues that for highly inflected languages like French, contextual probabilities (based on n-grams of genotypes) are more important and feasible to estimate than lexical probabilities due to data sparsity.
  • Flexibility in Tagsets: The system's design allows users to choose or define their own tagsets, making it adaptable to various NLP applications.
  • Morphological Richness is Key: The approach effectively leverages the rich morphological information available in French to disambiguate word senses, going beyond simple POS categories.

Key Results

  • High Accuracy: The tagger achieved a performance of 91% correct using unigram probabilities and 93% correct using bigram probabilities on a 10K-word training corpus, after applying negative constraints.
  • Reduced Ambiguity: The system effectively disambiguates a significant portion of French text, with the genotype approach proving more robust than relying solely on lexical probabilities.
  • Efficient Genotype Representation: The number of unique genotypes in a corpus grows much slower than the number of words, indicating an efficient representation of linguistic variation. For example, a 86% increase in corpus size led to only a 27% increase in unique genotypes.

Limitations and Future Work

  • Limitations:
    • The performance is still dependent on the quality and size of the training corpus, although significantly less so than purely lexical approaches.
  • The paper acknowledges that some words may remain ambiguous even after the system's processing, indicating areas where evidence is insufficient.
  • Future Work:
    • Improving system performance by using the entire training corpus.
  • Exploring the portability of the system to other languages, such as Spanish.
  • Further refinements to linguistic subtleties.
  • The authors suggest that this work is a step towards shallow parsing systems.

Summary updated

Abstract

This paper explores morpho-syntactic ambiguities for French to develop a strategy for part-of-speech disambiguation that a) reflects the complexity of French as an inflected language, b) optimizes the estimation of probabilities, c) allows the user flexibility in choosing a tagset. The problem in extracting lexical probabilities from a limited training corpus is that the statistical model may not necessarily represent the use of a particular word in a particular context. In a highly morphologically inflected language, this argument is particularly serious since a word can be tagged with a large number of parts of speech. Due to the lack of sufficient training data, we argue against estimating lexical probabilities to disambiguate parts of speech in unrestricted texts. Instead, we use the strength of contextual probabilities along with a feature we call ``genotype'', a set of tags associated with a word. Using this knowledge, we have built a part-of-speech tagger that combines linguistic and statistical approaches: contextual information is disambiguated by linguistic rules and n-gram probabilities on parts of speech only are estimated in order to disambiguate the remaining ambiguous tags.
BibTeX
@misc{tzoukermann1997tagging,
  author = {Evelyne Tzoukermann and Dragomir R. Radev and William A. Gale},
  title = {{Tagging French Without Lexical Probabilities - Combining Linguistic Knowledge And Statistical Learning}},
  year = {1997},
  eprint = {cmp-lg/9710002},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/cmp-lg/9710002},
}
Figure