Greek Word Segmentation Using Minimal Information

C. Anton RyttingNAACL 2004

TL;DR

This paper explores how infants might learn to segment words from continuous speech by proposing a computational framework that tests minimal information cues. Using Modern Greek data, the study demonstrates that even very basic segmental information at utterance boundaries can significantly improve word segmentation accuracy over random chance, suggesting that infants may rely on simpler cues than previously thought.

What Problem Does This Paper Address?

A fundamental challenge in first-language acquisition is the "word segmentation problem": how do infants, with limited vocabulary and processing power, learn to identify individual words within a continuous stream of speech? While many computational models and infant studies have focused on English, there's a need to test these strategies on languages with different phonological structures to understand universal principles. This paper specifically investigates whether minimal information, particularly segmental cues at utterance boundaries, can be effective for word segmentation in Modern Greek, a language distinct from English.

Methods

The authors propose a finite-state framework to test various word segmentation models. This framework allows for the systematic comparison of different information sources. For this study, they focus on a minimal information approach, specifically examining the likelihood of a word boundary following a given segment.

The core of their method involves constructing finite-state machines (FSMs) to represent different information sources:

  • Segment Identity: A basic FSM that simply accepts the sequence of segments in an utterance.
  • Number of Words: An unweighted FSM that incorporates knowledge of the total number of words in an utterance (used as a baseline comparison).
  • Utterance-Boundary Probabilities: This is the primary focus. They test three variants:
  • Upper Bound: Uses the true probability of a word boundary given the preceding segment (simulating a supervised scenario with known word boundaries).
  • Utterance-Boundary Approximation: Approximates the word boundary probability using the probability of an utterance boundary following a segment. This is the "minimal information" cue they are most interested in, inspired by prior work.
  • Lower Bound: Uses an unweighted or equally weighted version of the utterance-boundary probabilities, focusing on the set of possible word-final segments rather than their specific likelihoods.

These FSMs are then used to process the Greek data, and the output is compared to the original text to evaluate segmentation accuracy.

Datasets

The study uses data from the Greek CHILDES corpus (Stephany, 1995). This corpus consists of transcribed conversations between children and caretakers. For their experiments, they used a subset of adult input for both training and testing. Non-segmental information like punctuation and dysfluencies were removed.

  • Training Corpus: 367 utterance tokens, 1066 word tokens (319 types). Average utterance length: 12.8 phonemes.
  • Test Corpus: 273 utterance tokens, 699 word tokens (229 types).

This dataset is notable for being from Modern Greek, a language with phonological patterns distinct from English, allowing for cross-linguistic comparison.

Key Insights and Ideas

  • Minimal Cues Can Be Powerful: The study demonstrates that even a very simple cue – the probability of a word boundary following a segment, approximated by utterance boundaries – can significantly outperform random chance in word segmentation.
  • Cross-Linguistic Applicability: The findings suggest that utterance-final information, a cue explored in English, is also relevant for word segmentation in Modern Greek, hinting at potentially universal principles in infant language acquisition.
  • Finite-State Framework Utility: The finite-state approach proves to be a flexible and effective tool for systematically testing and comparing different segmentation algorithms and information sources.
  • Supervised Learning Advantage: As expected, the "upper bound" condition, which simulates having access to known word boundaries (akin to a learned vocabulary), yields the highest accuracy, highlighting the importance of learned vocabulary in segmentation.
  • Rudimentary Knowledge Matters: Even the "lower bound" condition, which relies on a very basic understanding of possible word-final segments, performs significantly better than a random approach, emphasizing that any form of linguistic knowledge contributes to segmentation.

Key Results

The results are presented in two tables, comparing performance with and without the "number of words" constraint.

With the "Number of Words" Constraint (Table 2):

  • Upper Bound: Achieved the highest precision and recall for both words (39.6%) and word boundaries (77.3%).
  • Utterance-Boundary Approximation: Achieved 32.4% precision and 74.2% recall for word boundaries.
  • Lower Bound: Achieved 31.6% precision and 72.9% recall for word boundaries.
  • Random Walk: Achieved only 17.0% precision and 65.7% recall for word boundaries.

Without the "Number of Words" Constraint (Table 3):

  • The "Utterance-Boundary Approximation" achieved 22.7% precision and 76.0% recall for word boundaries.
  • The "Lower Bound" achieved 12.2% precision and 51.7% recall for word boundaries.
  • The "Random Walk" performed poorly with only 2.5% precision and 41.6% recall for word boundaries.

Overall, the utterance-boundary approximation consistently performed better than the random baseline, and the upper bound condition demonstrated the significant advantage of having learned vocabulary.

Limitations and Future Work

  • Minimal Information Sufficiency: While the minimal information cue performed better than chance, its accuracy is still relatively low, especially without the "number of words" constraint. This suggests that while it's a useful cue, it's likely insufficient on its own for robust segmentation.
  • Approximation of Utterance Boundaries: The paper acknowledges that using utterance boundaries to approximate word boundaries is a simplification.
  • Future Work: The authors suggest several directions:
  • Incorporating transitional probability and mutual information measures between adjacent segments.
  • Developing more sophisticated models for approximating word-length distributions from utterance-length information, pause durations, and other salient cues.
  • Investigating the role of stress cues, both independently and in conjunction with segmental information.
  • Expanding the finite-state framework to incorporate more complex linguistic information and potentially other techniques.

Summary updated

BibTeX
@inproceedings{rytting2004greek,
  author = {C. Anton Rytting},
  title = {{Greek Word Segmentation Using Minimal Information}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2004},
}
Figure