Prosodic boundary information helps unsupervised word segmentation

Bogdan Ludusan, Gabriel Synnaeve, Emmanuel Dupoux

Introduction

Human speech lacks the clear physical boundaries found in written text. For both infants and computational models, identifying where one word ends and the next begins—the "word segmentation" problem—is a fundamental challenge in language acquisition. While most unsupervised models rely on statistical patterns in phoneme sequences, they often overlook prosody: the rhythm, stress, and intonation of spoken language. Developmental psychology suggests that infants leverage prosodic cues, such as syllable lengthening or pitch drops, to identify word limits. This research explores whether integrating these acoustic "hints" into a Bayesian grammar model can enhance machine word segmentation.

Methodology

The researchers integrated a novel prosodic detection system into a hierarchical Bayesian model known as Adaptor Grammars (AGs), specifically the "Colloc3-Syll" variant. This model assumes a structure where sentences are composed of word collocations, which are themselves composed of syllables.

Acoustic cues used for prosodic boundary detection, including pause duration, nucleus lengthening, and pitch (F0) changes.

The system extracts four primary acoustic cues from raw audio:

  • Pause duration: Silence following a syllable.
  • Nucleus duration: The lengthening of the vowel (final lengthening).
  • Onset distance: The timing between the start of successive syllables.
  • F0 reset: Significant changes in fundamental frequency (pitch) between syllables.

These cues are combined into a detector function. When the function exceeds a specific threshold, a boundary marker (|) is inserted into the grammar. The study compared "Oracle" boundaries (hand-annotated) against "Automatic" boundaries (algorithmically extracted) to assess the model's tolerance for noise.

Key Insights

A critical finding of the study is the importance of precision over recall in prosodic boundary detection. For word segmentation, it is more beneficial to identify a few highly accurate boundaries than to capture all boundaries at the cost of "false alarms." Incorrectly marking a boundary within a word creates a "hard constraint" that the model struggles to overcome.

Furthermore, prosodic boundaries help prevent "over-segmentation," a common issue where unsupervised models break words into excessively small units. By acting as structural anchors, these boundaries force the model to recognize larger, more accurate linguistic units.

Experimental Results

The approach was tested across three datasets: the Boston University (BU) Radio News (English), the LUCID Corpus (spontaneous English), and the Corpus of Spontaneous Japanese (CSJ).

Token F-score improvements across BU (English) and CSJ (Japanese) datasets when incorporating prosodic information.

The results demonstrate significant performance gains:

  • English (BU): Incorporating Oracle prosodic information yielded a 7% absolute increase in token F-score. Automatic detection also provided a substantial boost.
  • Japanese (CSJ): The model showed a massive improvement, with the F-score jumping from a baseline of ~44% to 55% when using automatic boundaries tuned for high precision.
  • Cross-Linguistic Robustness: The benefits were consistent across both languages. The larger gain in Japanese may be due to its higher frequency of polysyllabic words and complex embedded structures, where prosodic "hints" are particularly valuable for disambiguation.

Conclusion and Future Work

While the integration of prosody significantly improves segmentation, limitations remain. The current system relies on "Oracle" segmental information (pre-defined phoneme locations). A fully autonomous system would need to discover both phonemes and prosody simultaneously from raw audio. Additionally, performance on spontaneous speech (LUCID) was lower due to irregular rhythms and hesitations. Future research aims to move toward probabilistic models that weigh the likelihood of prosodic breaks alongside statistical phonemic patterns, rather than relying on binary boundary decisions.

Summary updated

Abstract

It is well known that prosodic information is used by infants in early language acquisition. In particular, prosodic boundaries have been shown to help infants with sentence and wordlevel segmentation. In this study, we extend an unsupervised method for word segmentation to include information about prosodic boundaries. The boundary information used was either derived from oracle data (handannotated), or extracted automatically with a system that employs only acoustic cues for boundary detection. The approach was tested on two different languages, English and Japanese, and the results show that boundary information helps word segmentation in both cases. The performance gain obtained for two typologically distinct languages shows the robustness of prosodic information for word segmentation. Furthermore, the improvements are not limited to the use of oracle information, similar performances being obtained also with automatically extracted boundaries.
BibTeX
@inproceedings{ludusan2015prosodic,
  author = {Bogdan Ludusan and Gabriel Synnaeve and Emmanuel Dupoux},
  title = {{Prosodic boundary information helps unsupervised word segmentation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2015},
  doi = {10.3115/v1/n15-1096},
}
Figure