Extending the punctuation module for european portuguese

Fernando Batista, Helena Moniz, Isabel Trancoso, Hugo Meinedo, Ana Isabel Mata, Nuno MamedeInterspeech 2010

TL;DR

This paper enhances automatic subtitle generation for European Portuguese by extending its punctuation module to include question marks, leveraging prosodic features alongside lexical and acoustic cues. The work demonstrates that prosodic information significantly improves punctuation accuracy, particularly for distinguishing between different types of interrogative sentences.

What Problem Does This Paper Address?

Automatic subtitle generation, especially for live broadcasts, requires highly efficient and accurate processing of spoken language. A critical component of this is accurately placing punctuation marks, which not only aids readability but also helps in segmenting speech into meaningful units (sentence boundaries). Previous work on European Portuguese (EP) focused on full stops and commas using lexical and acoustic features, but lacked the ability to detect question marks. This paper addresses the challenge of extending this module to handle question marks, recognizing that interrogative sentences, especially in EP, have unique linguistic characteristics that require more sophisticated feature sets.

Methods

The authors build upon their existing discriminative maximum entropy (ME) model for punctuation. The core of their contribution lies in incorporating prosodic features into the model. These features, extracted from the speech signal, include:

  • Pitch: Analyzing pitch contours, slopes, and values at word and syllable boundaries.
  • Energy: Similar to pitch, examining energy levels and slopes.
  • Duration: Measuring the length of phones, words, and inter-word pauses, with a focus on final syllables and phones.

They experimented with different units of analysis (phones, syllables, words) and found that word-based prosodic features, combined with pitch and energy, yielded the best results for full stops and commas. For question marks, they treated the entire sentence as the unit of analysis and combined lexical, acoustic, and prosodic features.

Datasets

  • Portuguese Broadcast News (BN) Corpus: This corpus, collected between 2000 and 2001, was used for training and evaluation. It includes both planned and spontaneous speech. The manual transcriptions were revised by an expert linguist.
  • Written Corpora: For question mark detection, a large written corpus (approximately 150 million words) was used in conjunction with the BN corpus to overcome the limited size of the broadcast news data for this specific task.

Key Insights and Ideas

  • Prosody is Crucial for Punctuation: The study strongly reinforces the idea that prosodic cues (pitch, energy, duration) are not just helpful but essential for accurate sentence boundary detection and punctuation, even for languages like Portuguese.
  • Language-Specific Nuances Matter: While some prosodic features are language-independent, the specific strategies EP uses for intonational phrases (e.g., epenthetic vowels, segmental elongation, elision) make duration an informative cue in certain contexts.
  • Question Mark Detection is Challenging: Detecting question marks is inherently harder than full stops and commas, often requiring a broader sentence-level context and a combination of diverse features.
  • Interrogative Type Matters: The paper highlights that different types of questions (yes/no, wh-, tag, alternative) have varying frequencies and are better detected by different feature sets. This suggests that a more nuanced approach tailored to interrogative types could further improve performance.

Key Results

  • Full Stop and Comma Improvement: Combining prosodic features with lexical and acoustic cues significantly improved performance for full stop and comma detection. The best results were achieved by combining pitch and energy features at the word level.
  • Question Mark Extension: The introduction of prosodic features for question mark detection led to an absolute increase of 1% in F-measure and SER compared to using only lexical and acoustic features.
  • Feature Importance: Pitch-related features (specifically word-based pitch values, W_P) were found to be the most significant individual prosodic features for full stop and comma detection. Combining pitch and energy (W_PE) yielded the best overall results.
  • Interrogative Distribution: In the BN corpus, yes/no questions constitute the majority (47.0%), followed by wh-questions (40.4%).

Limitations and Future Work

  • Question Mark Performance Gap: While improved, question mark detection performance is still lower than for full stops and commas. The authors suggest that using larger training data and extending the analysis of pitch slopes with discourse functionalities could further enhance this.
  • Corpus Dependency: The authors acknowledge that the weight and effectiveness of features might vary across different corpora and depend on the dominant types of interrogatives present.
  • Future Directions: Future work includes extending the analysis to other corpora to assess feature weight dependency and exploring more sophisticated ways to incorporate discourse functionalities and analyze pitch slopes for better question mark detection. They also aim to investigate the impact of different interrogative subtypes on performance.

Summary updated

BibTeX
@inproceedings{batista2010extending,
  author = {Fernando Batista and Helena Moniz and Isabel Trancoso and Hugo Meinedo and Ana Isabel Mata and Nuno Mamede},
  title = {{Extending the punctuation module for european portuguese}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure