TL;DR
This paper introduces Quantized Contour Modeling (QCM), a novel Bayesian approach for classifying prosodic events like pitch accents and phrase endings in speech. QCM quantizes pitch and intensity contours into discrete bins, creating a low-dimensional representation that can be modeled effectively, achieving promising results on standard corpora.
What Problem Does This Paper Address?
Understanding the nuances of spoken language goes beyond just the words themselves. Intonation, the rise and fall of a speaker's pitch, carries crucial information about meaning, emphasis, and sentence structure. For instance, a rising pitch at the end of a sentence can signal a question, while a steeper pitch rise can indicate contrast. Automatically detecting and classifying these prosodic events (like pitch accents and phrase boundaries) is vital for natural language understanding systems to grasp this rich layer of meaning, which is otherwise lost in transcribed text. The ToBI (Tones and Break Indices) framework provides a standard for describing these intonational contours, but automatically classifying the specific types of tones and their associated events remains a challenge.
Methods
The core innovation here is Quantized Contour Modeling (QCM). The authors propose a two-step process:
- Quantization: The acoustic contour (specifically, pitch (f0) and optionally intensity) of a prosodic event is first normalized. The pitch is scaled to a range between its minimum and maximum values within the contour, and then divided into $N$ equally sized bins. Similarly, the time dimension of the contour is normalized to [0,1] and divided into $M$ equally sized bins. This creates a grid-like representation of the contour.
- Bayesian Classification: For each prosodic event type (e.g., a specific pitch accent), a multinomial mixture model is trained on these quantized contours. The model learns the probability distribution of quantized values within each time bin for a given event type. Classification then uses a Bayesian approach to determine the most likely event type for a new contour based on these learned models.
The authors explore several variations of QCM:
- Standard Shape Modeling: Models each time bin independently.
- Sequential f0 Modeling: Incorporates a Markov assumption, modeling the current quantized value based on the previous one, capturing temporal dependencies.
- Standard f0 + I Modeling: Extends the standard model to include intensity contours.
- Sequential f0 + I Modeling: Combines sequential modeling with intensity.
Datasets
The experiments were conducted on two corpora:
- Boston University Radio News Corpus (BURNC): A corpus of professionally read radio news, prosodically annotated according to the ToBI standard. The paper uses a 2.35-hour subset from six speakers.
- Boston Directions Corpus (BDC): Consists of elicited monologues. The paper evaluates two sub-corpora:
- BDC-read: 50 minutes of read speech.
- BDC-spon: 60 minutes of spontaneous speech.
Key Insights and Ideas
- Quantization as a powerful representation: Discretizing continuous acoustic contours into a fixed grid effectively captures the essential shape information for classification, creating a low-dimensional representation.
- Sequential modeling helps, but not always: Incorporating temporal dependencies (sequential modeling) improved classification accuracy for pitch accents, especially on the BDC-read corpus. However, its impact on Combined Error Rate (CER) was not always statistically significant across all datasets.
- Intensity modeling is tricky: Simultaneously modeling pitch and intensity contours did not consistently improve pitch accent classification. While it showed promise for phrase-ending tone classification, the interaction between pitch and intensity for these events is noted as an under-researched area.
- Combined Error Rate (CER) is a better metric: For imbalanced datasets (where one class dominates, like H* pitch accents), accuracy can be misleading. The authors advocate for CER, which penalizes both false positives and false negatives more equitably, providing a more informative evaluation, especially for minority classes.
Key Results
- Pitch Accent Classification: On the BURNC corpus, QCM achieved 63.99% accuracy and a .4481 Combined Error Rate (CER) using sequential f0 modeling.
- Phrase-Ending Tone Classification: On the BURNC corpus, QCM achieved 72.91% accuracy using sequential f0 and intensity modeling.
- Comparison to Baseline: The results represent significant improvements over a majority class baseline, which is high due to class imbalance.
- Intensity's limited role: Modeling intensity alongside pitch did not improve pitch accent classification accuracy but showed some benefit for phrase-ending tones.
Limitations and Future Work
- Reliance on manual annotations: The current QCM approach assumes that prosodic events (pitch accents and phrase boundaries) are already detected. The accuracy of the classification is thus dependent on the quality of these prior detections.
- Limited improvement from intensity: The paper notes that the interaction between pitch and intensity for phrase-ending tones is not well understood and warrants further investigation.
- Future Directions: The authors suggest that QCM could be applied to other prosodic phenomena like turn-taking behavior or dialog acts. They also propose its direct application to lexical tone modeling in tonal languages, requiring only a different segmentation routine. The potential for using QCM posteriors as features in other classifiers (like SVMs) is also highlighted.
Summary updated