Word Informativeness and Automatic Pitch Accent Modeling

Shimei Pan, Kathleen R. McKeownEMNLP 1999 · OpenReview

Abstract

In intonational phonology and speech synthesis research, it has been suggested that the relative informativeness of a word can be used to predict pitch prominence. The more information conveyed by a word, the more likely it will be accented. But there are others who express doubts about such a correlation. In this paper, we provide some empirical evidence to support the existence of such a correlation by employing two widely accepted measures of informativeness. Our experiments show that there is a positive correlation between the informativeness of a word and its pitch accent assignment. They also show that informativeness enables statistically significant improvements in pitch accent prediction. The computation of word informativeness is inexpensive and can be incorporated into speech synthesis systems easily.

Introduction

Natural human speech is characterized by "pitch accents"—variations in pitch, tempo, or rhythm that highlight specific words. For Text-to-Speech (TTS) systems to achieve human-like naturalness, they must accurately predict which words should be accented. Traditionally, these systems have relied on Part-of-Speech (POS) tags, operating on the heuristic that content words (like nouns) are more likely to be accented than function words (like prepositions).

However, POS tags alone fail to capture the "semantic weight" of a word. For instance, in the phrases "He killed a man" vs. "He killed a policeman," the latter word carries more specific information and is thus more likely to receive stress. This paper explores "word informativeness" as a computational proxy for this semantic weight.

Quantifying Informativeness

The study evaluates two statistical metrics to quantify informativeness:

  1. Information Content (IC): Defined as the negative log-likelihood of a word's occurrence ($IC(w) = -\log(P(w))$). Rarer words are considered more informative.
  2. TF*IDF (Term Frequency * Inverse Document Frequency): A metric that balances a word's frequency in a specific document against its commonality across a broader corpus.

These metrics were calculated using a large medical text corpus (1.24 million words), while pitch accent labels were derived from a hand-labeled speech corpus of medical briefings.

Correlation and Predictive Power

A primary goal was to determine if a statistically significant link exists between these metrics and actual pitch accents. The analysis revealed a strong, positive correlation for both IC and TF*IDF.

Correlation between informativeness features and pitch accent assignment.

As shown in the table above, the correlation coefficients ($\rho$) are significant, with extremely low p-values, providing empirical backing for the linguistic theory that informativeness drives prominence.

Performance Improvements

To test the practical utility of these measures, the authors integrated them into two machine learning models: RIPPER (rule-based) and Hidden Markov Models (HMM).

The baseline performance—simply accenting every word—yielded 52.02% accuracy. Using Information Content (IC) alone, the HMM model's accuracy jumped to 71.96%. Crucially, informativeness was found to be complementary to POS tags. While POS tags provide a structural baseline, IC captures nuances of rarity and importance that structural tags miss.

Performance improvement by combining Information Content (IC) with Part-of-Speech (POS) tags.

The combination of POS and IC features (POS+IC Model) achieved the highest performance, reaching 74.06% accuracy. This outperformed both the standalone POS model and the standalone IC model, demonstrating that informativeness provides unique predictive value. This integrated model also outperformed industry-standard TTS systems of the time, which achieved roughly 71.75% accuracy on the same medical corpus.

Conclusion

This research demonstrates that word informativeness, specifically Information Content, is a powerful and computationally inexpensive predictor of pitch accent. By incorporating these statistical measures, TTS systems can move beyond rigid POS-based rules toward more dynamic, context-aware prosody. Future work will focus on integrating these measures with discourse-level features to account for how "given" versus "new" information affects stress patterns in continuous conversation.

Summary updated

Abstract

In intonational phonology and speech synthesis research, it has been suggested that the relative informativeness of a word can be used to predict pitch prominence. The more information conveyed by a word, the more likely it will be accented. But there are others who express doubts about such a correlation. In this paper, we provide some empirical evidence to support the existence of such a correlation by employing two widely accepted measures of informativeness. Our experiments show that there is a positive correlation between the informativeness of a word and its pitch accent assignment. They also show that informativeness enables statistically signican t improvements in pitch accent prediction. The computation of word informativeness is inexpensive and can be incorporated into speech synthesis systems easily.
BibTeX
@inproceedings{pan1999word,
  author = {Shimei Pan and Kathleen R. McKeown},
  title = {{Word Informativeness and Automatic Pitch Accent Modeling}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {1999},
  url = {https://openreview.net/forum?id=BkWeIMzObr},
}
Figure