Abstract
Early text-to-speech (TTS) systems often produced robotic, repetitive prosody by relying solely on local linguistic features like part-of-speech tags. This paper, published by Julia Hirschberg at AT&T Bell Laboratories in 1990, presents a foundational shift toward discourse-aware synthesis. By introducing NewSpeak, a system that integrates attentional models of discourse, Hirschberg demonstrates how tracking "given" versus "new" information allows for more natural pitch accent assignment, effectively reducing the unnatural over-accentuation characteristic of early synthetic speech.
Introduction
In natural human communication, speakers use pitch accents—local variations in the fundamental frequency ($f_0$)—to highlight salient information. However, the decision of which words to emphasize is not merely a matter of grammar; it is deeply rooted in the context of the conversation. Traditional TTS systems in the late 1980s were largely "mind-blind," assigning accents based on simple dictionary categories. Hirschberg’s work addresses this limitation by proposing that a system must "remember" the discourse history to make intelligent prosodic choices.
The Problem of Prosodic Monotony
The primary challenge in speech synthesis is avoiding the repetitive accentuation of information that the listener already knows. For example, if a speaker mentions "a red house" followed by "the big house," a human narrator will typically de-emphasize the word "house" in the second instance because it is "given" information.
Standard TTS systems of the era relied on a binary distinction: "content words" (nouns, verbs, adjectives) were accented, while "function words" (articles, prepositions) were not. This approach fails to account for discourse flow, leading to a "shouty" or clunky delivery where every noun is treated as a new revelation, regardless of how many times it has been mentioned.
The NewSpeak Architecture
To solve this, Hirschberg developed NewSpeak, an interface for the Bell Laboratories TTS system. NewSpeak’s core innovation is its implementation of the "Attentional Structure" theory proposed by Grosz and Sidner. The system maintains a Focus Stack to track the discourse state:
- Focus Spaces: When a new topic or paragraph begins, a new space is pushed onto the stack, containing the roots of open-class words mentioned.
- Global vs. Local Focus: The system distinguishes between items mentioned at the start of a text (defining the overall topic) and items mentioned recently in the current paragraph (local context).
- The De-accentuation Rule: NewSpeak checks the focus stack before assigning an accent. If a word’s root is already present in the stack, it is marked as "given" and de-accented, even if it is a noun or verb.
- Structural Cues: The system uses orthographic markers, such as paragraph breaks, to "pop" the focus stack, mimicking the way humans "reset" their prosodic expectations when transitioning to a new topic.
Empirical Validation and Findings
Hirschberg validated the NewSpeak model using the FM Radio Newscasting Database, a corpus of professionally recorded and prosodically labeled newscasts from WBUR (Boston). By comparing the system’s output against human ground truth, several key insights emerged:
- Givenness is Not Binary: Simply being previously mentioned does not always warrant de-accentuation. Humans frequently continue to accent proper names (e.g., "Hennessy") to maintain the listener's focus on a specific entity.
- The Importance of Negatives: While most closed-class words are de-accented, negative markers (e.g., "no," "not") and negative modals (e.g., "can't") must be accented. De-accenting a negative can fundamentally alter or flip the intended meaning of a sentence for the listener.
- Refined Word Classes: The research moved beyond the "open vs. closed" dichotomy, categorizing words into three functional groups: Closed-Deaccented (articles), Closed-Accented (negatives and logical connectors like "ergo"), and Open (standard content words).
Results and Impact
NewSpeak successfully reduced the density of accents in long-form synthetic speech, resulting in a voice that sounds more like a narrator who understands the narrative arc rather than a machine reading a list of words. Beyond improving TTS quality, the paper proposed that pitch accent placement could serve as an objective metric for evaluating the accuracy of any discourse model’s ability to track conversational flow.
Conclusion
While Hirschberg acknowledged limitations—such as the lack of "intentional" modeling (why a speaker says something) and the scarcity of large prosodic datasets—this work remains a cornerstone of speech science. It proved that for synthetic speech to be truly human-like, it must possess a functional memory of the discourse context, ensuring that emphasis is placed not just on what is being said, but on what is truly new.
Summary updated