Introduction
As digital interaction shifts from screen-based search to voice-only assistants like Alexa or Google Home, a significant user experience challenge emerges: the loss of visual cues. On a screen, search engines can bold an "answer" within a text snippet, allowing users to skim and identify relevant information instantly. In a voice-only setting, communication is serial; users must listen to the entire response, placing a heavy burden on short-term memory and cognitive processing. This paper explores how modifying speech synthesis—using pauses, pitch shifts, and rate changes—can act as "audio bolding" to help users identify key information more effectively.
Methodology
The researchers proposed a framework to evaluate "prosodic modifications" using two Text-to-Speech (TTS) engines: IBM Watson and Google Wavenet. They tested four specific modifications applied to the "answer key" within a full sentence, derived from the SQuAD (Stanford Question Answering Dataset):
- Pause: Inserting silence before and after the key phrase.
- Rate: Slowing down the speaking speed for the key phrase.
- Pitch: Raising the pitch of the key phrase.
- Emphasis: A combination of increased pitch and slower rate (utilizing Google’s built-in
<emphasis>tag).
Evaluation was conducted via crowdsourcing, where workers provided feedback on informativeness (answering quality), elocution (pronunciation), interruption (unwanted breaks), and correctness (the ability to accurately identify and transcribe the highlighted answer).
Key Results and Insights
The study treats prosody not merely as a tool for naturalness, but as a functional UI element akin to CSS styling. The results indicate that users are willing to trade some naturalness for increased utility.
- The "Emphasis" Advantage: Google’s
emphasistag, which blends pitch and rate changes, was the most effective modification. It showed a +0.87 increase in informativeness and a +0.28 increase in correctness over standard speech baselines. - Context and Length Sensitivity: Highlighting techniques are context-dependent. For instance, pauses significantly improved informativeness for short answers (+1.09) but were perceived as unnatural and frustrating for longer phrases, where informativeness dropped by -0.97.
- Positional Effects: Modifications were most effective when the answer appeared toward the end of the sentence, likely because it reduced the cognitive load required to retain the answer until the audio finished.
Discussion and Future Work
While prosodic modifications often increased "interruption" scores (making the voice sound less natural), users consistently rated them as more useful for information digestion. This suggests that for task-oriented voice interactions, functional clarity can outweigh human-like prosody.
Future research should focus on optimizing the "strength" of these modifications dynamically based on answer length or sentence complexity. Additionally, as this study was limited to English, further investigation is required to determine how "audio bolding" rules translate to tonal languages or different linguistic contexts where prosodic cues carry different meanings.
Summary updated