Abstract
The majority of current Text-to-Speech (TTS) datasets, which are collections of individual utterances, contain few conversational aspects. In this paper, we introduce DailyTalk, a high-quality conversational speech dataset designed for conversational TTS. We sampled, modified, and recorded 2,541 dialogues from the open-domain dialogue dataset DailyDialog inheriting its annotated attributes. On top of our dataset, we extend prior work as our baseline, where a non-autoregressive TTS is conditioned on historical information in a dialogue. From the baseline experiment with both general and our novel metrics, we show that DailyTalk can be used as a general TTS dataset, and more than that, our baseline can represent contextual information from DailyTalk.
Introduction
Standard Text-to-Speech (TTS) systems are typically trained on datasets consisting of isolated, individual utterances, such as LJSpeech. While these models excel at reading single sentences clearly, they often fail in conversational settings where prosodic flow, emotional context, and filler words (e.g., "uh", "um") are crucial. Existing conversational datasets are often either proprietary or of low audio quality. To bridge this gap, we present DailyTalk, a clean, studio-quality, and publicly available benchmark for conversational TTS.
The DailyTalk Dataset
DailyTalk consists of approximately 20 hours of studio-recorded speech across 2,541 dialogues. The scripts were derived from the text-based DailyDialog dataset, refined to ensure dyadic interactions between one male and one female speaker.
The dataset includes annotations for emotion, speech acts, and topics. A key feature is the inclusion of "filling-gap" words (fgs) in about half the dialogues to mimic natural human hesitation and rhythm.
Context-Aware Baseline Model
To demonstrate the utility of DailyTalk, we developed a baseline architecture based on FastSpeech2. The model is enhanced with a Context Encoder that processes previous dialogue history, allowing the system to be "context-aware."
We also replaced the traditional Montreal Forced Aligner with a CTC-based aligner. This allows the model to learn phoneme durations in an end-to-end manner, which proves more robust for the expressive and varied nature of conversational speech.
Key Results and Insights
Our evaluation focused on two main areas: dataset integrity and conversational performance.
Dataset Integrity
DailyTalk achieved a Mean Opinion Score (MOS) of 3.82, which is comparable to the gold-standard LJSpeech (3.85). This confirms that DailyTalk is high-quality enough to serve as a general-purpose TTS dataset.
Conversational Superiority
In pairwise MOS tests evaluating adjacent utterances, the context-aware baseline scored 3.90, outperforming the standard FastSpeech2 (3.86). Interestingly, the benefit of context becomes more pronounced as a conversation progresses. Analysis of dialogues after the fifth turn showed significant gains in naturalness for the context-aware model, whereas standard models often saw performance degradation as complexity increased.
Conclusion
DailyTalk provides the research community with a high-quality, open-source resource for advancing conversational TTS. By incorporating dialogue history and natural speech features like fillers, we demonstrate that models can achieve significantly higher naturalness in multi-turn interactions. Future work will look into increasing speaker diversity and exploring more spontaneous linguistic patterns across different cultures.
Summary updated