All Together Now: The Living Audio Dataset

David A. Braude, Matthew P. Aylett, Caoimhín Laoide-Kemp, Simone Ashby, Kristen M. Scott, Brian Ó Raghallaigh, Anna Braudo, Alex Brouwer, Adriana StanInterspeech 2019

TL;DR

The "Living Audio Dataset" (LADs) is a new, publicly available, and multilingual dataset designed to address the data scarcity in speech technology research, particularly for low-resource languages. It emphasizes community-driven expansion and provenance tracking, offering a flexible platform for building and improving speech resources beyond traditional, static datasets.

What Problem Does This Paper Address?

Modern speech technology, especially for Text-to-Speech (TTS) and Automatic Speech Recognition (ASR), heavily relies on machine learning, which in turn demands vast amounts of high-quality data. However, existing datasets are often limited in scope, multilingualism, and accessibility, frequently being proprietary, static, or requiring significant technical expertise to contribute to. This creates a "data gap," especially for minority languages or non-standard accents, hindering the development of inclusive speech technologies. The paper tackles the challenge of creating a dynamic, community-expandable dataset that democratizes access to speech resources and encourages contributions from diverse linguistic communities.

Methods

The core of LADs is its emphasis on community involvement and a flexible, expandable format. The dataset is built around several key components:

  • Multilingualism and Community Focus: LADs aims to include data from various languages and accents, with a particular focus on those underserved by current resources. It actively seeks to involve non-technical communities, especially those speaking minority languages or regional dialects.
  • Provenance Tracking: A crucial aspect is the ability to track the origin and evolution of data. This is achieved by documenting how data is collected, curated, and modified, allowing for greater trust and the potential to leverage provenance for more effective crowd-building.
  • Tools for Resource Creation: The paper introduces a suite of Python tools to facilitate the creation of new language resources. These tools help in generating lexicons from sources like Wiktionary, converting IPA pronunciations to specific phonesets, and creating recording scripts from text corpora.
  • Dynamic and Expandable Format: Unlike static datasets, LADs is designed to be "living," meaning it can continuously grow and evolve. This is supported by a clear structure for adding new languages, accents, speakers, and improving existing entries.

Datasets

The paper introduces the Living Audio Dataset (LADs) itself. At the time of publication, LADs contains audio resources, recording scripts, and lexicons for:

  • Dutch (nl)
  • Received Pronunciation English (en)
  • General Irish (ga)
  • Russian (ru)

The dataset is designed to be expanded with new languages and accents. For training and development, the authors mention using existing resources like Wiktionary for lexicon bootstrapping and potentially Wikipedia for text corpora. They also reference established TTS datasets like LJ Speech and CMU Arctic as examples of static datasets that LADs aims to improve upon.

Key Insights and Ideas

  • Democratizing Speech Data: The most significant idea is shifting from top-down data collection to a community-driven, commons-based approach. This empowers communities to build resources for their own languages, fostering inclusivity.
  • Provenance as a Trust Mechanism: Explicitly tracking the origin of data (e.g., hand-annotated by an L1 expert vs. generated by an LTS system) is vital for building trust in crowd-sourced resources. This can also guide future contributions.
  • Linguistic Resources Remain Relevant: Despite the rise of end-to-end deep learning models that can implicitly learn phonetics, the paper argues that traditional linguistic resources like pronunciation lexicons are still crucial, especially for low-resource languages, as they offer a more straightforward way to update and correct pronunciations.
  • "Living" Datasets for Evolving Language: Language is dynamic, and speech datasets should reflect this. LADs's design as a continuously expandable platform is a forward-thinking approach to keep pace with linguistic evolution.

Key Results

The paper primarily focuses on the framework and methodology for creating and expanding the Living Audio Dataset. Specific quantitative results on ASR or TTS performance are not presented, as the paper is about the dataset's creation and infrastructure. However, the key "results" are:

  • Establishment of the LADs framework: A structured approach for collecting, organizing, and expanding multilingual speech data.
  • Development of supporting tools: Python tools are available to assist in lexicon generation, IPA-to-phoneset conversion, and recording script creation.
  • Initial multilingual data: The dataset has initial resources for Dutch, RP English, Irish, and Russian, all in the public domain.
  • A blueprint for community-driven data collection: The paper provides a clear methodology for how other communities or researchers can contribute to LADs.

Limitations and Future Work

Limitations:

  • Initial Data Scarcity: While the dataset is designed for expansion, the initial collection is limited in scope and languages.
  • Tool Usability for Non-Experts: The paper acknowledges that it's "not yet clear how to build the tools for non-experts," suggesting a current reliance on technically literate contributors for initial resource creation.
  • Ethical Considerations: Ensuring volunteers understand the implications of uploading their audio to the public domain is a challenge that requires careful communication.

Future Work:

  • User-Friendly Interfaces: Developing more intuitive interfaces to make it easier for non-technical community members to contribute audio and improve lexicon entries.
  • Enhanced Provenance System: Further developing the provenance system to automatically leverage confidence scores and guide community contributions more effectively.
  • Broader Language Coverage: Expanding the dataset to include a wider range of languages and accents.
  • Integration with Existing Systems: Facilitating the use of LADs data with various TTS and ASR systems beyond Idlak Tangle.
  • Ethical Guidelines: Providing clear information sheets and informed consent processes for data contributors.

Summary updated

BibTeX
@inproceedings{braude2019all,
  author = {David A. Braude and Matthew P. Aylett and Caoimhín Laoide-Kemp and Simone Ashby and Kristen M. Scott and Brian Ó Raghallaigh and Anna Braudo and Alex Brouwer and Adriana Stan},
  title = {{All Together Now: The Living Audio Dataset}},
  booktitle = {Interspeech},
  year = {2019},
}
Figure