Resources for speech research: present and future infrastructure needs

Lou Boves, Rolf Carlson, Erhard Hinrichs, David House, Steven Krauwer, Lothar Lemnitzer, Martti Vainio, Peter WittenburgInterspeech 2009

TL;DR

The CLARIN project aims to build a unified, sustainable infrastructure for language and speech research across Europe, addressing the fragmentation of existing resources and tools. This paper outlines CLARIN's vision to make these resources accessible to humanities and social sciences researchers, highlighting the specific needs and challenges within speech research.

What Problem Does This Paper Address?

The paper addresses the significant challenge of fragmented and inaccessible language and speech resources for researchers in the humanities and social sciences. Currently, valuable data and sophisticated tools are often siloed, difficult to find, and require specialized technical expertise to use. This hinders interdisciplinary research and the exploration of complex linguistic phenomena, especially for less common languages or specialized areas like child-directed speech, accent shifts, and emotional speech. The authors argue that a unified, user-friendly infrastructure is crucial to unlock the full potential of speech research and its applications.

Methods

The paper doesn't detail specific algorithmic methods but focuses on the infrastructure and resource integration approach. CLARIN's method involves:

  • Federation of Existing Archives: Creating a unified access point to diverse digital archives containing text, speech, and multimodal data.
  • Unified Access: Implementing a single sign-on system for researchers to access all federated resources.
  • Web-based Services: Offering standardized web services for language and speech analysis, allowing users to perform operations without deep technical knowledge.
  • Standardization: Working towards agreement on standards for data exchange and interoperability to ensure seamless integration.
  • Experimental Prototype: Developing a prototype to validate the proposed infrastructure and specifications.

Datasets

The paper does not introduce new datasets or benchmarks. Instead, it discusses the need to integrate and make accessible a wide range of existing datasets, including:

  • General Language Resources: Mentioned in the context of the ELDA and LDC for large languages.
  • Humanities-Oriented Speech Data: Examples include child-directed speech, accent shifts, dialogues in different environments, and second language learner data.
  • Specific Examples:
    • Interviews with Jewish immigrants (mentioned as a hypothetical example).
  • A corpus of ten 90-minute Dutch spontaneous conversations (collected by Ernestus).
  • The Spoken Dutch Corpus (CGN).

The paper emphasizes the challenges in acquiring, annotating, and standardizing these diverse datasets, particularly for spontaneous and multimodal speech.

Key Insights and Ideas

  • Speech Research is "Special": The authors argue that speech research faces unique challenges due to the nature of the data and the diverse disciplinary approaches to studying it.
  • The "User-Friendly" Imperative: A core insight is that future infrastructure must be accessible to non-technical users (humanities scholars), moving beyond tools designed solely for engineers.
  • Fragmentation is a Major Bottleneck: The current landscape of scattered resources and tools is a significant impediment to progress.
  • Beyond Technology-Centric Resources: The shift towards content-based processing of speech data, rather than just using it for training ASR/TTS, opens new avenues for humanities and social sciences research.
  • The Importance of Multimodal Data: The growing interest in expressive speech, non-verbal aspects, and multimodal communication highlights the need for integrated data and tools.
  • Sustainability is Key: The CLARIN vision emphasizes creating a sustainable infrastructure, not just a temporary project, requiring long-term funding and governance.

Key Results

The paper is primarily a proposal and vision statement for the CLARIN project, rather than a report on experimental results. Therefore, there are no quantitative results presented. The "results" are conceptual:

  • A clear roadmap for building a unified language and speech research infrastructure.
  • Identification of critical needs and gaps in current resources and tools.
  • A compelling argument for the necessity of CLARIN to advance interdisciplinary research.

Limitations and Future Work

Limitations:

  • Funding Dependency: The success of CLARIN hinges on sustained financial support from funding agencies.
  • Adaptation Effort: Adapting existing tools and resources to common standards and for a non-technical audience is a significant and underestimated challenge.
  • Uneven Resource Coverage: Some languages are severely under-resourced, making it difficult to meet even minimal standards.
  • Complexity of Spontaneous Speech: Handling issues like overlapping speech (cross-talk) in spontaneous conversation corpora requires substantial effort for automatic processing.

Future Work:

  • Construction Phase: Moving from the preparatory phase (2008-2010) to the actual construction of the infrastructure (3-5 years).
  • Populating the Infrastructure: Actively working with researchers and funding agencies to populate CLARIN with necessary resources and tools.
  • Developing New Resources: Creating specialized resources for emerging research areas like emotional and affective speech, and human-human/human-machine interaction.
  • Improving Tool Usability: Making speech tools more robust, user-friendly, and well-documented for a broader research community.
  • Addressing Legal and Ethical Issues: Establishing clear guidelines for access, authentication, IPR, and licensing.

Summary updated

BibTeX
@inproceedings{boves2009resources,
  author = {Lou Boves and Rolf Carlson and Erhard Hinrichs and David House and Steven Krauwer and Lothar Lemnitzer and Martti Vainio and Peter Wittenburg},
  title = {{Resources for speech research: present and future infrastructure needs}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure