Euskoparl: a speech and text Spanish-basque parallel corpus

Alicia Pérez, José M. Alcaide, María-Inés TorresInterspeech 2012

TL;DR

This paper introduces EuskoParl, a novel parallel corpus of Spanish and Basque speech and text, designed to foster research in under-resourced languages. By mirroring the structure of the well-established Europarl corpus, EuskoParl enables direct comparisons and advancements in machine translation and speech technologies for Basque.

What Problem Does This Paper Address?

Developing advanced language technologies, particularly for minority languages, is heavily reliant on large, high-quality datasets. Many languages, including Basque, are considered "under-resourced," meaning they lack the extensive parallel corpora needed to train robust statistical models for tasks like machine translation and speech recognition. This gap hinders the development of these technologies for these languages, limiting their accessibility and utility. The authors aim to bridge this gap by creating a comprehensive parallel corpus for Spanish and Basque, mirroring the domain and size of the widely used Europarl corpus.

Methods

The paper details a two-pronged approach to corpus acquisition: parallel text and speech.

For parallel text, the authors adapted the acquisition procedure of the Europarl corpus. This involved:

  • Crawling the web to identify and map bilingual data sources.
  • Extracting raw data from the Basque Parliament's proceedings, noting that translations were often done at the paragraph level.
  • Preprocessing the raw text to make it usable for machine learning. This included handling non-printable characters, joining lines within paragraphs, separating punctuation from words, and lowercasing the text while preserving a list of names.
  • Segmenting paragraphs into sentences using an extension of the Thot toolkit, which employs alignment models based on IBM models. This resulted in a significant increase in the number of sentence pairs compared to paragraph pairs.
  • Shuffling the sentence pairs and extracting a test set for evaluation.

For speech data, the acquisition involved collecting speeches in either Spanish or Basque, along with their transcriptions and translations. A key challenge noted is that speakers often switch languages within the same speech segment, requiring bilingual handling of transcription and translation.

Datasets

The primary dataset introduced in this paper is EuskoParl. It is a parallel corpus of Spanish and Basque, comprising both text and speech.

  • Text Data: The corpus is aligned at both paragraph and sentence levels. The training set contains 741,780 paragraph pairs and 771,780 sentence pairs. The test set consists of 30,000 sentence pairs.
  • Speech Data: The speech component includes approximately 189 hours of audio from 81 speakers, with associated transcriptions and translations. The total number of running words in the transcriptions is 1,001,147.

EuskoParl is designed to be comparable in domain and size to the Europarl corpus, specifically the Spanish-English portion, to facilitate direct comparisons and benchmarking.

Key Insights and Ideas

  • The importance of mirroring existing corpora: By designing EuskoParl to be similar in domain and size to Europarl, the authors enable direct comparison of machine translation systems and other language technologies, accelerating progress for under-resourced languages.
  • Challenges of minority language morphology: Basque's highly inflected nature leads to a significantly larger vocabulary and a higher number of singletons (words appearing only once) compared to Spanish. This sparsity poses challenges for machine learning models.
  • Language switching in speech: The natural occurrence of speakers switching languages within a single speech segment presents a unique challenge for language identification and speech processing tasks.
  • Paragraph-level alignment as a starting point: While sentence-level alignment is ideal for machine translation, the initial acquisition at the paragraph level from the Basque Parliament provided a workable foundation, with subsequent segmentation refining the data.

Key Results

  • Corpus Size: EuskoParl contains 771,780 sentence pairs for text and 189 hours of speech.
  • Vocabulary Differences: The Basque vocabulary is 2.3 times larger than the Spanish vocabulary within the corpus.
  • Running Words: Spanish has 1.2 times more running words than Basque.
  • Preliminary Machine Translation: Initial experiments with Moses on the EuskoParl corpus yielded BLEU scores of 11.8 for Spanish-to-Basque and 12.0 for Basque-to-Spanish. These scores are noted as being comparable to those achieved with Europarl when translating into less-resourced languages like Finnish.

Limitations and Future Work

  • Limitations:
    • The authors acknowledge that some paragraphs might not be perfectly aligned due to differences in translation style or document segmentation.
  • The preliminary machine translation results show high Word Error Rates (WER) and relatively low BLEU scores, indicating room for improvement.
  • The large number of Out-Of-Vocabulary (OOV) words in Basque is identified as a significant source of errors in machine translation.
  • The speech data contains unannotated silences (pauses, claps, etc.).
  • Future Work:
    • Refining segmentation and detecting mis-aligned paragraphs using inferred-fertility-based methods.
  • Tackling singleton and OOV words using tools like FOMA for tagging.
  • Further tuning of machine translation models, including exploring more factors and full mert-tuning.
  • Developing speech recognition, speech translation, and acoustic modeling systems.
  • Investigating language identification challenges arising from language switching.
  • Utilizing spell-checking and morphological analysis tools to enhance the corpus.

Summary updated

BibTeX
@inproceedings{perez2012euskoparl,
  author = {Alicia Pérez and José M. Alcaide and María-Inés Torres},
  title = {{Euskoparl: a speech and text Spanish-basque parallel corpus}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure