This paper introduces an extension to a tiny parallel corpus for Griko, an endangered Italo-Greek dialect spoken in Southern Italy. By providing rich annotations and baseline results for speech translation and word discovery, the authors aim to bolster the field of Computational Language Documentation (CLD) and provide a benchmark for extreme low-resource scenarios.
TL;DR
Researchers have released an enriched 20-minute speech corpus of the endangered Griko language, complete with Italian translations and word-level alignments. Their experiments demonstrate that in such extreme low-resource settings, traditional unsupervised alignment methods significantly outperform modern neural attentional models.
What Problem Does This Paper Address?
Linguists estimate that over 50% of the world’s spoken languages may vanish by the year 2100. Many of these are unwritten, making speech the only medium for preservation. While modern AI has revolutionized language processing, these "data-hungry" models typically require thousands of hours of audio.
The CLD community faces a "cold start" problem: how do we build useful computational tools for languages where we only have minutes of data? This paper addresses the lack of publicly available, high-quality datasets for these languages, specifically focusing on Griko—a language with fewer than 20,000 (mostly elderly) speakers.
Methods
The authors took an existing collection of 330 utterances (the UoI corpus) and extended it with several layers of annotation:
- Free-form Italian translations: Created by a bilingual speaker.
- Gold-standard alignments: Manual speech-to-transcription and speech-to-translation alignments.
- Acoustic Unit Discovery (AUD): Using an unsupervised method to generate "pseudo-phones" (machine-discovered speech units) to simulate a scenario where no written script exists.
- ZRC Compatibility: They formatted the data to be compatible with the Zero Resource Challenge (ZRC) metrics, allowing for standardized evaluation of word discovery.
To test the corpus, they evaluated three alignment strategies:
- Proportional Baseline: A naive method that segments audio based on the character length of translation words.
- Neural Alignment: An end-to-end sequence-to-sequence model with attention.
- DTW-EM: An unsupervised model that combines the IBM Model 2 alignment with Dynamic Time Warping.
Datasets
The primary dataset is the Griko-Italian speech translation corpus.
- Size: 330 sentences (~20 minutes of speech).
- Speakers: 9 native speakers (5 male, 4 female).
- Content: Elicited utterances focused on specific morphosyntactic features.
As shown in Figure 1, the Griko and Italian sides of the corpus are remarkably similar in terms of average token length and sentence structure, likely due to centuries of contact between the two languages.
Key Insights and Ideas
- Neural Models Struggle at Small Scales: With only 330 sentences, the neural attentional model failed to learn robust alignments, performing significantly worse than simpler baselines.
- The Power of Language Similarity: Because Griko and Italian share similar structures, the "proportional" baseline—which assumes no reordering—is surprisingly competitive.
- Error Propagation in Zero-Resource Pipelines: Unsupervised Word Discovery (UWD) works reasonably well on text (graphemes), but performance drops sharply when using machine-discovered pseudo-phones. This highlights the difficulty of "cascading" unsupervised systems where errors in unit discovery propagate to word discovery.
Key Results
The quantitative results highlight the gap between traditional and neural methods in this niche:
- Speech-to-Translation Alignment: The unsupervised DTW-EM model achieved the highest F-score (53.8), while the neural model lagged behind at 27.0.
- Word Discovery (Graphemes): The Bayesian model
dpsegwas the clear winner, retrieving 75.1% of correct boundaries (Recall) with a 71.6 F-score. - Word Discovery (Pseudo-phones): Performance was much lower across all methods. The "merged neural" approach, which averages soft-alignment matrices, proved slightly more effective here with an F-score of 30.0.
Limitations and Future Work
The most obvious limitation is the corpus size. While 20 minutes is a realistic reflection of what a field linguist might collect in a short trip, it is an extreme challenge for computational methods.
Additionally, the authors note that their speech-to-grapheme alignment is an approximation (dividing word time-windows equally among graphemes), which means subword discovery results should be interpreted with caution. Future work involves comparing these results against other low-resource corpora to see if these findings generalize across different language families.
Summary updated