Abstract
This paper presents two unsupervised approaches for acoustic unit discovery in the context of the ZeroSpeech 2019 Challenge. By incorporating temporal continuity priors into a Self-Organizing Map (SOM) and a Correspondence Recurrent Sparse Autoencoder (CoRSA), the proposed methods effectively extract sub-word units from raw audio without textual supervision. Experimental results on the Indonesian "surprise" language demonstrate significant improvements in signal discriminability (ABX error) and data compression (bitrate) compared to the challenge baseline, highlighting the importance of temporal awareness in unsupervised speech processing.
Introduction
Modern Automatic Speech Recognition (ASR) systems typically require thousands of hours of transcribed audio. This "data-hungry" nature poses a significant barrier for the majority of the world's 7,000+ languages, many of which are low-resource or lack a standard writing system. The ZeroSpeech 2019 Challenge addresses this by tasking researchers to build speech systems using zero text labels, necessitating the automatic discovery of acoustic units (e.g., phonemes) directly from raw audio. A primary challenge in this domain is that standard clustering techniques often ignore the temporal sequentiality of speech, where neighboring frames are highly likely to belong to the same acoustic unit.
Methodology
The authors propose two distinct architectures designed to prioritize temporal awareness during the discovery process.
SOM-RNN Hybrid
The first approach utilizes a Temporally Biased SOM. Unlike standard SOMs that cluster individual frames, this model updates clusters using a weighted average of neighboring frames, ensuring that the resulting units represent stable acoustic events. To further refine these units, "Unit Pooling" is employed to map context-dependent sounds into broader, context-independent categories. Finally, a Gated Recurrent Unit (GRU) network is trained to predict these pooled labels from raw audio features.
Correspondence Recurrent Sparse Autoencoder (CoRSA)
The second approach, CoRSA, leverages a GRU-based autoencoder to compress speech into a bottleneck layer.
Key components of CoRSA include:
- Sparsity Constraints: A negative L2-norm penalty forces the network to select a single "unit" at each time step, resulting in a sequence of one-hot vectors.
- Correspondence Training: The model is fine-tuned using pairs of similar audio segments identified via Unsupervised Term Discovery (UTD). By learning to reconstruct one segment from its "twin," the model learns to ignore speaker-specific noise and focus on linguistic content.
- Temporal Filtering: A median filter is applied to the output to smooth state assignments, which significantly reduces the bitrate by eliminating high-frequency "jitter."
Experimental Results
The models were evaluated on the ZeroSpeech 2019 datasets, including English (development) and Indonesian (surprise test language). Performance was measured using the ABX error rate (discriminability) and Bitrate (compression).
As shown in the results, the CoRSA system achieved an ABX error of 10.87% on the Indonesian test set, a substantial improvement over the 27.46% baseline. Furthermore, the CoRSA system achieved a bitrate of 29.46, which is lower than both the baseline (74.55) and the supervised "Topline" system (35.20). While the discovered units proved highly discriminative, the Mean Opinion Score (MOS) for synthesized speech remained comparable to the baseline, indicating that while unit discovery has improved, high-quality speech reconstruction remains a challenge.
Conclusion
The integration of temporal continuity as a prior—whether through biased SOM updates or post-processing filters—is crucial for effective acoustic unit discovery. The CoRSA approach, in particular, demonstrates that correspondence learning and temporal smoothing can yield units that are both highly discriminative and efficient. Future work will focus on integrating these units with advanced neural vocoders to improve synthesis quality.
Summary updated