TL;DR
This paper introduces MoRoVoc, the largest dataset for analyzing regional variations in spoken Romanian, featuring over 93 hours of audio and comprehensive demographic metadata. They also propose a novel multi-target adversarial training framework that uses demographic attributes (like age and gender) as adversarial targets to improve dialect identification models, achieving significant accuracy gains.
What Problem Does This Paper Address?
Dialect identification in speech processing is crucial for understanding linguistic diversity, but it's particularly challenging for low-resource languages like Romanian, which has historically been underrepresented in speech technology research. Existing datasets are often scarce, limiting the development of robust models. Furthermore, speech models can inadvertently learn to rely on demographic cues (like age or gender) rather than the primary linguistic features, hindering their generalization. This paper aims to address both the lack of a large-scale, well-annotated Romanian dialect dataset and the challenge of building models that are discriminative for dialect while invariant to demographic attributes.
Methods
The core methodological contribution is a multi-target adversarial training framework for fine-tuning pre-trained speech models (like Wav2Vec2). The key idea is to train the model to perform a primary task (e.g., dialect identification) while simultaneously making it "confused" about secondary attributes (e.g., gender, age). This is achieved by using gradient reversal on the losses associated with these secondary attributes.
What's novel here is:
- Multi-target Adversarial Training: Instead of just one adversarial target, they use multiple demographic attributes simultaneously.
- Meta-Learning for Coefficient Adaptation: The influence of each adversarial loss (controlled by coefficients $\lambda_1, \lambda_2$) is dynamically adjusted using a gradient-based meta-learning strategy. This allows the model to learn the optimal balance between task-specific discrimination and attribute invariance without manual tuning.
The framework is applied to Wav2Vec2 models by adding task-specific classification heads. The training objective combines the main task loss with the adversarial losses, with the meta-learning component optimizing the adversarial coefficients.
Datasets
- MoRoVoc: This is the primary contribution – a newly introduced dataset for Romanian spoken dialect identification.
- Size: Over 93 hours of audio, with 88,192 audio samples.
- Content: Balanced between standard Romanian and Moldavian dialects, sourced from parliamentary recordings.
- Annotations: Includes comprehensive demographic metadata (age and gender) for speakers.
- Splits: Disjoint train/validation/test splits are provided, ensuring no speaker overlap.
- RoDia: Mentioned for comparison, it's a smaller dataset for Romanian dialect identification (only 2 hours of annotated speech).
Key Insights and Ideas
- Adversarial Training Benefits Multiple Tasks: The multi-target adversarial approach significantly improves performance not only for dialect identification but also for gender and age classification tasks, demonstrating its broad applicability.
- Model Size Matters for Adversarial Robustness: Larger models (like Wav2Vec2-Large) seem better equipped to handle multiple adversarial constraints simultaneously, achieving higher performance when multiple demographic attributes are used as adversarial targets.
- Dialect Identification is Asymmetric: The error analysis reveals that Romanian speakers are more likely to be misclassified as Moldavian than vice-versa, suggesting subtle linguistic differences or perhaps the influence of regional variations within standard Romanian.
- Formal Speech Attenuates Age Cues: The poor performance on age classification, even with advanced models, suggests that formal parliamentary discourse significantly masks age-distinctive acoustic features, making it difficult for models to infer age from speech alone.
- Meta-Learning is Key for Balancing: The meta-learning approach for adapting adversarial coefficients is crucial for finding the optimal trade-off between discriminative power for the main task and invariance to secondary attributes, a complex optimization problem.
Key Results
- Dialect Identification: Wav2Vec2-Base achieved 78.21% accuracy (a 5.20% improvement over baseline) using gender as an adversarial target. Wav2Vec2-Large reached 77.53% accuracy when both gender and age were adversarial targets.
- Gender Classification: Wav2Vec2-Large achieved 93.08% accuracy when using both dialect and age as adversarial targets.
- Age Classification: Performance was generally lower, but multi-target adversarial training still yielded improvements. For Wav2Vec2-Large, using dialect and gender as adversarial targets resulted in 31.83% accuracy.
- New Models: Experiments with HuBERT and WavLM-Base also showed significant improvements with the proposed adversarial framework, with WavLM-Base achieving the highest dialect identification F1-score of 81.67%.
Limitations and Future Work
- Dataset Bias: The MoRoVoc dataset is exclusively based on formal parliamentary recordings, which may not generalize well to colloquial speech. The demographic distribution is also skewed towards middle-aged and older male speakers.
- Simplified Dialect Landscape: The dataset focuses on a binary dialect classification (standard Romanian vs. Moldavian), which simplifies the richer dialectal landscape of Romania.
- Future Work: The authors suggest diversifying speech registers, improving demographic balance, evaluating performance in various acoustic environments, and exploring the multi-target adversarial framework for other speech recognition tasks beyond dialect identification.
Summary updated