NoMusic - The Norwegian Multi-Dialectal Slot and Intent Detection Corpus

Petter Mæhlum, Yves ScherrerNAACL 2024

TL;DR

This paper introduces NoMusic, a novel multi-dialectal corpus of written Norwegian designed for spoken language understanding (SLU) tasks like slot and intent detection. By translating an existing English SLU dataset into Norwegian Bokmål and eight distinct dialects, NoMusic provides a unique resource for evaluating SLU system robustness across linguistic variations and for studying dialectal differences in digital communication.

What Problem Does This Paper Address?

Developing robust spoken language understanding (SLU) systems, particularly for tasks like slot and intent detection, often relies on large, diverse datasets. While many resources exist for major languages, low-resource languages and their dialects are significantly underrepresented. This gap is particularly pronounced for Norwegian, which has two official written standards (Bokmål and Nynorsk) and a rich landscape of spoken dialects. Existing Norwegian dialect resources are often transcriptions of interviews, which can be difficult to compare and may not reflect natural written dialect usage. Furthermore, evaluating SLU systems on non-standard varieties is crucial for ensuring equitable access to voice-based technologies. This paper addresses the need for a comparable, natively written, multi-dialectal dataset for Norwegian SLU tasks.

Methods

The creation of the NoMusic corpus involved a multi-phase process:

  1. Translation: English sentences from the xSID dataset (a benchmark for SLU) were translated into standard Norwegian Bokmål and into eight Norwegian dialects. These translations were performed by university students who self-identified as regular writers of their respective dialects.
  2. Annotation Transfer: The slot and intent annotations from the original English xSID dataset were semi-automatically transferred to the Norwegian translations. This involved annotating the Bokmål translations first, using the English sentences as a guide, and then annotating the dialectal translations, using the Bokmål annotations as a guide.
  3. Annotation Interface: The INCEPTION platform was used for the annotation process, facilitating the interleaving of annotated English sentences with their Norwegian translations to guide the annotators.

Datasets

  • xSID Corpus: The English validation and test sets of the xSID corpus served as the source material for translation. This corpus is designed for slot and intent detection.
  • NoMusic Corpus: This is the newly introduced benchmark. It consists of annotated translations of the xSID sentences into standard Norwegian Bokmål and eight Norwegian dialects. It is a multi-parallel corpus, meaning all translations share the same meanings and sentence structures as the original.

Key Insights and Ideas

  • Natively Written Dialect Data: NoMusic is a valuable resource because it comprises natively written dialectal data, rather than transcriptions of spoken language. This better reflects how people actually write their dialects in everyday digital communication.
  • Multi-Dialectal Comparability: The multi-parallel nature of the corpus, where each dialect translates the same set of English sentences, allows for direct comparison of linguistic variations across dialects and with the standard language.
  • Dialectal Variation in SLU: The dataset highlights that dialectal variations can manifest in various linguistic aspects, including spelling, verb morphology (e.g., infinitive endings), nominal morphology (gender and plural forms), and the use of function words (pronouns, determiners). These variations can pose challenges for SLU systems trained on standard language.
  • Translation Challenges: The study reveals that even with guidelines, translators may exhibit different strategies for handling time expressions, named entities, and grammatical nuances, leading to variations that are not always strictly dialect-specific but rather reflect individual writing habits.

Key Results

  • The NoMusic corpus contains translations of 300 development sentences and 500 test sentences from the xSID dataset.
  • Eight Norwegian dialects were represented, covering three of the four major Norwegian dialect areas. Eastern Norwegian dialects were notably absent.
  • Analysis of the translations revealed variations in spelling, verb and nominal morphology, and the use of function words. For instance, different dialects showed distinct patterns in infinitive endings (-a, -e, -Ø) and the use of definite markers.
  • Lexical overlap between dialect translations and Bokmål was analyzed, showing moderate to strong correlations with geographical distance, though p-values were too high for definitive conclusions.

Limitations and Future Work

  • Geographical Coverage: The corpus lacks representation from Eastern Norwegian dialects, which the authors attribute to linguistic factors and the availability of qualified student translators. Future work may involve extending the corpus to cover these areas.
  • Non-Professional Translators: The annotators were students, not professional translators. This might have led to difficulties in producing perfectly natural dialect writing in a translation context and potential unfamiliarity with certain cultural references.
  • Written vs. Spoken: The current corpus is written. The authors suggest that pairing these transcripts with recorded speech would be beneficial for developing SLU systems that handle both modalities.
  • Harmonization of Annotation Standards: The paper notes that the slot and intent label sets differ between xSID and other Norwegian SLU datasets like MASSIVE, suggesting future work on harmonizing these standards.
  • Ethical Considerations: Translators were compensated for their work, participation was voluntary, and all agreed to share their productions under a CC-BY-SA 4.0 license. The source English data was curated to avoid harmful content.

Summary updated

BibTeX
@inproceedings{mhlum2024nomusic,
  author = {Petter Mæhlum and Yves Scherrer},
  title = {{NoMusic - The Norwegian Multi-Dialectal Slot and Intent Detection Corpus}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
}
Figure