WaCadie: Towards an Acadian French Corpus

Jeremy Robichaud, Paul CookCOLING 2024

TL;DR

This paper addresses the lack of readily available digital text for Acadian French, a minority dialect, by constructing three new web corpora. They demonstrate that existing NLP tools perform poorly on Acadian French, highlighting the need for such corpora to improve language technologies for this under-resourced variety.

What Problem Does This Paper Address?

Natural Language Processing (NLP) relies heavily on large, diverse datasets (corpora) to train models for tasks like translation, sentiment analysis, and speech recognition. However, many languages and their regional variations, like Acadian French, lack these essential digital resources. This absence means that off-the-shelf NLP tools, often trained on more common languages or dialects, perform poorly when applied to Acadian French. This paper aims to bridge this gap by creating Acadian French corpora and developing methods to evaluate their "Acadianness," thereby paving the way for better NLP tools for this linguistic community.

Methods

The authors employed a "web-as-corpus" approach, leveraging the internet as a source of text. They utilized three primary methods:

  • BootCaT: This technique uses commercial search engines to query for specific word combinations (seed words) likely to appear in Acadian French text. The resulting URLs are then scraped for content.
  • Domain Crawling: This method involves collecting data from websites within a specific domain, such as .nb.ca (New Brunswick, Canada), which is known to have a high Acadian population. They used the Common Crawl dataset for this.
  • Social Media Scraping: They focused on the Reddit subreddit r/acadie to gather posts and comments, treating each post as a document.

A crucial part of their methodology was developing 22 statistical measures to quantify the "Acadian French" characteristics of a corpus. These measures are derived from linguistic literature on Acadian French, including specific features of its sub-varieties like Brayon and Chiac. They also implemented a robust post-processing pipeline to clean the scraped data, including character encoding normalization, boilerplate removal, and language identification.

Datasets

  • Acadian Dictionary: A corpus created from example sentences in an Acadian French dictionary (Cormier and Wooldridge, 2000). This served as a known Acadian French reference.
  • WiktionaryFR: A corpus created from example sentences in the French Wiktionary, serving as a standard French reference.
  • CC_NB_Domain: A corpus built by domain crawling .nb.ca websites from the Common Crawl dataset.
  • r/acadie: A social media corpus scraped from the Reddit subreddit r/acadie.
  • BootCaT Corpus: A corpus constructed using the BootCaT method with Acadian French seed words.
  • frWaC: A large, publicly available French web corpus (Baroni et al., 2009) used as a general reference corpus for comparison.

Key Insights and Ideas

  • NLP Tools Struggle with Acadian French: Off-the-shelf NLP tools exhibit significantly lower performance on Acadian French compared to standard French, particularly in masked language modeling and part-of-speech tagging. This empirically validates the need for specialized resources.
  • Web-as-Corpus is Viable for Low-Resource Languages: The paper demonstrates that diverse web-scraping techniques can successfully create corpora for under-resourced language varieties like Acadian French.
  • Quantifying "Acadianness": The development of 22 statistical measures provides a novel and systematic way to evaluate how "Acadian" a given text corpus is, drawing on established linguistic features.
  • Social Media as a Rich Source: The r/acadie corpus, despite its smaller size, showed a high concentration of Acadian French characteristics, suggesting social media can be a particularly valuable source for niche dialects.

Key Results

  • NLP Performance Gap: XLM-RoBERTa showed significant differences (p < 0.05) in accuracy for masked word prediction between standard French (WiktionaryFR) and Acadian French (AcadianDictionary). Similarly, POS tagging revealed a higher rate of "X" tags (unknown words) for Acadian French.
  • Corpus Creation Success: All three newly constructed corpora (BootCaT, CC_NB_Domain, and r/acadie) were found to contain Acadian French content based on the developed statistical measures.
  • Corpus Characteristics:
    • The r/acadie corpus exhibited the most significant Chiac measures and the highest number of Acadian characteristics overall, though it was the smallest.
  • The BootCaT corpus was the largest and showed similar results to the Acadian Dictionary reference corpus for many measures.
  • The CC_NB_Domain corpus had many non-significant measures, likely due to its focus on provincial education websites which tend to use standard French.

Limitations and Future Work

  • Limited Current NLP Tools: The study acknowledges that the performance of some measures was impacted by the lack of NLP tools specifically trained on Acadian French. This limitation is also a motivation for the work.
  • Distinguishing Closely Related Varieties: The authors note that their measures might have difficulty distinguishing Acadian French from other closely related French varieties (e.g., Quebec French, Franco-Ontarian French).
  • Future Work:
    • Confirming that the created corpora can indeed improve the performance of off-the-shelf NLP tools when used for training.
  • Investigating the use of Twitter as a social media source, given its geotagging capabilities.
  • Developing measures to differentiate Acadian French from other similar French varieties.

Summary updated

BibTeX
@inproceedings{robichaud2024wacadie,
  author = {Jeremy Robichaud and Paul Cook},
  title = {{WaCadie: Towards an Acadian French Corpus}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2024},
}
Figure