Low Saxon dialect distances at the orthographic and syntactic level

Janine Siewert, Yves Scherrer, Martijn WielingACL 2022

TL;DR

This paper investigates how Low Saxon dialects have evolved orthographically and syntactically between the 19th and 21st centuries, revealing distinct patterns of convergence towards majority languages (German and Dutch). By comparing character and Part-of-Speech (PoS) n-grams, the authors demonstrate that syntactic changes are more pronounced, suggesting a stronger influence of majority language education and media on written dialect over time.

What Problem Does This Paper Address?

Low Saxon, a West Germanic language spoken in parts of Germany and the Netherlands, faces challenges in linguistic research due to significant dialectal variation and the lack of a standardized orthography. While traditional dialectology often focuses on phonological and morphological traits, this paper tackles the question of how orthographic and syntactic levels of Low Saxon dialects have changed over time, particularly in relation to the influence of dominant languages like German and Dutch. The authors aim to understand if and how these dialects are converging towards their respective majority languages, and whether this convergence is more evident at the orthographic or syntactic level. This is crucial for understanding language change, dialectology, and for developing computational linguistic tools for under-resourced languages.

Methods

The study employs a comparative approach using two primary methods for analyzing dialectal similarity:

  • Character n-grams: The researchers extract character bigrams and trigrams from tokenized and lower-cased text. This method captures orthographic patterns and can indirectly reflect phonological and morphological features.
  • Part-of-Speech (PoS) n-grams: They utilize PoS tag sequences to analyze syntactic structures. This involves automatically tagging the text using a Stanza tagger trained on various languages, including manually annotated Low Saxon data.

For both methods, the raw n-gram counts are normalized using TF-IDF. To visualize and analyze dialect distances, Principal Component Analysis (PCA) is used to reduce the data to two dimensions, followed by k-means clustering. The results are also compared with hierarchical clustering.

Datasets

The primary dataset used is the LSDC (Low Saxon Dialect Corpus), which is annotated for dialect and century. This dataset was supplemented with relevant prose texts from:

  • Leopold and Leopold (1882) for 19th-century data.
  • The Twentse Taalbank (van der Vliet, 2021) for 19th-century data.

The study focuses on five Low Saxon dialects: German North Saxon (DNS), Dutch North Saxon (NNS), German Westphalian (DWF), Dutch Westphalian (NWF), and Eastphalian (OFL), analyzed for both the 19th and 21st centuries. For comparison, they also used Universal Dependencies (UD) data for Standard German and Standard Dutch.

Key Insights and Ideas

  • Orthographic vs. Syntactic Convergence: The study reveals that syntactic (PoS n-gram) distances show a clearer convergence of Low Saxon dialects towards their respective majority languages (Dutch and German) compared to orthographic (character n-gram) distances.
  • Time as a Factor: The 21st-century Low Saxon dialects appear closer to each other syntactically than their 19th-century counterparts, suggesting that syntactic structures are more susceptible to external influence over time.
  • Country Borders Matter Orthographically: Character n-gram analysis shows a tendency for dialects to cluster according to national borders (Germany vs. Netherlands), indicating that orthographic practices might be more strongly influenced by national educational systems.
  • Dutch Low Saxon Adapts More: Dutch Low Saxon dialects show a greater tendency to conform to the principles of Dutch orthography in the 21st century compared to their 19th-century counterparts, a trend not as clearly observed for German Low Saxon.
  • Surprising Closeness of Majority Languages: The relatively close clustering of Standard Dutch and Standard German in the PoS-based analysis was unexpected, given the expectation of greater divergence.

Key Results

  • Character n-grams: In a two-cluster scenario, dialects group by country borders. German Low Saxon clusters with Standard German, albeit at a distance. Dutch Low Saxon shows increased closeness to Standard Dutch in the 21st century.
  • PoS n-grams: Clustering into three groups separates 19th-century Low Saxon, 21st-century Low Saxon, and the majority languages. When clustered into two groups, Low Saxon dialects form one cluster, and Dutch/German form another. The syntactic distance between Dutch Low Saxon and Standard Dutch is comparable to that between German Low Saxon and Standard German.
  • Internal Dialect Convergence: Within their respective countries, some dialects show increased closeness over time. For instance, Dutch North Saxon and Dutch Westphalian appear to have approached each other, as have German North Saxon and Eastphalian.

Limitations and Future Work

  • Data Availability: The study acknowledges the lack of annotated data for Mecklenburgish-West-Pomeranian, Brandenburgish-South-Marchian, and Low Prussian for the 20th and 21st centuries, which prevents diachronic comparisons for these regions. Data from only two dialects for the 20th century also limited its inclusion.
  • PoS Tagger Accuracy: The PoS tagger achieved an accuracy of around 85%, which the authors consider acceptable but note that higher accuracy would be beneficial. Morphological tagging accuracy is currently too low (60-70%) for inclusion.
  • Genre Influence: While the authors suggest genre differences might play a role in the observed distances, they deem it less likely as the data primarily consists of fiction texts. However, they note that the comparison with modern Dutch and German data is restricted due to the likely more formal nature of those datasets.
  • Future Research: The authors plan to include more Low Saxon dialects (especially eastern ones), incorporate 20th-century data, and improve morphological tagging. They also intend to investigate which features are most central in dimensionality reduction and explore finer-grained dialect subdivisions. The goal is to facilitate the development of NLP tools and research for Low Saxon.

Summary updated

BibTeX
@inproceedings{siewert2022low,
  author = {Janine Siewert and Yves Scherrer and Martijn Wieling},
  title = {{Low Saxon dialect distances at the orthographic and syntactic level}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure