Statistical and Neural Methods for Hawaiian Orthography Modernization

Jaden Kapali, Keaton Williamson, Winston WuEMNLP 2025

TL;DR

This paper tackles the challenge of converting between two distinct Hawaiian orthography systems, a task crucial for language revitalization. Surprisingly, simple statistical n-gram models outperformed complex neural networks and large language models, highlighting the effectiveness of traditional methods in low-resource settings.

What Problem Does This Paper Address?

The Hawaiian language, ‘Ōlelo Hawai‘i, has two primary written forms: an older "Traditional Spelling" (TS) and a newer "Modern Spelling" (MS). The TS system, developed in the 1820s, lacked explicit markers for glottal stops and long vowels, leading to ambiguity. The MS system, refined in the mid-20th century, introduced the ‘okina (glottal stop marker) and kahakō (long vowel marker), creating a more one-to-one mapping between spelling and sound. This modernization was vital for preserving pronunciation and aiding language learning.

However, the existence of both systems creates a challenge. While MS is preferred by many fluent speakers, some communities, particularly on Ni‘ihau where the older dialect persists, find TS easier to read due to its visual simplicity and familiarity with older texts like the Hawaiian Bible. This necessitates methods to convert between the two orthographies to improve text intelligibility and support language revitalization efforts. The paper aims to develop and evaluate computational approaches for this conversion task.

Methods

The authors explored several methods for converting Hawaiian text from Traditional Spelling (TS) to Modern Spelling (MS).

  • Deterministic Conversion (MS to TS): This was straightforward: remove all ‘okina and replace kahakō-marked vowels with their plain counterparts.
  • N-gram Models (TS to MS): This was the core of their statistical approach. They used Bayes' Rule to find the most likely MS variant for a TS word based on its left context. They experimented with n-gram orders from 1 to 5, using unigrams (most common variant) as a baseline. They also leveraged pre-computed n-gram counts from the HawCorpus.
  • Sequence-to-Sequence (Seq2Seq) Models (TS to MS): Framing the task as machine translation, they employed two popular encoder-decoder architectures: a 2-layer LSTM and a 6-layer Transformer. They experimented with different vocabulary sizes for tokenization (character-level vs. subword) to see their impact.
  • Large Language Models (LLMs) (TS to MS): They tested few-shot prompting with Llama 3.2-1b and Llama 3.1-8b, and finetuned Qwen3-4b-instruct-2507. They carefully designed prompts to elicit only the converted text.

Datasets

The primary dataset used was audio transcripts from "Kani‘aina," a collection of radio and TV programs recorded in the 1970s-1980s. These transcripts, originally in Modern Spelling (MS), were preprocessed to remove English sentences and boilerplate text. To create the Traditional Spelling (TS) data for training, they deterministically removed all ‘okina and kahakō diacritics from the MS text. This resulted in a dataset of 76,980 sentences, split into 80% training, 10% validation, and 10% testing. They also utilized n-gram counts from the HawCorpus for some experiments.

Key Insights and Ideas

  • Simplicity Wins: The most striking finding is that traditional statistical n-gram models, particularly trigrams and 4-grams, significantly outperformed complex neural models (LSTM, Transformer) and even large language models (LLMs) on this task. This challenges the common assumption that neural methods are always superior, especially in low-resource scenarios.
  • Context is King (for N-grams): Increasing the order of n-grams improved performance, demonstrating that local context is crucial for disambiguating TS words into their correct MS forms. However, gains plateaued after 4-grams.
  • LLMs Struggle with Low-Resource Languages: Despite their power, LLMs showed very poor performance, likely due to a lack of sufficient Hawaiian text in their training data. Their attempts to "rewrite" words rather than just correct orthography were a notable issue.
  • Character-Level Tokenization is Key for Seq2Seq: For neural models, character-level tokenization proved more effective than subword tokenization, suggesting that finer-grained input is beneficial for this specific orthographic task.
  • High-Frequency Function Words are Tricky: Errors predominantly occur on common function words (particles, adpositions, etc.) where the presence of ‘okina and kahakō is highly context-dependent.

Key Results

The paper reports Word Error Rate (WER) and Character Error Rate (CER) for various models.

  • Best Performing Models: The 4-gram and 5-gram models achieved the lowest WER (3.98%) and CER (1.26%).
  • Neural Models: The Transformer seq2seq model achieved a WER of 9.33% and CER of 8.09%, significantly better than the LSTM (WER 11.96%, CER 9.65%), but still considerably worse than the best n-gram models.
  • LLMs: LLMs performed very poorly, with few-shot Llama models showing WERs over 50% and CERs around 20%, and finetuned models still struggling with WERs around 20%.
  • HawCorpus N-grams: N-gram models trained on HawCorpus counts performed poorly (WER > 20%), likely due to the corpus containing older texts where TS forms are more prevalent.

Limitations and Future Work

  • Resource Constraints: The authors acknowledge that resource limitations (a single GPU) prevented them from experimenting with larger LLMs or more extensive hyperparameter tuning for neural models.
  • Synthetic Data: The current evaluation relies on synthetic TS data derived from MS. Future work should involve assembling and evaluating on a curated set of authentic pre-1950s documents originally written in TS, with manual MS annotations.
  • LLM Improvement: While LLMs performed poorly, the authors suggest that with additional pretraining or finetuning on Hawaiian data, they might become viable in the future.
  • Longer Texts: For LLMs, experimenting with longer sentences or paragraphs could mitigate issues with truncation and improve performance.
  • Error Analysis: The authors identified that most errors involve high-frequency function words and small, local edits. Further work could focus on improving the handling of these specific cases.

Summary updated

BibTeX
@inproceedings{kapali2025statistical,
  author = {Jaden Kapali and Keaton Williamson and Winston Wu},
  title = {{Statistical and Neural Methods for Hawaiian Orthography Modernization}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure