Abstract
Translation of ancient languages provides critical insights into historical natural phenomena, medicine, and science. However, the manual translation of ancient Korean is a slow process requiring highly specialized experts. This paper proposes the first ancient Korean neural machine translation (NMT) model using a Transformer architecture. To address the unique linguistic characteristics of ancient Korean, we introduce Share Vocabulary and Entity Restriction Byte Pair Encoding (SVER BPE). This method improves performance by 5.25 BLEU points over conventional subword tokenization. Combined with decoding strategies like n-gram blocking and ensemble modeling, the system achieves a significant performance gain. The model is publicly available as a software application to assist researchers in generating draft translations of vast historical archives.
Introduction
Ancient Korean literature, such as the Annals of the Joseon Dynasty, represents a massive repository of historical knowledge. Despite its value, translating these texts into modern Korean is a significant bottleneck. There are currently only about 200 qualified experts, and training a new one takes over a decade. At the present rate, manual translation of untranslated documents would take more than 80 years.
Neural Machine Translation (NMT) offers a potential solution. However, ancient Korean presents unique challenges: archaic grammar, high density of specific entities (names, titles, locations), and a lack of existing computational literature. This research bridges the gap by developing a specialized Transformer-based model tailored to these nuances.
Methodology
The core of the system is a Transformer-based NMT model, chosen for its effective self-attention mechanism. The primary innovation lies in the preprocessing stage:
SVER BPE (Share Vocabulary and Entity Restriction Byte Pair Encoding)
Standard subword tokenization methods like Byte Pair Encoding (BPE) often fragment words into smaller pieces to manage rare vocabulary. In ancient texts, proper nouns (entities) such as government offices or historical figures are critical anchors for meaning. SVER BPE prevents the model from splitting these entities into meaningless subwords, treating them as atomic units. Additionally, it shares the vocabulary between the source (ancient) and target (modern) languages to enhance consistency.
Decoding and Ensemble Strategies
To further refine output quality, the researchers implemented:
- N-gram blocking: Prevents the model from generating repetitive loops.
- Ensemble modeling: Combines the strengths of both LSTM-based and Transformer-based architectures, capturing diverse linguistic nuances.
Datasets and Source Material
The researchers constructed a parallel corpus by crawling data from the Institute for the Translation of Korean Classics (ITKC).
The dataset consists of 52,778 parallel sentences. Analysis reveals that ancient Korean is significantly more information-dense than modern Korean; an average ancient sentence of 39 syllables expands to roughly 92 syllables in modern translation. The data was split into training, validation (5,000 sentences), and testing (3,000 sentences) sets.
Results and Discussion
The proposed SVER BPE method proved highly effective, outperforming standard BPE by 5.25 BLEU points. By integrating ensemble models and optimized decoding (beam search size 3 and 8-gram blocking), the system reached a final score of 32.57 BLEU.
Key insights from the study include:
- Entity Preservation: Protecting proper nouns during tokenization was the most effective single factor in boosting translation quality.
- Tokenization Priority: For low-resource, niche domains like ancient languages, the method of data representation (tokenization) can be more impactful than the specific neural architecture.
- Architectural Synergy: While Transformers generally lead in performance, ensembling them with LSTMs captured additional linguistic nuances, yielding the best overall results.
Conclusion
This work introduces the first NMT system for ancient Korean, providing a vital tool for historians and researchers. While the current dataset is relatively small (52,000 sentences), the model already offers high-quality draft translations that can significantly accelerate the preservation of Korean heritage. Future work will focus on expanding the parallel corpus and testing the generalizability of the SVER BPE approach to other ancient languages with high entity density.
Summary updated