Ancient Korean Neural Machine Translation

Chanjun Park, Chanhee Lee, YeongWook Yang, Heuiseok LimIEEE Access 2020 · OpenReview

Abstract

Translation of ancient languages provides critical insights into historical natural phenomena, medicine, and science. However, the manual translation of ancient Korean is a slow process requiring highly specialized experts. This paper proposes the first ancient Korean neural machine translation (NMT) model using a Transformer architecture. To address the unique linguistic characteristics of ancient Korean, we introduce Share Vocabulary and Entity Restriction Byte Pair Encoding (SVER BPE). This method improves performance by 5.25 BLEU points over conventional subword tokenization. Combined with decoding strategies like n-gram blocking and ensemble modeling, the system achieves a significant performance gain. The model is publicly available as a software application to assist researchers in generating draft translations of vast historical archives.

Introduction

Ancient Korean literature, such as the Annals of the Joseon Dynasty, represents a massive repository of historical knowledge. Despite its value, translating these texts into modern Korean is a significant bottleneck. There are currently only about 200 qualified experts, and training a new one takes over a decade. At the present rate, manual translation of untranslated documents would take more than 80 years.

Neural Machine Translation (NMT) offers a potential solution. However, ancient Korean presents unique challenges: archaic grammar, high density of specific entities (names, titles, locations), and a lack of existing computational literature. This research bridges the gap by developing a specialized Transformer-based model tailored to these nuances.

Methodology

The core of the system is a Transformer-based NMT model, chosen for its effective self-attention mechanism. The primary innovation lies in the preprocessing stage:

SVER BPE (Share Vocabulary and Entity Restriction Byte Pair Encoding)

Standard subword tokenization methods like Byte Pair Encoding (BPE) often fragment words into smaller pieces to manage rare vocabulary. In ancient texts, proper nouns (entities) such as government offices or historical figures are critical anchors for meaning. SVER BPE prevents the model from splitting these entities into meaningless subwords, treating them as atomic units. Additionally, it shares the vocabulary between the source (ancient) and target (modern) languages to enhance consistency.

Decoding and Ensemble Strategies

To further refine output quality, the researchers implemented:

  • N-gram blocking: Prevents the model from generating repetitive loops.
  • Ensemble modeling: Combines the strengths of both LSTM-based and Transformer-based architectures, capturing diverse linguistic nuances.

Datasets and Source Material

The researchers constructed a parallel corpus by crawling data from the Institute for the Translation of Korean Classics (ITKC).

Example of ancient Korean source material, showing the cover and internal text of historical records.

The dataset consists of 52,778 parallel sentences. Analysis reveals that ancient Korean is significantly more information-dense than modern Korean; an average ancient sentence of 39 syllables expands to roughly 92 syllables in modern translation. The data was split into training, validation (5,000 sentences), and testing (3,000 sentences) sets.

Results and Discussion

The proposed SVER BPE method proved highly effective, outperforming standard BPE by 5.25 BLEU points. By integrating ensemble models and optimized decoding (beam search size 3 and 8-gram blocking), the system reached a final score of 32.57 BLEU.

Key insights from the study include:

  • Entity Preservation: Protecting proper nouns during tokenization was the most effective single factor in boosting translation quality.
  • Tokenization Priority: For low-resource, niche domains like ancient languages, the method of data representation (tokenization) can be more impactful than the specific neural architecture.
  • Architectural Synergy: While Transformers generally lead in performance, ensembling them with LSTMs captured additional linguistic nuances, yielding the best overall results.

Conclusion

This work introduces the first NMT system for ancient Korean, providing a vital tool for historians and researchers. While the current dataset is relatively small (52,000 sentences), the model already offers high-quality draft translations that can significantly accelerate the preservation of Korean heritage. Future work will focus on expanding the parallel corpus and testing the generalizability of the SVER BPE approach to other ancient languages with high entity density.

Summary updated

Abstract

Translation of the languages of ancient times can serve as a source for the content of various digital media and can be helpful in various fields such as natural phenomena, medicine, and science. Owing to these needs, there has been a global movement to translate ancient languages, but expert minds are required for this purpose. It is difficult to train language experts, and more importantly, manual translation is a slow process. Consequently, the recovery of ancient characters using machine translation has been recently investigated, but there is currently no literature on the machine translation of ancient Korean. This paper proposes the first ancient Korean neural machine translation model using a Transformer. This model can improve the efficiency of a translator by quickly providing a draft translation for a number of untranslated ancient documents. Furthermore, a new subword tokenization method called the Share Vocabulary and Entity Restriction Byte Pair Encoding is proposed based on the characteristics of ancient Korean sentences. This proposed method yields an increase in the performance of the original conventional subword tokenization methods such as byte pair encoding by 5.25 BLEU points. In addition, various decoding strategies such as n-gram blocking and ensemble models further improve the performance by 2.89 BLEU points. The model has been made publicly available as a software application.
BibTeX
@inproceedings{park2020ancient,
  author = {Chanjun Park and Chanhee Lee and YeongWook Yang and Heuiseok Lim},
  title = {{Ancient Korean Neural Machine Translation}},
  booktitle = {IEEE Access},
  year = {2020},
  url = {https://openreview.net/forum?id=P51Y4u0hMv6},
}
Figure