A Sub-Character Architecture for Korean Language Processing

Karl StratosEMNLP 2017 · DOI

Korean is a "language isolate" characterized by a highly productive morphology, where words can take numerous forms. This linguistic complexity leads to a significant data sparsity problem in Natural Language Processing (NLP). In the Universal Treebank, for instance, over 57% of words in the test set are "Out-Of-Vocabulary" (OOV), meaning they were never encountered during training. This paper, presented at EMNLP 2017, proposes a novel neural architecture that addresses this by looking "under the hood" of individual characters.

The Jamo Architecture

While standard NLP approaches operate at the word or character level, Korean characters are combinatorial. A single character is composed of multiple phonetic components called jamo. For example, the character "산" (mountain) is composed of three units: "ㅅ", "ㅏ", and "ㄴ".

The core innovation of this work is a three-step hierarchy that exploits this systematic structure:

  1. Deterministic Decomposition: Using Unicode values, the model mathematically extracts the three components of any Korean character: the head consonant, the vowel, and the tail consonant. This process is entirely rule-based and requires no manual labeling.
  2. Compositional Layers: The model learns embeddings for the 51 basic jamo letters. These are combined via a non-linear transformation to create character representations, which are then fed into a Bidirectional LSTM (BiLSTM) to induce word-level representations.
  3. Parsing Engine: These representations are plugged into a transition-based BiLSTM parser for dependency parsing.
Figure 1: Hierarchical decomposition of the Korean phrase "산 을 갔다" (went to the mountain) into words, characters, and finally jamo phonetic units.

Key Insights and Efficiency

A major advantage of this approach is the capture of syntactic signals. For example, the tail consonant "ㅆ" is a strong indicator of the past tense (e.g., 갔다 - went). While a word-level model sees "went" as an atomic unit, a jamo-level model recognizes the grammatical function of "ㅆ" even in previously unseen verbs.

Furthermore, by shifting focus to jamo, the model reduces the "observation space" (the number of unique units to learn) to just 1.6% of the original character-level space. A model using only jamo (500 parameters for embeddings) can outperform a word-level model requiring nearly 300,000 parameters.

Experimental Results

The researchers evaluated the "KoreanNet" architecture against several strong baselines on the Universal Treebank. The results demonstrate that sub-character information provides a transformative gain in accuracy.

Figure 6: Performance comparison of various parsing systems. KoreanNet using jamo features achieves a Labeled Attachment Score (LAS) of 91.46, significantly outperforming word-level baselines.

As shown in the results table, the baseline lexical BiLSTM parser (K&G16) achieved a Labeled Attachment Score (LAS) of 82.77. The jamo-integrated KoreanNet reached 91.46, even outperforming feature-rich parsers that had access to "gold" (perfect) Part-of-Speech tags.

Conclusion

By leveraging the deterministic, mathematical nature of the Korean script, this sub-character architecture effectively solves the OOV problem. While the study focused on dependency parsing, the architecture is generalizable. Future work may explore its utility in informal text (where jamo-only shorthand is common) or as a bridge between text processing and phonetic speech recognition.

Summary updated

BibTeX
@inproceedings{stratos2017sub,
  author = {Karl Stratos},
  title = {{A Sub-Character Architecture for Korean Language Processing}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1075},
}
Figure