Conditional Random Fields for Korean Morpheme Segmentation and POS Tagging

Seung-Hoon Na

Abstract

There has been recent interest in statistical approaches to Korean morphological analysis. However, previous studies have been based mostly on generative models, including a hidden Markov model (HMM), without utilizing discriminative models such as a conditional random field (CRF). We present a two-stage discriminative approach based on CRFs for Korean morphological analysis. Similar to methods used for Chinese, we perform two disambiguation procedures based on CRFs: (1) morpheme segmentation and (2) POS tagging. In morpheme segmentation, an input sentence is segmented into sequences of morphemes, where a morpheme unit is either atomic or compound. In the POS tagging procedure, each morpheme (atomic or compound) is assigned a POS tag. Once POS tagging is complete, we carry out a post-processing of the compound morphemes, where each compound morpheme is further decomposed into atomic morphemes, which is based on pre-analyzed patterns and generalized HMMs obtained from the given tagged corpus. Experimental results show the promise of our proposed method.
BibTeX
@article{na2015conditional,
  author = {Seung-Hoon Na},
  title = {{Conditional Random Fields for Korean Morpheme Segmentation and POS Tagging}},
  journal = {ACM Trans. Asian low Resour. Lang. Inf. Process},
  year = {2015},
  doi = {10.1145/2700051},
}

Loading related content...

Figure