Word Segmentation of Vietnamese Texts: a Comparison of Approaches

Quang Thang Dinh, Hong Phuong Le, Nguyên Thi Minh Huyên, Cam-Tu Nguyen, Mathias Rossignol, Xuân Luong VuLREC 2008 · OpenReview

Abstract

We present in this paper a comparison between three segmentation systems for the Vietnamese language. Indeed, the majority of Vietnamese words is built by semantic composition from about 7,000 syllables, which also have a meaning as isolated words. So the identification of word boundaries in a text is not a simple task, and ambiguities often appear. Beyond the presentation of the tested systems, we also propose a standard definition for word segmentation in Vietnamese, and introduce a reference corpus developed for the purpose of evaluating such a task. The results observed confirm that it can be relatively well treated by automatic means, although a solution needs to be found to take into account out-of-vocabulary words.
BibTeX
@inproceedings{dinh2008word,
  author = {Quang Thang Dinh and Hong Phuong Le and Nguyên Thi Minh Huyên and Cam-Tu Nguyen and Mathias Rossignol and Xuân Luong Vu},
  title = {{Word Segmentation of Vietnamese Texts: a Comparison of Approaches}},
  booktitle = {LREC},
  year = {2008},
  url = {https://openreview.net/forum?id=nu6ONI76jsU},
}

Loading related content...

Figure