Sentence Segmentation for Classical Chinese Based on LSTM with Radical Embedding

Xu Han, Hongsu Wang, Sanqian Zhang, Qunchao Fu, Jun S. Liu

TL;DR

This paper introduces a Bi-LSTM-CRF model specifically designed for segmenting sentences in classical Chinese, a language that historically lacks both spaces and punctuation. By incorporating a novel "radical embedding" layer—which captures the semantic "roots" of Chinese characters—the authors achieved significant performance gains over standard character-level models, particularly on challenging historical texts like Tang Dynasty epitaphs.

What Problem Does This Paper Address?

Classical Chinese (spanning roughly 1600 BC to 1800 AD) presents a unique challenge for Natural Language Processing (NLP). Unlike modern Chinese, classical texts were written without any punctuation or spacing between words and sentences. While massive digitization projects have made billions of these characters available via OCR, the lack of sentence boundaries makes automated analysis—such as translation or named entity recognition—extremely difficult.

Existing tools for modern Chinese often fail because classical Chinese has a vastly different grammar, and individual characters often carry much richer, more ambiguous meanings. Furthermore, historical texts retrieved via OCR are often "noisy" due to physical damage to the original source material (like weathered stone monuments).

Methods

The authors propose a sequence labeling approach using a Bidirectional Long Short-Term Memory (Bi-LSTM) network with a Conditional Random Field (CRF) layer. The core innovation lies in the input features:

  1. Radical Embedding: Most Chinese characters contain a "radical" (a graphical component that often indicates the character's meaning). The authors argue that radicals are analogous to roots or prefixes in English. They developed a modified Continuous Bag-of-Words (CBOW) model to generate embeddings for 214 standard radicals.
  2. Feature Concatenation: For every character in a sequence, the model receives a concatenated vector consisting of the standard character embedding and its specific radical embedding.
  3. Tagging Scheme: The model uses a simplified BIO-style tagging system:
  4. B: Beginning of a sentence.
  5. E: End of a sentence.
  6. O: Middle of a sentence.
  7. Dynasty-Specific Training: Recognizing that Chinese evolved over millennia, the authors trained separate models for different dynasties to capture specific linguistic styles.

Datasets

The study utilized a massive corpus of 150 ancient books (approximately 44 million characters) sourced from the Chinese Biographical Database Project (CBDB). The data was categorized into:

  • Dynasty Tang (A.D. 618–907)
  • Dynasty Ming (A.D. 1368–1644)
  • Dynasty Qing (A.D. 1644–1912)
  • Tang’s Epitaphs: A specific subset of texts engraved on stone, which included "unsure characters" (marked as □) due to corrosion.

Key Insights and Ideas

  • Sub-character Semantics Matter: While previous research suggested that radicals don't help much in modern Chinese segmentation, this paper proves they are vital for classical Chinese. In ancient texts, the "shape" and component parts of a character carry more weight because the language is more telegraphic and less reliant on multi-character words.
  • Style Consistency: The model performed exceptionally well on Tang Dynasty epitaphs. The authors attribute this to the "standardized" and formal literary style of funeral inscriptions compared to more varied prose.
  • Evolutionary Modeling: The decision to train separate models for different dynasties highlights the importance of temporal context in historical NLP; a model trained on 18th-century Qing texts is not necessarily optimal for 7th-century Tang texts.

Key Results

The Bi-LSTM-CRF with radical embeddings consistently outperformed standard CRF and basic LSTM models across all datasets:

  • Tang Epitaphs: Achieved the highest performance with an F1-score of 81.34% (Precision: 0.814, Recall: 0.813).
  • Qing Dynasty: Performed the next best (F1: 0.772), likely due to having the largest training set and a grammar closer to modern Chinese.
  • Comparison: The inclusion of radical embeddings provided a measurable boost over the standard Bi-LSTM-CRF without them, confirming that sub-character features provide a "better handle" on classical syntax.

Limitations and Future Work

  • OCR Noise: While the authors filtered out sentences with more than five consecutive "unknown" characters (□), the model still struggles with heavily corroded texts where the context is lost.
  • Punctuation Variety: Currently, the model only identifies where a sentence ends (segmentation). It does not yet distinguish between different types of punctuation (commas, periods, exclamation marks), which the authors identify as a goal for future research.
  • POS Tagging: The lack of a high-quality Part-of-Speech (POS) tagged corpus for classical Chinese remains a bottleneck for improving these models further.

Summary updated

Abstract

In this paper, we develop a low than character feature embedding called radical embedding, and apply it on LSTM model for sentence segmentation of pre modern Chinese texts. The datasets includes over 150 classical Chinese books from 3 different dynasties and contains different literary styles. LSTM CRF model is a state of art method for the sequence labeling problem. Our new model adds a component of radical embedding, which leads to improved performances. Experimental results based on the aforementioned Chinese books demonstrates a better accuracy than earlier methods on sentence segmentation, especial in Tang Epitaph texts.
BibTeX
@misc{han2018sentence,
  author = {Xu Han and Hongsu Wang and Sanqian Zhang and Qunchao Fu and Jun S. Liu},
  title = {{Sentence Segmentation for Classical Chinese Based on LSTM with Radical Embedding}},
  year = {2018},
  eprint = {1810.03479},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1810.03479},
}
Figure