That Slepen Al the Nyght with Open Ye! Cross-era Sequence Segmentation with Switch-memory

Xuemei Tang, Qi SuACL 2022 · DOI

Abstract

Chinese Word Segmentation (CWS) faces significant challenges when applied across different historical eras due to the diachronic linguistic gap. We present CROSSWISE (CROsS-era Segmentation WIth Switch-mEmory), a framework designed to bridge this gap by integrating era-specific knowledge into a neural architecture. By utilizing a "Switch-memory" module that dynamically queries era-specific dictionaries, CROSSWISE achieves superior performance on both ancient and modern Chinese texts, particularly in recovering out-of-vocabulary (OOV) words.

Introduction

Language evolution over centuries creates a "diachronic linguistic gap," where grammar, vocabulary, and semantics shift significantly. In Chinese, this evolution spans Archaic, Middle Ancient, Near Ancient, and Modern stages. Most existing NLP tools are specialized for a single era; for instance, a segmenter trained on modern news often fails on classical texts like the Analects of Confucius. This specialization hinders the processing of "era-hybrid" texts—modern prose that incorporates classical citations. The challenge of Cross-era Chinese Word Segmentation (CECWS) lies in creating a unified model capable of handling these diverse linguistic nuances simultaneously.

Methodology

The CROSSWISE framework addresses the CECWS task through three integrated components:

  1. Encoder-Decoder Base: The model employs a pre-trained BERT encoder to extract general linguistic features, followed by a Conditional Random Field (CRF) decoder to determine word boundaries.
  2. Era Discriminator: A classifier built upon the BERT [CLS] token identifies the era of the input sentence. This component acts as a "switcher," determining the influence of era-specific information.
  3. Switch-memory (SM) Module: This core innovation utilizes four era-specific dictionaries. For each character, the model retrieves candidate words from these dictionaries and stores them in Key-Value Memory Networks.
The CROSSWISE architecture featuring the Switch-memory module and era-specific memory cells.

The "Switcher" governs the integration of these memories. The authors evaluated two modes: a Hard-switcher, which selects the single most probable era, and a Soft-switcher, which performs a weighted fusion of all era-specific memories based on the discriminator's output.

Experimental Setup

The model was evaluated across four distinct corpora representing the evolution of the Chinese language:

  • AKIWI (Ancient): Academia Sinica Ancient Chinese Corpus (ASACC).
  • DKIWI (Middle Ancient): ASACC.
  • PKIWI (Near Ancient): ASACC.
  • MSR (Modern): SIGHAN 2005 bakeoff dataset.

To maintain consistency, traditional Chinese datasets were converted to simplified Chinese for the BERT encoder.

Results and Discussion

CROSSWISE demonstrated significant improvements over single-era baselines and multi-domain models, achieving an average F1-score of 98.24% across all datasets.

Comparison of F1 scores across different datasets using various switcher and fusion configurations.

The OOV Lifeline

The most notable gain was in Out-of-Vocabulary (OOV) recall ($R_{oov}$). In the Ancient Chinese (AKIWI) dataset, $R_{oov}$ increased from a baseline of 68.85% to 83.88%. The Switch-memory module effectively serves as a dynamic dictionary, allowing the model to identify archaic terms even when they are absent from the primary training data.

Knowledge Dissemination and Switching Modes

Analysis of the switching mechanisms revealed that the Soft-switcher was most effective for Modern Chinese (MSR). This suggests that modern Chinese benefits from "forward" knowledge dissemination, where understanding ancient roots aids in segmenting modern compounds. Conversely, Ancient Chinese datasets (AKIWI) performed better with the Hard-switcher, as modern linguistic patterns can introduce noise when interpreting archaic syntax.

Conclusion

CROSSWISE successfully bridges the diachronic gap in Chinese Word Segmentation by leveraging era-specific dictionary knowledge through a dynamic switching mechanism. Future work will explore more granular temporal modeling and the application of the Switch-memory module to other cross-era tasks such as Named Entity Recognition (NER) and Part-of-Speech (POS) tagging.

Summary updated

BibTeX
@inproceedings{tang2022that,
  author = {Xuemei Tang and Qi Su},
  title = {{That Slepen Al the Nyght with Open Ye! Cross-era Sequence Segmentation with Switch-memory}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
  doi = {10.18653/v1/2022.acl-long.540},
}
Figure