Domain Adaptation of Thai Word Segmentation Models using Stacked Ensemble

Peerat Limkonchotiwat, Wannaphong Phatthiyaphaibun, Raheem Sarwar, Ekapol Chuangsuwanich, Sarana NutanongEMNLP 2020 · DOI

Introduction

Word Segmentation (WS) is a critical preprocessing step for languages like Thai, Chinese, and Japanese, where words are not separated by spaces. While deep learning has significantly improved WS performance, these models often suffer from "domain shift." A model trained on formal news corpora typically fails when applied to the informal language, hashtags, and loanwords prevalent on social media.

Standard domain adaptation techniques, such as Transfer Learning (TL), require updating the internal weights of a pre-trained model. However, in many real-world scenarios, models are provided as "black boxes"—proprietary tools or third-party APIs where internal parameters are inaccessible. This paper introduces SEFR (Stacked-Ensemble Filter-and-Refine), a framework designed to adapt these locked models to new domains without modifying their internal structure.

The SEFR Framework

The SEFR framework employs a three-stage pipeline to achieve efficient domain adaptation.

  1. Domain-Generic (DG) Model: The process begins with a pre-trained "black-box" model (e.g., DeepCut or AttaCut). This model generates initial segmentation predictions and associated confidence scores (softmax or logistic probabilities).
  2. Uncertainty Filtering: To identify potential errors caused by domain shift, the system calculates the softmax entropy of the DG model’s predictions. High entropy indicates that the model is uncertain about a specific character boundary. The system then selects the top-$k$ most uncertain samples for further processing.
  3. Domain-Specific (DS) Model: These uncertain samples are passed to a lightweight, trainable model—typically a Conditional Random Field (CRF). This DS model is trained on a small amount of labeled data from the target domain, using n-grams, dictionary indices, and meta-features (the entropy and output from the DG model) to refine the final segmentation.
The SEFR framework architecture: from black-box prediction to entropy-based filtering and domain-specific refinement.

Experimental Evaluation

The authors evaluated SEFR across three languages using several benchmark corpora, including Thai (Wisesight social media and TNHC classical literature), Chinese (SIGHAN 2005), and Japanese (Universal Dependencies 2.5). This diverse testing ground aimed to prove the framework's utility across different delimiter-free languages.

Results and Discussion

Experimental results demonstrate that SEFR provides a significant performance boost, often reaching parity with full Transfer Learning. On the Thai Wisesight dataset, the SE+DeepCut ensemble achieved a 95.2% Character F1 score, a notable improvement over the base DeepCut model's 93.8%. The most dramatic gains were observed in Japanese, where the Nagisa model's F1 score jumped from 78.8% to 90.3% using the SEFR framework.

Performance scores (F1, Precision, Recall) across different k% candidate filtering thresholds for various models and datasets.

As shown in the performance graphs, the filtering threshold $k$ plays a vital role. By refining only the most uncertain candidates, the system maintains high computational efficiency while capturing the majority of domain-specific nuances. Ablation studies further revealed that "meta-features"—the raw output and entropy from the black-box model—were the most critical components for the refinement model's success.

Conclusion

SEFR proves that internal model access is not a prerequisite for effective domain adaptation. By treating the generic model as a noisy baseline and using entropy as a surgical detector for domain shift, the "filter-and-refine" approach offers a universal strategy for improving sequence labeling tasks in low-resource or black-box environments. Future work will explore more sophisticated uncertainty metrics and the application of SEFR to other tasks like Named Entity Recognition (NER).

Summary updated

BibTeX
@inproceedings{limkonchotiwat2020domain,
  author = {Peerat Limkonchotiwat and Wannaphong Phatthiyaphaibun and Raheem Sarwar and Ekapol Chuangsuwanich and Sarana Nutanong},
  title = {{Domain Adaptation of Thai Word Segmentation Models using Stacked Ensemble}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.315},
}
Figure