Bridging the Gap: Improve Part-of-speech Tagging for Chinese Social Media Texts with Foreign Words

Dingmin Wang, Meng Fang, Yan Song, Juntao Li

Introduction

Part-of-Speech (POS) tagging is a fundamental task in Natural Language Processing (NLP), serving as a prerequisite for downstream applications such as dependency parsing and sentiment analysis. However, the rise of social media has introduced significant challenges for traditional POS taggers. These challenges are primarily two-fold: domain mismatch and code-switching.

Domain mismatch occurs because most state-of-the-art taggers are trained on formal "newswire" corpora, such as the Wall Street Journal or Chinese Treebank, which differ significantly from the informal, slang-heavy language of social media. Code-switching, the practice of mixing multiple languages within a single utterance (e.g., inserting English words into Chinese sentences), further complicates the task. Standard Chinese taggers often fail to recognize the grammatical roles of foreign words, frequently labeling them as "unknown" or "foreign," which disrupts the linguistic analysis of the entire sentence.

Comparison of POS tagging results from different systems on a code-switched sentence.

Methodology

To address these issues, researchers from Tencent and Peking University proposed a language-agnostic POS tagger. The architecture is built upon a Bi-directional Long Short-Term Memory (BiLSTM) network, enhanced with two key components: cross-lingual word embeddings and adversarial training.

Cross-lingual Token Representation

The model leverages cross-lingual embeddings to map Chinese and English words into a shared semantic space. This allows the model to recognize that "government" and "政府" share similar grammatical and semantic properties, facilitating knowledge transfer from high-resource languages (English) to the target domain.

Adversarial Training

A critical innovation is the inclusion of a language discriminator. During training, the discriminator attempts to identify the language of the input word, while the rest of the network is trained to "trick" it. This adversarial process forces the BiLSTM to learn language-invariant features, ensuring the model focuses on the grammatical role of a word rather than its language of origin.

The proposed neural architecture featuring BiLSTM, cross-lingual embeddings, and an adversarial language discriminator.

Social Media Adaptation

The standard 12 universal POS tags were expanded to include 6 specialized tags for social media elements, such as Emojis, URLs, phone numbers, @-mentions, and hashtags, ensuring comprehensive coverage of informal text.

Datasets and Evaluation

The model was evaluated using a combination of standard and novel datasets:

  • Standard Training Data: Penn Treebank (English), Chinese Treebank, and ARK (English Twitter data).
  • S-weibo (Synthetic): 1,000 sentences where Chinese words were replaced with English equivalents to test robustness.
  • R-weibo (Real): A new, human-annotated benchmark consisting of 700 real Chinese-English mixed posts from Weibo.

Key Results

The proposed BiLSTM+ model demonstrated significant improvements over existing tools like Jieba, NLPIR, and the Stanford Tagger on mixed-language data. On the R-weibo dataset, the model achieved F1-scores of 0.812 for verbs and 0.731 for nouns for English tokens—categories where traditional Chinese taggers often failed completely. While specialized Chinese taggers maintained a slight edge (0.01-0.02 F1) on pure Chinese text, the BiLSTM+ model provided far superior overall robustness across code-switched sentences.

Conclusion

By combining cross-lingual embeddings with adversarial training, this research successfully bridges the gap between single-language training data and the multi-lingual reality of social media. The approach demonstrates that leveraging high-resource language data can effectively improve performance in low-resource, mixed-language domains. Future work may involve expanding this framework to other language pairs and scaling the annotated corpora for even greater accuracy.

Summary updated

BibTeX
@inproceedings{wang2019bridging,
  author = {Dingmin Wang and Meng Fang and Yan Song and Juntao Li},
  title = {{Bridging the Gap: Improve Part-of-speech Tagging for Chinese Social Media Texts with Foreign Words}},
  booktitle = {SemDeep @ IJCAI},
  year = {2019},
}
Figure