Introduction
In Chinese natural language processing, the traditional approach involves a pipeline that begins with Word Segmentation (CWS), followed by Part-of-Speech (POS) tagging, and finally syntactic parsing. However, this pipeline suffers from significant error propagation: a mistake in the initial segmentation phase often makes accurate parsing impossible. Furthermore, the definition of a "word" in Chinese is linguistically ambiguous, leading to inconsistencies across different datasets.
This paper proposes a shift in perspective, treating characters as the fundamental "atoms" of the Chinese language. By parsing at the character level, the model can capture the internal structure of words (intra-word syntax) and bypass the problematic word segmentation step entirely.
Methodology: The Joint Span Parser
The authors introduce an end-to-end Joint Span Parser designed to handle both constituent and dependency formalisms simultaneously.
Joint Span Structure
Inspired by Head-driven Phrase Structure Grammar (HPSG), the model integrates phrase-based (constituent) and relation-based (dependency) information into a single unified tree structure. This allows the model to leverage the strengths of both syntactic representations.
Neural Architecture
The framework utilizes a Transformer-based encoder-decoder architecture. Key features include:
- Biaffine Attention: Employed for dependency scoring to capture complex relations between characters.
- Multi-Task Learning (MTL): The model jointly learns POS tags and syntactic structures. By predicting POS tags early, the network gains a better understanding of the functional roles characters play before determining their syntactic placement.
- Restoration Strategy: To ensure compatibility with existing word-level benchmarks, the authors developed a "Constituent Categ" strategy to map character-level trees back to standard word-level representations.
Datasets and Evaluation
The model was evaluated on several key benchmarks:
- CTB 5.1 (Chinese Penn Treebank): The standard for word-level parsing.
- SCDT (SJTU Chinese Character Dependency Treebank): A new dataset providing fine-grained character-level dependencies.
- ZCTB: A character-level treebank used for comparative analysis.
Key Results and Insights
State-of-the-Art Performance
The Joint Span Parser achieved state-of-the-art (SOTA) results on character-level datasets, reaching an F1 score of 89.60 for constituent parsing and a Labeled Attachment Score (LAS) of 88.70 for dependency parsing. Even when converted back to word-level for comparison with traditional models, it remained highly competitive, achieving a 91.63 F1 score without requiring a gold segmenter.
Downstream Impact
The research demonstrates that character-level syntactic features provide a richer linguistic signal than word-level features. Integrating these structures improved performance in:
- Neural Machine Translation (NMT): Better handling of word internal logic.
- Text Classification: Improved accuracy on benchmarks like TNEWS and IFLYTEK.
- Language Modeling: The authors' LIMIT-BERT-char, which incorporates character-level syntax, outperformed standard BERT models in machine reading comprehension.
Conclusion
By modeling Chinese at the character level, this work effectively eliminates the "segmentation trap" and demonstrates that intra-word syntax is a critical signal for deep language understanding. While the $O(n^3)$ complexity of the CKY-style decoder presents a challenge for extremely long sequences, the robustness and downstream utility of the character-centric approach mark a significant advancement in Chinese NLP.
Summary updated