Abstract
Ancient Chinese presents a unique challenge for Natural Language Processing due to its significant syntactic divergence from modern Chinese and a severe lack of annotated dependency data. This paper introduces the confidence-based syntax encoding network (cSEN), a novel architecture designed to leverage "noisy" syntax trees derived from modern parsers or unsupervised methods. By employing a biaffine attention mechanism to score the reliability of syntactic connections and integrating local structural features, cSEN effectively filters out noise while capturing essential grammatical cues. Experimental results on poetry theme classification and ancient-to-modern translation demonstrate that cSEN significantly outperforms baseline models, particularly when paired with domain-specific encoders like Anchi-BERT.
Introduction
Ancient Chinese is the linguistic ancestor of modern Chinese, yet the two differ vastly in syntax and semantics. While modern NLP has flourished through massive annotated datasets, Ancient Chinese lacks the syntactic labels—such as dependency trees—necessary for deep structural understanding. Attempting to apply modern Chinese parsers to ancient texts often results in "noisy" and error-prone structures due to 6,000 years of grammatical evolution.
This research addresses the challenge of utilizing these imperfect structural hints. Rather than treating noisy syntax as a liability, the authors propose a method to filter and weight these signals, allowing models to benefit from structural context without being misled by inaccuracies.
Methodology
The core of the proposed approach is the confidence-based syntax encoding network (cSEN). This architecture enhances the standard Transformer encoder with a specialized layer for processing syntactic information.
Confidence-Based GAT
The model utilizes a Graph Attention Network (GAT) to represent sentences as graphs where tokens are nodes and dependencies are edges. To mitigate the impact of noise, a Biaffine attention mechanism calculates a confidence score for every predicted connection. Connections with low scores are downweighted, ensuring the model focuses on reliable syntactic links.
Left-Right Branch (LRB) Feature
Ancient Chinese, particularly in its poetic form, is highly concise. The authors introduced a "local" feature called the Left-Right Branch (LRB), which assumes words are likely related to their immediate neighbors. This captures local context that complex, error-prone parsers might overlook.
Gated Integration
A gating mechanism dynamically decides the balance between confidence-weighted syntax and local LRB features, producing a refined final representation of the text.
Experimental Setup
The model was evaluated on two primary tasks:
- Ancient Poetry Theme Classification: A new dataset of 22,360 poems categorized into 10 themes (e.g., #Homesickness, #War, #Farewell) was introduced as a benchmark.
- Ancient-Modern Chinese Translation: Using the NiuTrans parallel corpus (approx. 1 million pairs), the model was tested on its ability to translate literary Chinese into modern vernacular.
Results and Analysis
The cSEN model demonstrated superior performance across both tasks. In poetry classification, the model achieved a Micro F1 of 93.8 and a Macro F1 of 91.9, notably higher than the baseline and vanilla GAT implementations.
Key Findings
- Noise Management: Incorporating noisy syntax trees outperformed models that ignored syntax entirely, provided the "confidence" component was active. Ablation studies showed a significant performance drop when confidence scoring was removed.
- The Power of Locality: The simple LRB feature was surprisingly effective, often outperforming complex parsers by capturing the compact nature of ancient poetry.
- Domain-Specific Pre-training: Using Anchi-BERT (pre-trained on ancient texts) provided a much stronger foundation than modern Chinese BERT (BERT-wwm), highlighting the importance of domain-specific encoders.
- Scalability with Length: The benefits of cSEN were most pronounced in longer sentences, where syntax helps bridge the distance between related words that are far apart in the sequence.
Conclusion
The cSEN framework successfully demonstrates that noisy, derived syntax trees can be a valuable resource for low-resource languages like Ancient Chinese. By accounting for the reliability of syntactic links, the model achieves state-of-the-art results in classification and translation. Future work will focus on improving computational efficiency through parameter sharing to reduce the overhead of the dual-attention operations required for confidence scoring.
Summary updated