TL;DR
This paper introduces a novel multi-strategy Named Entity Recognition (NER) system for ancient Chinese texts, addressing the challenges of dataset heterogeneity and complex linguistic features. By employing a hybrid approach that combines Conditional Random Fields (CRF) for complex tasks and Softmax for simpler ones, the system achieves state-of-the-art performance on the EvaHan2025 benchmark, outperforming baselines and demonstrating adaptability across historical and medical domains.
What Problem Does This Paper Address?
Named Entity Recognition (NER) is a crucial task in information extraction, aiming to identify and categorize key entities like people, locations, and organizations within text. While NER has seen significant advancements in modern languages, applying it to ancient Chinese texts presents unique challenges. These include:
- Dataset Heterogeneity: Different historical periods and domains (e.g., historical records, medical texts) have varying entity types and linguistic styles.
- Linguistic Complexity: Ancient Chinese texts often lack punctuation, feature polysemy (words with multiple meanings), and have a continuous structure, making it difficult to accurately identify entity boundaries.
- Limited Resources: Publicly available, well-annotated datasets for ancient Chinese NER are scarce.
The EvaHan2025 competition aims to address this by providing a large, curated dataset of historical and medical classical texts. This paper tackles the problem of building an effective NER system for this complex and heterogeneous dataset.
Methods
The core of the proposed solution is a multi-strategy NER framework. This approach leverages a powerful pre-trained language model (PLM) called GujiRoBERTa_jian_fan, specifically trained on ancient Chinese texts, to generate contextual representations of the input text. The novelty lies in how this PLM is integrated with different downstream classifiers based on the complexity of the task:
- For Tasks A and C: These tasks involve six complex entity types and require capturing intricate dependencies between entities. The system employs a Conditional Random Field (CRF) layer on top of the PLM. CRF is well-suited for sequence labeling tasks where the relationship between adjacent labels is important.
- For Task B: This task has a simpler tagset with only three entity types. To optimize for efficiency, a lightweight Softmax classifier is used instead of CRF. This choice is justified by ablation studies showing that CRF's sequence modeling is less critical for simpler tasks, and Softmax offers significant computational advantages without sacrificing accuracy.
This hybrid approach allows the system to balance accuracy (by using CRF for complex dependencies) and efficiency (by using Softmax for simpler tasks).
Datasets
The primary dataset used in this paper is the EvaHan2025 dataset. This dataset comprises 500,000 characters from three distinct sources, each corresponding to a specific task:
- Task A: Shiji (historical records)
- Task B: Twenty-Four Histories (historical records)
- Task C: Traditional Chinese Medicine Classics
The dataset is split into training (80%), validation (20%), and an unlabeled blind test set for final evaluation. The paper notes that the dataset exhibits heterogeneity in terms of entity types and domain styles, which motivates their multi-strategy approach.
Key Insights and Ideas
- Hybrid Approach is Key: The most significant insight is that a one-size-fits-all approach (e.g., using only CRF or only Softmax for all tasks) is suboptimal. Tailoring the classifier to the complexity of each task (CRF for complex, Softmax for simple) leads to better overall performance and efficiency.
- Task B Benefits from Softmax: For simpler tasks like Task B, Softmax offers a substantial speedup in inference time (63% faster) with minimal impact on F1 score, making it a more practical choice.
- CRF Excels on Complex Dependencies: CRF's ability to model sequential dependencies is crucial for tasks with more entity types (Tasks A and C), leading to higher F1 scores, especially in domains with structured patterns like TCM Classics.
- Single-Strategy Models Struggle with Overlap: When trained on all tasks simultaneously, single-strategy models (especially CRF) can suffer from interference, particularly when one task's tagset is a subset of another's (like Task B within Task A). The multi-strategy approach mitigates this by isolating task-specific training.
Key Results
The proposed multi-strategy NER system achieved impressive results on the EvaHan2025 blind tests:
- Overall F1 Improvement: The system achieved an overall F1 score improvement of 4.30% compared to the baseline.
- Task-Specific Performance:
- Test A: F1 score of 83.94% (outperforming baseline by 2.46%).
- Test B: F1 score of 88.31% (outperforming baseline by 0.81%).
- Test C: F1 score of 82.15% (outperforming baseline by 9.75%).
- Efficiency Gains: For Task B, the Softmax classifier reduced blind test inference time by 63% compared to CRF, while also significantly decreasing training/validation time.
Limitations and Future Work
The authors acknowledge several limitations:
- Inconsistent Generalization: The system's performance varies across tasks. While Task C's F1 improved significantly, Task A's F1 dropped due to potential overfitting to its diverse data.
- Rare Entities: Entities with low support (rare entities) exhibit inconsistent performance, posing a challenge for robust recognition.
Future work directions suggested include:
- Adaptive Hyperparameter Tuning: Optimizing hyperparameters dynamically for each task.
- Tagset Refinement: Exploring ways to improve the tagset design for better generalization.
- Cross-Domain Validation: Using validation strategies that better account for domain shifts.
- Data Augmentation: Employing techniques to increase the representation of rare entities.
Summary updated