TL;DR
Researchers from the Beijing University of Posts and Telecommunications have developed a high-performing Named Entity Recognition (NER) system specifically for ancient Chinese texts. By combining the domain-specific GujiRoBERTa language model with a Conditional Random Field (CRF) decoding layer, they outperformed official baselines in the EvaHan 2025 competition, achieving an overall F1 score of 83.74%.
What Problem Does This Paper Address?
Named Entity Recognition (NER)—the task of identifying people, places, and organizations in text—is a foundational step in Natural Language Processing. While modern Chinese NER is well-studied, ancient Chinese presents a unique set of "boss-level" challenges:
- Archaic Grammar: The structure of the language differs significantly from modern Mandarin.
- Lack of Punctuation: Traditional texts often lack the visual cues we rely on to segment thoughts.
- Lexical Evolution: Words have changed meanings or disappeared entirely over millennia.
Standard models pre-trained on modern Chinese (like BERT) often fail to capture these linguistic nuances. This paper tackles these hurdles within the context of the EvaHan 2025 shared task, focusing on historical and medical classics.
Methods
The authors adopted a "closed modality" approach, meaning they used specific pre-approved models and datasets. Their architecture consists of three main components:
- The Backbone (GujiRoBERTa): Instead of a general-purpose model, they used GujiRoBERTa-jian-fan, which is specifically pre-trained on ancient Chinese corpora. This ensures the model "understands" the specific vocabulary and syntax of the era.
- The Decoder (CRF): While many modern systems use a simple Softmax layer to predict tags, the authors implemented a Conditional Random Field (CRF). A CRF is excellent for NER because it doesn't just look at one token at a time; it models the dependencies between tags (e.g., it knows that an "End-Entity" tag must follow a "Begin-Entity" tag), ensuring the final output is globally coherent.
- Optimization Strategy: They experimented with various learning rate schedulers and found that a Linear Schedule with Warmup provided the most stable and effective training compared to cosine or constant schedules.
- Long Sentence Handling: To deal with the Transformer's token limit, they developed a strategy to split long sentences based on punctuation (where available) to maintain semantic completeness before processing.
Datasets
The study utilized three distinct datasets provided by the EvaHan 2025 competition:
- Dataset A: Sourced from the Shiji (Records of the Grand Historian).
- Dataset B: Extracted from the Twenty-Four Histories.
- Dataset C: Focused on texts from Traditional Chinese Medicine (TCM) Classics.
These datasets are considered "gold standard" because they were manually annotated by experts in ancient Chinese literature.
Key Insights and Ideas
- Domain Pre-training is Non-Negotiable: The success of the model highlights that for specialized domains like ancient history or medicine, general-purpose LLMs are insufficient. The "linguistic gap" between modern and ancient Chinese is wide enough to require dedicated pre-training.
- CRF Still Holds Its Weight: In an era of "Attention is All You Need," this paper shows that traditional probabilistic graphical models like CRFs still provide a significant edge in sequence labeling tasks by enforcing structural constraints that Transformers might otherwise miss.
- The "Medical" Difficulty Spike: The model performed significantly better on historical texts (Datasets A and B) than on medical texts (Dataset C). This suggests that medical terminology in ancient Chinese is particularly sparse and difficult to generalize, even for specialized models.
Key Results
The proposed model consistently outperformed the official baseline (which used SikuRoBERTa-BiLSTM-CRF):
- Overall Performance: The model achieved a Total F1 score of 83.74%, a 3.13% improvement over the baseline.
- Precision and Recall: It saw a 1.51% increase in Precision and a substantial 4.74% increase in Recall.
- Dataset Specifics: The most dramatic improvement was seen in Dataset C (Medical), where the F1 score jumped from 72.40% (baseline) to 80.23%. This indicates their approach is particularly robust for highly specialized technical vocabulary.
Limitations and Future Work
Despite the strong results, the authors noted a few areas for improvement:
- Entity Confusion: The model occasionally confuses official titles or time-related terms with other entity types in historical texts.
- Data Scarcity in Medicine: In Dataset C, certain medical terms appear infrequently, making it harder for the model to learn them effectively.
- Future Directions: The team plans to investigate ways to reduce confusion between similar entity categories and improve the model's ability to handle rare, "long-tail" entities that appear only a handful of times in the training data.
Summary updated