Incorporating Lexicon-Aligned Prompting in Large Language Model for Tangut–Chinese Translation

Yuxi Zheng, Jingsong YuNAACL 2025

TL;DR

This paper tackles the challenging task of translating Tangut, a low-resource historical language, into Chinese using large language models (LLMs). By integrating expert-curated dictionaries and a novel chain-of-thought prompting strategy, they achieve state-of-the-art results for both literal and idiomatic translations, paving the way for broader digital preservation of such endangered scripts.

What Problem Does This Paper Address?

The Tangut script, used by the Western Xia dynasty, is a rich source of historical and cultural information but remains difficult to access due to its complex nature, lack of continuous usage, and a severe scarcity of parallel translation data. Traditional manual translation methods are labor-intensive and not scalable. While LLMs have shown promise for low-resource languages, Tangut translation has been largely unexplored. This paper aims to bridge this gap by developing a computational approach for Tangut-Chinese translation that leverages LLMs while addressing the unique challenges posed by this historical script.

Methods

The authors propose a hybrid framework that combines a pre-trained Classical Chinese LLM (QwenClassical) with expert knowledge and advanced prompting techniques.

  • Lexicon Integration: They incorporate definitions from the "Concise Tangut-Chinese Dictionary." For each Tangut character in the input, they provide either full or simplified (single-keyword) definitions as part of the prompt. This helps the LLM disambiguate characters and ground its translations in accurate semantics.
  • Chain-of-Thought (CoT) Prompting: For idiomatic translation, they employ a two-step CoT strategy. First, the model generates a provisional literal translation. Second, it refines this literal translation into fluent Chinese using in-context examples. This mimics human translation workflows and helps reduce error propagation.
  • Fine-tuning: The QwenClassical LLM is fine-tuned on a newly compiled Tangut-Chinese parallel corpus. They also utilize LoRA finetuning with the Zero2 technique for efficiency.

Datasets

  • Primary Dataset: A Tangut-Chinese parallel corpus compiled from two sources:
  • "Three Generations Illuminated Collection": 569 sentence pairs with four-line aligned translations. This was split into 541 training pairs and 28 test pairs.
  • "Avatamsaka Sūtra Vol. 77": 525 sentence pairs, used as a generalization dataset.
  • Lexical Resources: The "Concise Tangut-Chinese Dictionary" containing 6,703 Tangut characters with 8,245 Chinese definitions.

Key Insights and Ideas

  • Lexicons are Crucial for Low-Resource Scripts: Integrating even simplified dictionary definitions significantly boosts translation performance, especially for literal translation. This highlights how domain-specific knowledge can compensate for limited parallel data.
  • CoT Prompting Unlocks Idiomatic Translation: The two-step CoT approach for idiomatic translation is highly effective, outperforming direct prompting by a substantial margin. This suggests that breaking down complex translation tasks into intermediate steps is beneficial for LLMs.
  • LLMs Can Adapt to Historical Languages: The QwenClassical model, pre-trained on classical Chinese, demonstrates strong adaptability to Tangut, a distinct historical script, after fine-tuning.
  • Few-Shot Learning is Powerful: The model shows impressive performance even with very small training sets, indicating its potential for rapid adaptation to new Tangut genres or related low-resource languages.

Key Results

  • Literal Translation: The best performance was achieved by QwenClassical with simplified dictionary definitions, reaching a BLEU-4 score of 72.33, outperforming the base Qwen model by 2.83 points.
  • Idiomatic Translation: The combination of QwenClassical, simplified definitions, and CoT prompting achieved a BLEU-4 score of 64.20, significantly outperforming direct prompting by 12.14 points.
  • Data Efficiency: With only 100 training samples, the model achieved a BLEU-4 score of 62.83 for literal translation, demonstrating strong few-shot learning capabilities.
  • Cross-Domain Generalization: The model showed adaptability to new Tangut genres, achieving BLEU-4 scores of 30.62 (literal) and 37.00 (idiomatic) on the Avatamsaka Sūtra dataset.
  • Human Evaluation: Expert raters found the QwenClassical+DictSingle+PromptCoT model to have the highest fluency (4.12/5) and faithfulness (4.35/5).

Limitations and Future Work

The authors acknowledge several limitations and suggest future directions:

  • Data Scarcity: Expanding the parallel corpus to include more genres (legal documents, poetry) and dialects is crucial for better generalization.
  • Context Handling: Current models struggle with multi-sentence dependencies. Future work should explore document-level translation and multimodal approaches (e.g., integrating textual and glyph features).
  • Human-in-the-Loop Optimization: Developing interactive tools for expert validation and error correction would enhance practical utility.

Summary updated

BibTeX
@inproceedings{zheng2025incorporating,
  author = {Yuxi Zheng and Jingsong Yu},
  title = {{Incorporating Lexicon-Aligned Prompting in Large Language Model for Tangut–Chinese Translation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure