Introduction
Archives serve as the fundamental backbone of institutional memory and historical preservation. However, the rapid digitization of records has created a significant data management challenge; for instance, electronic archives in China reached over 1,600 Terabytes by 2021. Manually processing this volume of data for retention, confidentiality, and public access is increasingly untenable. While general-purpose Large Language Models (LLMs) like GPT-4 have shown promise in broad natural language tasks, they often struggle with the specialized administrative jargon, historical linguistic variations, and complex regulatory frameworks inherent in archival science. To address this, we introduce ArcGPT, the first LLM specifically pre-trained for the archival domain, alongside AMBLE, a comprehensive benchmark for evaluating AI in real-world archival workflows.
Methodology
ArcGPT is a 7-billion parameter model built upon the BatGPT architecture. Its development focused on three core pillars:
- Domain-Specific Pre-training: The model was exposed to a massive corpus of archival data, including professional journals and historical records. This allows ArcGPT to internalize the evolution of administrative language over decades.
- Contextual Awareness: By training on actual archival records, the model learns the specific phraseology and structural patterns used by professional archivists.
- Instruction Tuning: ArcGPT underwent fine-tuning to follow complex instructions related to archival tasks, such as determining document sensitivity or predicting preservation timelines.
The AMBLE Benchmark
To rigorously evaluate performance, we developed the Archival Multi-task Benchmark for LLM Evaluation (AMBLE). This dataset comprises 14,469 training samples and 1,000 test samples, all meticulously labeled by archival science experts. AMBLE targets four critical tasks: Retention Period Prediction, Open Access Identification, Confidentiality Prediction, and Post-OCR Processing.
Performance Evaluation
Experimental results demonstrate that ArcGPT significantly outperforms other generative models in archival contexts. In classification-heavy tasks like confidentiality prediction, ArcGPT achieved an F1 score of 94.40, surpassing general-purpose models like ChatGLM and Chinese-LLaMA-Alpaca.
While ArcGPT is the leading generative model, the evaluation revealed a "performance gap" where smaller, task-specific predictive models (such as RoBERTa) still maintain a slight edge in pure classification accuracy. Furthermore, in Post-OCR processing—the task of correcting text from noisy scans—ArcGPT proved competitive with other LLMs but trailed behind specialized Chinese Spelling Correction (CSC) models like Mengzi-T5. This suggests that while LLMs excel at contextual understanding, specialized tools remain superior for low-level text cleaning.
Conclusion and Future Work
ArcGPT represents a significant advancement in applying generative AI to specialized administrative domains. By bridging the gap between general linguistic capability and archival expertise, it provides a scalable solution for modern record management. Future research will focus on optimizing the model's OCR correction capabilities and further refining its generative architecture to match the precision of specialized predictive classifiers. The authors encourage the archival community to integrate ArcGPT into active workflows to facilitate iterative, real-world improvement.
Summary updated