EpiAgent: An Agent-Centric System for Ancient Inscription Restoration

Shipeng Zhu, Ang Chen, Na Nie, Pengfei Fang, Min-Ling Zhang, Hui Xue

TL;DR

EpiAgent is an intelligent, agent-centric system that mimics human epigraphers to restore ancient, degraded inscriptions. By replacing rigid, single-pass AI pipelines with an iterative Observe–Conceive–Execute–Reevaluate loop governed by an LLM central planner, it dynamically coordinates language models, visual tools, and historical context to achieve expert-level restoration of both textual meaning and calligraphic style.

What Problem Does This Paper Address?

Ancient inscriptions and paper rubbings are invaluable cultural artifacts that preserve both historical text and the artistic essence of ancient calligraphy. However, centuries of weathering, physical damage, and human intervention leave them severely degraded.

Existing AI approaches typically treat this as a standard image-to-image translation task, using fixed neural network pipelines to map degraded characters directly to clean ones. This rigid strategy frequently fails because inscription degradation is highly complex and spatially coupled. As a result, standard models suffer from "over-restoration" (inventing fake stroke structures), "under-restoration" (leaving noise behind), or severe glyph distortion that ruins the calligraphic authenticity of the script.

Methods

EpiAgent addresses these challenges by formalizing the workflow of human experts into a hierarchical, closed-loop paradigm called Observe–Conceive–Execute–Reevaluate (OCER), orchestrated by a Large Language Model (LLM) central planner (Kimi-K2).

Figure 2. Illustration of the EpiAgent framework, which mimics the restoration workflow of human epigraphers.

The process operates in four distinct stages:

  1. Observe: The system builds a multi-modal understanding of the inscription. A Multimodal LLM (MLLM) generates an initial layout and text hypothesis. Concurrently, a specialized Corrective Language Model (CLM) enhanced with Retrieval-Augmented Generation (RAG) queries historical Chinese corpora to fix character misidentifications. Finally, visual modules segment the pixel-level degradation and assess its severity.
  2. Conceive: Instead of applying a generic fix, the central planner analyzes the observation data alongside historical execution logs (distilled experience) to devise a customized, character-by-character restoration plan.
  3. Execute: The planner schedules and deploys a composable toolkit of specialized, diffusion-based tools:
  4. Background Denoising ($f_{deno}$): Cleans up surface noise while protecting stroke structures.
  5. Stroke Completion ($f_{inp}$): Inpaints missing or severely damaged regions.
  6. Font Imitation ($f_{imi}$): Synthesizes stylistically consistent glyphs based on high-quality exemplars from the same historical stele.
  7. Character Retrieval ($f_{ret}$): Acts as a fallback by finding identical, intact characters elsewhere in the inscription.
  8. Reevaluate: The restored image is passed through an automated evaluation block that scores Text Authenticity (via OCR accuracy) and Style Consistency (via calligraphic embedding similarity). It can also ingest optional human expert feedback. If the results fall short of defined thresholds, the planner updates its strategy and triggers another iteration of the loop.

Datasets

The authors evaluated EpiAgent using the Chinese Inscription Rubbing Images (CIRI) dataset, which contains:

  • 24,000 synthetic inscription images (20k for training, 4k for testing).
  • 2,000 real-world ancient rubbings, split into two evaluation subsets: Type I (R-I), which shares some degradation patterns seen in the synthetic training data, and Type II (R-II), which features entirely unseen, real-world degradation patterns to test generalization.

Key Insights and Ideas

  • Language Guides Vision: One of the system's biggest strengths is using linguistic context to solve visual problems. By correcting the textual reading using historical corpora before running visual restoration tools, the agent knows exactly what character structure it needs to reconstruct, making visual inpainting far less ambiguous.
  • Tool Sequencing Matters: The paper demonstrates that applying restoration tools in the wrong order (e.g., attempting stroke completion before background denoising) introduces severe artifacts. Adaptive planning based on past success rates prevents the system from making these destructive mistakes.
  • The Power of Reflection: Incorporating a multi-perspective self-reflection loop allows the agent to catch its own mistakes (like over-restoration) and roll back or alter its approach, mimicking the meticulous nature of human conservators.

Key Results

EpiAgent significantly outperformed state-of-the-art unified image restoration models (like Restormer and MambaIR) and specialized text enhancement networks (like CharFormer and DocDiff).

Figure 1. Illustration of the restored ancient inscription samples.

Quantitative Performance (Testing Set S)

  • Image Quality: EpiAgent achieved a PSNR of 22.14 and SSIM of 0.9684, markedly better than the closest specialized baseline, IR3 (PSNR: 21.15, SSIM: 0.9540).
  • Text Recognition Accuracy: It achieved a Top-1 Accuracy of 98.89% and a Normalized Edit Distance score (1-NED) of 0.9069, coming remarkably close to the performance metrics of completely intact, undamaged inscriptions.

Human Expert Evaluation

In a blind user study involving over twenty professional epigraphers and scholars, EpiAgent was overwhelmingly preferred, capturing a 59.66% Top-1 Ranking, compared to just 15.60% for the next-best method.

Limitations and Future Work

While EpiAgent marks a major step forward, its performance relies heavily on the availability of high-quality reference exemplars from the same stele to perform accurate font imitation. If an inscription is written in a completely unique or highly rare calligraphic style with no surviving reference material, style replication remains difficult.

Future work could explore unsupervised style-extraction techniques that can infer calligraphic rules directly from a few partially damaged characters without requiring external historical exemplars.

Summary updated

Abstract

Ancient inscriptions, as repositories of cultural memory, have suffered from centuries of environmental and human-induced degradation. Restoring their intertwined visual and textual integrity poses one of the most demanding challenges in digital heritage preservation. However, existing AI-based approaches often rely on rigid pipelines, struggling to generalize across such complex and heterogeneous real-world degradations. Inspired by the skill-coordinated workflow of human epigraphers, we propose EpiAgent, an agent-centric system that formulates inscription restoration as a hierarchical planning problem. Following an Observe-Conceive-Execute-Reevaluate paradigm, an LLM-based central planner orchestrates collaboration among multimodal analysis, historical experience, specialized restoration tools, and iterative self-refinement. This agent-centric coordination enables a flexible and adaptive restoration process beyond conventional single-pass methods. Across real-world degraded inscriptions, EpiAgent achieves superior restoration quality and stronger generalization compared to existing methods. Our work marks an important step toward expert-level agent-driven restoration of cultural heritage. The code is available at https://github.com/blackprotoss/EpiAgent.
BibTeX
@inproceedings{zhu2026epiagent,
  author = {Shipeng Zhu and Ang Chen and Na Nie and Pengfei Fang and Min-Ling Zhang and Hui Xue},
  title = {{EpiAgent: An Agent-Centric System for Ancient Inscription Restoration}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2026},
  url = {https://openreview.net/forum?id=jzlBJVrOpB},
}
Figure