TL;DR
EpiAgent is an intelligent, agent-centric system that mimics human epigraphers to restore ancient, degraded inscriptions. By replacing rigid, single-pass AI pipelines with an iterative Observe–Conceive–Execute–Reevaluate loop governed by an LLM central planner, it dynamically coordinates language models, visual tools, and historical context to achieve expert-level restoration of both textual meaning and calligraphic style.
What Problem Does This Paper Address?
Ancient inscriptions and paper rubbings are invaluable cultural artifacts that preserve both historical text and the artistic essence of ancient calligraphy. However, centuries of weathering, physical damage, and human intervention leave them severely degraded.
Existing AI approaches typically treat this as a standard image-to-image translation task, using fixed neural network pipelines to map degraded characters directly to clean ones. This rigid strategy frequently fails because inscription degradation is highly complex and spatially coupled. As a result, standard models suffer from "over-restoration" (inventing fake stroke structures), "under-restoration" (leaving noise behind), or severe glyph distortion that ruins the calligraphic authenticity of the script.
Methods
EpiAgent addresses these challenges by formalizing the workflow of human experts into a hierarchical, closed-loop paradigm called Observe–Conceive–Execute–Reevaluate (OCER), orchestrated by a Large Language Model (LLM) central planner (Kimi-K2).
The process operates in four distinct stages:
- Observe: The system builds a multi-modal understanding of the inscription. A Multimodal LLM (MLLM) generates an initial layout and text hypothesis. Concurrently, a specialized Corrective Language Model (CLM) enhanced with Retrieval-Augmented Generation (RAG) queries historical Chinese corpora to fix character misidentifications. Finally, visual modules segment the pixel-level degradation and assess its severity.
- Conceive: Instead of applying a generic fix, the central planner analyzes the observation data alongside historical execution logs (distilled experience) to devise a customized, character-by-character restoration plan.
- Execute: The planner schedules and deploys a composable toolkit of specialized, diffusion-based tools:
- Background Denoising ($f_{deno}$): Cleans up surface noise while protecting stroke structures.
- Stroke Completion ($f_{inp}$): Inpaints missing or severely damaged regions.
- Font Imitation ($f_{imi}$): Synthesizes stylistically consistent glyphs based on high-quality exemplars from the same historical stele.
- Character Retrieval ($f_{ret}$): Acts as a fallback by finding identical, intact characters elsewhere in the inscription.
- Reevaluate: The restored image is passed through an automated evaluation block that scores Text Authenticity (via OCR accuracy) and Style Consistency (via calligraphic embedding similarity). It can also ingest optional human expert feedback. If the results fall short of defined thresholds, the planner updates its strategy and triggers another iteration of the loop.
Datasets
The authors evaluated EpiAgent using the Chinese Inscription Rubbing Images (CIRI) dataset, which contains:
- 24,000 synthetic inscription images (20k for training, 4k for testing).
- 2,000 real-world ancient rubbings, split into two evaluation subsets: Type I (R-I), which shares some degradation patterns seen in the synthetic training data, and Type II (R-II), which features entirely unseen, real-world degradation patterns to test generalization.
Key Insights and Ideas
- Language Guides Vision: One of the system's biggest strengths is using linguistic context to solve visual problems. By correcting the textual reading using historical corpora before running visual restoration tools, the agent knows exactly what character structure it needs to reconstruct, making visual inpainting far less ambiguous.
- Tool Sequencing Matters: The paper demonstrates that applying restoration tools in the wrong order (e.g., attempting stroke completion before background denoising) introduces severe artifacts. Adaptive planning based on past success rates prevents the system from making these destructive mistakes.
- The Power of Reflection: Incorporating a multi-perspective self-reflection loop allows the agent to catch its own mistakes (like over-restoration) and roll back or alter its approach, mimicking the meticulous nature of human conservators.
Key Results
EpiAgent significantly outperformed state-of-the-art unified image restoration models (like Restormer and MambaIR) and specialized text enhancement networks (like CharFormer and DocDiff).
Quantitative Performance (Testing Set S)
- Image Quality: EpiAgent achieved a PSNR of 22.14 and SSIM of 0.9684, markedly better than the closest specialized baseline, IR3 (PSNR: 21.15, SSIM: 0.9540).
- Text Recognition Accuracy: It achieved a Top-1 Accuracy of 98.89% and a Normalized Edit Distance score (1-NED) of 0.9069, coming remarkably close to the performance metrics of completely intact, undamaged inscriptions.
Human Expert Evaluation
In a blind user study involving over twenty professional epigraphers and scholars, EpiAgent was overwhelmingly preferred, capturing a 59.66% Top-1 Ranking, compared to just 15.60% for the next-best method.
Limitations and Future Work
While EpiAgent marks a major step forward, its performance relies heavily on the availability of high-quality reference exemplars from the same stele to perform accurate font imitation. If an inscription is written in a completely unique or highly rare calligraphic style with no surviving reference material, style replication remains difficult.
Future work could explore unsupervised style-extraction techniques that can infer calligraphic rules directly from a few partially damaged characters without requiring external historical exemplars.
Summary updated