Image Translation for Oracle Bone Character Interpretation

Feng Gao, Jingping Zhang, Yongge Liu, Yahong HanSymmetry 2022 · DOI · OpenReview

TL;DR

This paper tackles the challenging task of interpreting ancient Oracle Bone Characters (OBCs) by proposing a novel symmetrical image translation method using generative adversarial networks (GANs). By translating OBCs into modern Chinese characters, the model provides crucial glyph information to assist experts in deciphering these historical artifacts.

What Problem Does This Paper Address?

The interpretation of Oracle Bone Characters (OBCs), the earliest known form of Chinese writing, is a significant challenge for ancient Chinese language experts. This difficulty stems from several factors: a limited number of discovered OBCs (around 4500), the labor-intensive process of detecting and organizing them, and a substantial visual gap between OBCs and modern Chinese characters. Consequently, a large portion of OBCs remain undeciphered. While previous research has focused on OBC detection and recognition, these methods struggle with uninterpreted characters. This paper aims to bridge this gap by leveraging the inherent relationship between OBC glyphs and their modern Chinese character descendants to aid in interpretation.

Methods

The core of this paper's contribution lies in a symmetrical image translation method based on generative adversarial networks (GANs). This approach is inspired by the success of GANs in image-to-image translation tasks. The proposed method utilizes a two-stage training process:

  • Stage 1: Encoder-Decoder Training: A GAN, specifically a conditional GAN (cGAN) due to the need for conditional generation, is trained to translate Oracle Bone Characters into modern Chinese characters. This stage uses paired OBC images and their corresponding modern Chinese character interpretations. The symmetrical structure involves an encoder to capture the essence of the OBC and a decoder to reconstruct it as a modern character. Adversarial loss and L₁ loss are used to ensure realistic image generation and minimize blur.
  • Stage 2: Knowledge Expansion (Decoder Fine-tuning): To address the challenge of generalizing to unseen modern Chinese characters (those not present in the training set), the encoder from Stage 1 is replaced with a randomly initialized one. The decoder is then fine-tuned using modern Chinese characters as input. This "knowledge expansion" allows the decoder to better understand the rendering of modern characters, improving its ability to translate OBCs into a wider range of modern glyphs.

The authors highlight that this symmetrical structure, combined with knowledge expansion, is a novel approach for OBC interpretation.

Datasets

The paper utilizes a dataset of Oracle Bone Character (OBC) images, which is split into three parts:

  • Training Set: Contains 19,526 OBC images interpreted as 1205 traditional Chinese characters.
  • Test Set: Contains 1520 OBC images interpreted as 50 traditional Chinese characters, crucially, these characters do not appear in the training set. This set is used to evaluate the model's generalization ability.
  • Uninterpreted Data Set: A significant set of 20,927 OBC images that have not been interpreted by experts. This dataset is used to demonstrate the practical application of the method in assisting interpretation.

No new benchmark datasets were introduced; existing OBC image data was used and categorized for the specific tasks.

Key Insights and Ideas

  • Bridging the Glyph Gap: The fundamental insight is that the evolutionary link between OBCs and modern Chinese characters provides a learnable glyph relationship. This relationship can be exploited by image translation models.
  • Symmetrical Structure for Interpretation: A symmetrical encoder-decoder architecture is particularly well-suited for this task, allowing for the representation of OBC features and their subsequent translation into a familiar modern form.
  • Knowledge Expansion for Generalization: The two-stage training, especially the decoder fine-tuning with modern characters, is crucial for overcoming the limited number of interpreted OBCs and the vastness of the modern Chinese character set. This addresses the challenge of generalization to unseen characters.
  • Assisting, Not Replacing, Experts: The method is positioned as a tool to aid ancient Chinese language experts by providing glyph information, rather than fully automating the interpretation process.

Key Results

  • Training Set Performance: The model demonstrated strong performance on the training set, accurately translating OBC images into their corresponding modern Chinese characters. Retrieval accuracy reached up to 99.4% for the top 200 predictions.
  • Test Set Performance: On the test set (containing unseen modern characters), the model showed consistency between its output and ground truth images, indicating its ability to generalize. However, retrieval accuracy was lower (up to 49.0% for top 200), highlighting the inherent difficulty of the task.
  • Uninterpreted Data Set Results: The method successfully translated uninterpreted OBC images into glyphs that were similar to modern Chinese characters. These translated images were then evaluated by experts and provided valuable information for their interpretation.
  • L₁ Distance: The L₁ distance between the model's output and ground truth on the test set initially decreased but then increased after 100 epochs, suggesting an overfitting problem beyond that point.

Limitations and Future Work

  • Limitations:
    • The inherent large gap between OBCs and modern Chinese characters still poses a significant challenge, leading to occasional blurred outputs for uninterpreted characters.
  • The model's effectiveness is still limited, and further improvement is needed for translating OBCs to modern Chinese characters.
  • Overfitting was observed after 100 epochs of training on the test set.
  • Future Work:
    • The authors plan to translate OBCs into other interpreted ancient Chinese characters that have a smaller glyph gap with OBCs. This is expected to yield better performance.
  • Further research is needed to improve the accuracy and clarity of translations, especially for unseen and uninterpreted OBCs.

Summary updated

Abstract

The Oracle Bone Characters are the earliest known ancient Chinese characters and are an important record of the civilization of ancient China.The interpretation of the Oracle Bone Characters is challenging and requires professional knowledge from ancient Chinese language experts. Although some works have utilized deep learning to perform image detection and recognition using the Oracle Bone Characters, these methods have proven difficult to use for the interpretation of uninterpreted Oracle Bone Character images. Inspired by the prior knowledge that there exists a relation between glyphs from Oracle Bone Character images and images of modern Chinese characters, we proposed a method of image translation from Oracle Bone Characters to modern Chinese characters based on the use of a generative adversarial network to capture the implicit relationship between glyphs from Oracle Bone Characters and modern Chinese characters. The image translation process between Oracle Bone Characters and the modern Chinese characters forms a symmetrical structure, comprising an encoder and decoder. To our knowledge, our symmetrical image translation method is the first of its kind used for the task of interpreting Oracle Bone Characters. Our experiments indicated that our image translation method can provide glyph information to aid in the interpretation of Oracle Bone Characters.
BibTeX
@article{gao2022image,
  author = {Feng Gao and Jingping Zhang and Yongge Liu and Yahong Han},
  title = {{Image Translation for Oracle Bone Character Interpretation}},
  journal = {Symmetry},
  year = {2022},
  doi = {10.3390/sym14040743},
}
Figure