Chinese Character Image Clustering and Classification Based on Object Embedding Model (Student Abstract)

Mengting Wang, Xun Liang, Yang XueAAAI 2021 · DOI

Abstract

Identifying calligraphic styles and fonts in Chinese characters typically requires large labeled datasets for supervised learning. This study introduces Image2Vec, an unsupervised approach that treats character images as sequential data, applying Natural Language Processing (NLP) embedding techniques to extract structural features. By utilizing the Paragraph Vector - Distributed Memory (PV-DM) model, the researchers successfully clustered and classified both historical and modern Chinese scripts. The model achieved up to 100% precision in certain font styles, significantly outperforming traditional unsupervised benchmarks and demonstrating the potential for automated identification in art history and digital humanities.

Introduction

The identification of specific writers or fonts in Chinese calligraphy is challenging due to intricate stroke patterns and subtle stylistic variations. While supervised Convolutional Neural Networks (CNNs) achieve high accuracy, they depend on massive amounts of labeled data, which is often unavailable for ancient manuscripts or unknown calligraphers. Historically, unsupervised methods have struggled with feature extraction, typically yielding accuracy around 60%. This research bridges the gap by applying deep learning embeddings to unlabeled Chinese typographic data.

Methodology: The Image2Vec Model

The core innovation of this work is the adaptation of the "distribution hypothesis" from linguistics—the idea that words in similar contexts share similar meanings—to the visual domain. The researchers propose that pixels in similar spatial contexts represent similar structural "meanings."

The Image2Vec process consists of two primary stages:

  1. Preparation Step: Images are converted into grayscale matrices. Given the sequential nature of Chinese text and the lack of rotation in the dataset, the researchers focused on x-axis information, treating the horizontal pixel sequence as a "document."
  2. Object Embedding Step: The model employs the PV-DM (Paragraph Vector - Distributed Memory) algorithm. Rather than predicting words, the model learns to represent the entire character image as a high-dimensional vector.
Figure 1: The Image2Vec architecture utilizing PV-DM for unsupervised learning of image vectors.

Following embedding, T-SNE is used for visual clustering, and Logistic Regression is applied to evaluate the discriminative power of the learned features.

Datasets

The model was evaluated on two distinct datasets:

  • Dataset 1: Historical regular script works by four master calligraphers: Zongyuan Liu, Xun Ouyang, Zhenqing Yan, and Mengfu Zhao.
  • Dataset 2: Modern fonts including Kai (standard script), Imitation Song Dynasty, and Hei (sans-serif).

Key Results and Discussion

Image2Vec demonstrated a substantial performance leap over existing unsupervised methods. While traditional subtractive clustering and Super Paramagnetic Clustering (SPC) typically reach 61.3% and 79% accuracy respectively, Image2Vec achieved near-perfect results in several categories. On Dataset 2, the model achieved 100% precision for the Imitation Song style and approximately 90% for the Kai and Hei styles.

Figure 4: T-SNE visualization showing clear, distinct clusters for different font styles learned without labels.

As shown in the T-SNE plots, the model successfully learned meaningful geometric properties of the characters on its own, resulting in well-separated clusters. This suggests that unsupervised deep learning can capture fine-grained details—like stroke thickness and shape—that traditional clustering methods miss.

Conclusion and Future Work

The success of Image2Vec demonstrates that NLP-inspired embedding techniques can be effectively repurposed for visual feature extraction. A current limitation is the exclusion of y-axis information, which may be necessary for more complex or rotated layouts. Future research will focus on incorporating vertical context and applying the model to identify works by unknown calligraphers, potentially providing an expert-level tool for art historians and digital archivists.

Summary updated

BibTeX
@inproceedings{wang2021chinese,
  author = {Mengting Wang and Xun Liang and Yang Xue},
  title = {{Chinese Character Image Clustering and Classification Based on Object Embedding Model (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i18.17953},
}

Loading related content...

Figure