ArcAid: Analysis of Archaeological Artifacts using Drawings

Offry Hayon, Stefan Münger, Ilan Shimshoni, Ayellet Tal

Abstract

Archaeology is an intriguing domain for computer vision. It suffers not only from shortage in (labeled) data, but also from highly-challenging data, which is often extremely abraded and damaged. This paper proposes a novel semi-supervised model for classification and retrieval of images of archaeological artifacts. This model utilizes unique data that exists in the domain—manual drawings made by special artists. These are used during training to implicitly transfer the domain knowledge from the drawings to their corresponding images, improving their classification results. We show that while learning how to classify, our model also learns how to generate drawings of the artifacts, an important documentation task, which is currently performed manually. Last but not least, we collected a new dataset of stamp-seals of the Southern Levant.

Introduction

Archaeological artifacts present a unique challenge for computer vision. Unlike standard datasets, ancient items like stamp-seals and scarabs are often eroded, broken, or stained. Furthermore, because these items were handcrafted thousands of years ago, visual consistency is low even within the same category. Expert labeling is also a significant bottleneck, requiring specialized knowledge.

ArcAid addresses these issues by leveraging a "hidden" resource in archaeology: expert manual drawings. These illustrations emphasize diagnostic features while ignoring surface noise and damage. By aligning photographic embeddings with these clean, expert-interpreted drawings, ArcAid "teaches" models to see through thousands of years of wear.

Overview of the ArcAid framework: (a) Input image, (b) Classification by shape, (c) Classification by period, and (d) Automated drawing generation.

Methodology

The ArcAid framework uses a semi-supervised, multi-modal learning approach. It employs two separate encoders—one for drawings ($Enc_{Draw}$) and one for images ($Enc_{Im}$)—connected to a shared classification head.

The training is guided by three key loss functions:

  1. Similarity Loss ($\mathcal{L}_{Sim}$): Forces the embedding of a photograph to align with its corresponding drawing, encouraging the model to focus on the features highlighted by the artist.
  2. Classification Loss ($\mathcal{L}_{CE}$): A standard cross-entropy loss for labeled data (e.g., shape or historical period).
  3. Generation Loss ($\mathcal{L}_{Gen}$): A decoder reconstructs a drawing from the image embedding, ensuring the encoder preserves essential geometric details.

This semi-supervised approach allows the model to benefit from thousands of unlabeled image-drawing pairs, learning robust feature representations even without explicit class labels.

The CSSL Dataset

The researchers introduced the Corpus of the Stamp-seals of the Southern Levant (CSSL), a new benchmark containing 6,636 image-drawing pairs. Of these, 1,020 are expert-labeled into 10 shape classes and 3 historical periods (Middle Bronze, Late Bronze, and Iron Age). The model's generality was also validated on Egyptian hieroglyphs.

Key Results

ArcAid significantly outperforms standard backbones across various tasks:

  • Classification: On the CSSL dataset, shape classification accuracy rose from 80.8% to 90.5% when using the ArcAid framework with a DenseNet161 backbone.
  • Retrieval: Mean Average Precision (mAP) for artifact retrieval improved by up to 0.23.
  • Dating: The model successfully dated artifacts to specific sub-periods, a task typically requiring years of expert training.
  • Generation: The model generates high-quality archaeological illustrations that outperform standard edge detectors like DexiNed, capturing the "intent" of the original artist.
An example of an automatically generated archaeological drawing (Generation task).

Conclusion

ArcAid demonstrates that expert domain knowledge, captured in manual drawings, can be effectively transferred to deep learning models to overcome the challenges of noisy and scarce archaeological data. Beyond classification, the ability to automatically generate "archaeologically correct" drawings offers a valuable tool for documentation. Future work will explore 3D scans and other cultural heritage artifacts like pottery and coins.

Summary updated

Abstract

Archaeology is an intriguing domain for computer vision. It suffers not only from shortage in (labeled) data, but also from highly-challenging data, which is often extremely abraded and damaged. This paper proposes a novel semi-supervised model for classification and retrieval of images of archaeological artifacts. This model utilizes unique data that exists in the domain—manual drawings made by special artists. These are used during training to implicitly transfer the domain knowledge from the drawings to their corresponding images, improving their classification results. We show that while learning how to classify, our model also learns how to generate drawings of the artifacts, an important documentation task, which is currently performed manually. Last but not least, we collected a new dataset of stamp-seals of the Southern Levant. Our code 1 and dataset 2 are publicly available.
BibTeX
@inproceedings{hayon2024arcaid,
  author = {Offry Hayon and Stefan Münger and Ilan Shimshoni and Ayellet Tal},
  title = {{ArcAid: Analysis of Archaeological Artifacts using Drawings}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
  url = {https://openreview.net/forum?id=O7ddiXwsY1},
}
Figure