From Classification to Generation: Vision-Language Models for Roman Imperial Coin Description

Pedro Lourenco, Ognjen ArandjelovićarXiv 2026 · arXiv

TL;DR

This paper introduces a generative approach to Roman Imperial coin cataloguing, moving away from rigid classification to a flexible image-to-text framework. By fine-tuning the Qwen3-VL-8B-Instruct model on roughly 30,000 auction records, the authors achieved a 40% reduction in transcription errors compared to GPT-4.1, demonstrating that Vision-Language Models (VLMs) can learn to "read" ancient inscriptions even on worn, centuries-old surfaces.

What Problem Does This Paper Address?

Cataloguing ancient coins is a highly specialized task that requires identifying legends (inscriptions) and complex pictorial motifs to match a specimen to standard reference catalogues like Roman Imperial Coinage (RIC).

Traditional computer vision methods treat this as a classification problem: "Is this Emperor Augustus or Nero?" or "Is this a 'Victory' or 'Hercules' motif?" However, this approach fails to scale. There are tens of thousands of coin types, many with only slight variations in a single letter or a small visual detail. Furthermore, classification models can't handle "out-of-vocabulary" coins—specimens that don't fit into a predefined label set. General-purpose AI models also struggle because they aren't trained on the specific, abbreviated Latin and specialized numismatic terminology used in this field.

Methods

The authors reframe coin description as four distinct image-to-text generation tasks:

  1. Obverse Legend Transcription: Reading the text around the portrait.
  2. Reverse Legend Transcription: Reading the text on the back.
  3. Obverse Description: Describing the portrait’s attributes (e.g., headwear, orientation).
  4. Reverse Description: Describing the central imagery and objects.

Key techniques include: * Parameter-Efficient Fine-Tuning (PEFT): They used QLoRA to train separate adapters for each task. This allowed the 8-billion parameter model to be adapted on a single 20 GB GPU. * Structured Label Extraction: Since auction records are often messy and multilingual, the authors used GPT-4.1-mini to extract structured data, which was then deterministically normalized (e.g., mapping 'u' to 'v' for Latin inscriptions). * Rim-Focused Augmentation: To prevent the model from simply "guessing" the legend based on the central portrait, they created augmented training images where the center was blurred, forcing the model to attend to the characters along the rim. * Bust-Type Taxonomy: Obverse descriptions were standardized against a formal numismatic taxonomy, with missing labels filled in by dedicated Vision Transformer (ViT) classifiers.

Datasets

The study utilized a dataset derived from ACSearch, an online archive of ancient coin auctions.

  • Size: Approximately 30,000 Roman Imperial coins.
  • Processing: Images were split into separate obverse and reverse crops and resized to 512x512.
  • Benchmarks: The authors evaluated performance on a standard test set and a more challenging "unseen-legend" subset, containing inscriptions that never appeared in the training data. This specifically tests the model's ability to transcribe new text rather than just memorizing known strings.

Key Insights and Ideas

  • Generation > Classification: Generative models are more robust to the "long tail" of rare coin types. They can produce partial transcriptions for worn coins and describe unique scenes that don't fit a fixed category.
  • The Domain Gap is Real: General-purpose VLMs like GPT-4.1 perform poorly on coins out of the box. Fine-tuning is essential to teach the model the specific "visual language" of numismatics.
  • Reading vs. Memorizing: The model's success on the "unseen-legend" subset is a major "aha" moment. It proves the VLM actually learned to recognize Latin characters on metal surfaces, a transferable skill that goes beyond simple pattern matching.
  • Transparency: Unlike a classifier that just spits out a label, a generative model provides an intermediate description. An expert can look at the generated text, see exactly where the model might have misread a letter, and correct it.

Key Results

The fine-tuned (SFT) model significantly outperformed both the base Qwen model and GPT-4.1:

  • Legend Transcription: On the full test set, the SFT model reduced the Normalized Levenshtein Distance (NLD)—a measure of edit distance—from 0.33 (GPT-4.1) to 0.19.
  • Reverse Imagery: The "Structured Score" (measuring the recovery of key entities and objects) jumped from 0.20 for GPT-4.1 to 0.50 for the SFT model.
  • Portrait Identification: The base Qwen model was nearly blind to portrait subjects (3% accuracy), while the fine-tuned version reached 68%, a massive leap in domain-specific recognition.
Fig. 1. Sample images from the dataset, each showing both sides of a single coin.

Limitations and Future Work

  • Imperfect Teachers: The training labels come from auction descriptions, which can be inconsistent or incomplete. The model is essentially learning from "noisy" experts.
  • "In the Wild" Performance: The dataset uses high-quality auction photos. It remains to be seen how well these models handle lower-quality photos of corroded coins found in the field or excavations.
  • Closing the Loop: The current system generates descriptions but doesn't yet automatically link them to a specific catalogue entry. The authors suggest adding a semantic search step to match generated descriptions to a reference database for full automated identification.
  • Uncertainty: Future iterations could benefit from the model explicitly marking unreadable characters or providing a "reasoning trace" for its transcriptions.

Summary updated

Abstract

Cataloguing a Roman Imperial coin involves reading its legends and identifying its imagery in order to match it to an entry in the standard reference catalogue. Previous work has automated individual parts of this process, such as identifying the issuer or recognizing reverse motifs, by formulating each part as a classification task over a predefined set of labels. We instead formulate coin description as image-to-text generation and adapt an open-weight vision-language model (VLM), Qwen3-VL-8B-Instruct, to four tasks: transcribing the obverse and reverse legends, and describing the obverse and reverse imagery. To obtain training data, we convert approximately 30,000 auction records into structured image-text pairs. We separate the two sides of each coin, extract and normalize legends and descriptions, and standardize obverse descriptions against a bust-type taxonomy. We then train a separate low-rank adapter for each task on a single 20GB GPU, and evaluate each task with metrics that we define to capture the key components of numismatic descriptions. The adapted models outperform both the unadapted base model and GPT-4.1 on all four tasks, reducing the normalized edit distance of legend transcriptions by over 40% relative to GPT-4.1. The improvement is smaller, but still present, on legends absent from the training data.
BibTeX
@misc{lourenco2026from,
  author = {Pedro Lourenco and Ognjen Arandjelović},
  title = {{From Classification to Generation: Vision-Language Models for Roman Imperial Coin Description}},
  year = {2026},
  eprint = {2611.14821},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.14821},
}
Figure