TL;DR
This paper introduces a generative approach to Roman Imperial coin cataloguing, moving away from rigid classification to a flexible image-to-text framework. By fine-tuning the Qwen3-VL-8B-Instruct model on roughly 30,000 auction records, the authors achieved a 40% reduction in transcription errors compared to GPT-4.1, demonstrating that Vision-Language Models (VLMs) can learn to "read" ancient inscriptions even on worn, centuries-old surfaces.
What Problem Does This Paper Address?
Cataloguing ancient coins is a highly specialized task that requires identifying legends (inscriptions) and complex pictorial motifs to match a specimen to standard reference catalogues like Roman Imperial Coinage (RIC).
Traditional computer vision methods treat this as a classification problem: "Is this Emperor Augustus or Nero?" or "Is this a 'Victory' or 'Hercules' motif?" However, this approach fails to scale. There are tens of thousands of coin types, many with only slight variations in a single letter or a small visual detail. Furthermore, classification models can't handle "out-of-vocabulary" coins—specimens that don't fit into a predefined label set. General-purpose AI models also struggle because they aren't trained on the specific, abbreviated Latin and specialized numismatic terminology used in this field.
Methods
The authors reframe coin description as four distinct image-to-text generation tasks:
- Obverse Legend Transcription: Reading the text around the portrait.
- Reverse Legend Transcription: Reading the text on the back.
- Obverse Description: Describing the portrait’s attributes (e.g., headwear, orientation).
- Reverse Description: Describing the central imagery and objects.
Key techniques include: * Parameter-Efficient Fine-Tuning (PEFT): They used QLoRA to train separate adapters for each task. This allowed the 8-billion parameter model to be adapted on a single 20 GB GPU. * Structured Label Extraction: Since auction records are often messy and multilingual, the authors used GPT-4.1-mini to extract structured data, which was then deterministically normalized (e.g., mapping 'u' to 'v' for Latin inscriptions). * Rim-Focused Augmentation: To prevent the model from simply "guessing" the legend based on the central portrait, they created augmented training images where the center was blurred, forcing the model to attend to the characters along the rim. * Bust-Type Taxonomy: Obverse descriptions were standardized against a formal numismatic taxonomy, with missing labels filled in by dedicated Vision Transformer (ViT) classifiers.
Datasets
The study utilized a dataset derived from ACSearch, an online archive of ancient coin auctions.
- Size: Approximately 30,000 Roman Imperial coins.
- Processing: Images were split into separate obverse and reverse crops and resized to 512x512.
- Benchmarks: The authors evaluated performance on a standard test set and a more challenging "unseen-legend" subset, containing inscriptions that never appeared in the training data. This specifically tests the model's ability to transcribe new text rather than just memorizing known strings.
Key Insights and Ideas
- Generation > Classification: Generative models are more robust to the "long tail" of rare coin types. They can produce partial transcriptions for worn coins and describe unique scenes that don't fit a fixed category.
- The Domain Gap is Real: General-purpose VLMs like GPT-4.1 perform poorly on coins out of the box. Fine-tuning is essential to teach the model the specific "visual language" of numismatics.
- Reading vs. Memorizing: The model's success on the "unseen-legend" subset is a major "aha" moment. It proves the VLM actually learned to recognize Latin characters on metal surfaces, a transferable skill that goes beyond simple pattern matching.
- Transparency: Unlike a classifier that just spits out a label, a generative model provides an intermediate description. An expert can look at the generated text, see exactly where the model might have misread a letter, and correct it.
Key Results
The fine-tuned (SFT) model significantly outperformed both the base Qwen model and GPT-4.1:
- Legend Transcription: On the full test set, the SFT model reduced the Normalized Levenshtein Distance (NLD)—a measure of edit distance—from 0.33 (GPT-4.1) to 0.19.
- Reverse Imagery: The "Structured Score" (measuring the recovery of key entities and objects) jumped from 0.20 for GPT-4.1 to 0.50 for the SFT model.
- Portrait Identification: The base Qwen model was nearly blind to portrait subjects (3% accuracy), while the fine-tuned version reached 68%, a massive leap in domain-specific recognition.
Limitations and Future Work
- Imperfect Teachers: The training labels come from auction descriptions, which can be inconsistent or incomplete. The model is essentially learning from "noisy" experts.
- "In the Wild" Performance: The dataset uses high-quality auction photos. It remains to be seen how well these models handle lower-quality photos of corroded coins found in the field or excavations.
- Closing the Loop: The current system generates descriptions but doesn't yet automatically link them to a specific catalogue entry. The authors suggest adding a semantic search step to match generated descriptions to a reference database for full automated identification.
- Uncertainty: Future iterations could benefit from the model explicitly marking unreadable characters or providing a "reasoning trace" for its transcriptions.
Summary updated