Images of Roman Imperial Denarii: A Curated Data Set for the Evaluation of Computer Vision Algorithms Applied to Ancient Numismatics, and an Overview of Challenges in the Field

Ognjen Arandjelović, Marios ZachariouSci 2020 · DOI

TL;DR

This paper introduces the first curated dataset of Roman Imperial Denarii, specifically designed to evaluate computer vision algorithms for ancient coin analysis, accounting for coin condition. It also provides a comprehensive overview of the challenges in this field, highlighting the need for standardized evaluation and robust datasets to drive progress.

What Problem Does This Paper Address?

The field of applying computer vision and machine learning to ancient numismatics is growing, but progress is hampered by a lack of standardized evaluation frameworks and a limited understanding of how factors like coin preservation affect algorithm performance. Existing research often uses datasets that are not representative of real-world conditions, leading to potentially misleading results. This paper aims to address this by providing a structured overview of the field's challenges and introducing a carefully curated dataset that explicitly considers coin condition (grading).

Methods

The paper's primary contribution is the creation and description of a new dataset. The authors meticulously curated this dataset by:

  • Focusing on Denarii: They chose denarii due to their prevalence, relatively stable material (silver), and wide historical span, making them ideal for studying variations across grades and issues.
  • Stratifying by Grade: The dataset includes multiple examples of each coin issue across three distinct preservation grades: Fine (F), Very Fine (VF), and Extremely Fine (XF). This is crucial because coin condition significantly impacts visual features and thus algorithm performance.
  • Ensuring Diversity: The dataset comprises 100 different coin issues from 44 issuing authorities, spanning approximately 300 years (27 BC to 244 AD), totaling over 650 specimens.
  • Expert Curation: The grading was performed by experts, and data was sourced from reputable dealers and auction houses, with accompanying labels, to ensure accuracy and minimize bias.

Beyond the dataset, the paper also provides a detailed review of existing challenges in the field, including:

  • Terminology: Clarifying the distinction between a "coin" (physical object) and an "issue" (abstract concept).
  • Grading: Explaining the quasi-objective system used to describe coin condition.
  • Practical Applications: Outlining key tasks like specimen matching, denomination categorization, issuing authority recognition, issue matching, and legend readout.
  • Technical Challenges: Discussing issues like illumination variability, clutter, scale, viewpoint, and the sheer number of coin issues.

Datasets

The paper introduces a novel dataset:

  • Roman Imperial Denarii Dataset: This dataset consists of 100 different coin issues, with multiple specimens (over 650 in total) for each issue across three grades: Fine (F), Very Fine (VF), and Extremely Fine (XF). It covers 44 issuing authorities and spans from 27 BC to 244 AD. This dataset is made freely available for research purposes.

The paper also references existing datasets used in prior research, often highlighting their limitations (e.g., lack of condition stratification).

Key Insights and Ideas

  • Coin Condition is Paramount: The paper strongly emphasizes that coin preservation grade is a critical, yet often overlooked, factor in the performance of computer vision algorithms for numismatics. Algorithms that perform well on pristine coins may fail dramatically on worn ones, and vice-versa.
  • Lack of Standardized Evaluation Hinders Progress: The field suffers from a "veering direction of effort" due to the absence of a common, robust framework for evaluating algorithms. This makes it difficult to compare results across studies and build upon previous work.
  • The "Issue" vs. "Specimen" Distinction is Key: Understanding that multiple physical coins (specimens) can belong to the same abstract issue is fundamental to numismatic analysis and poses a significant challenge for automated systems.
  • The Sheer Scale of Numismatic Data is Daunting: With tens of thousands of distinct coin issues, creating comprehensive annotated datasets for supervised learning is practically impossible, necessitating alternative approaches like semantic analysis.
  • Implementation Details Matter: The paper highlights a crucial caveat: even seemingly identical algorithms (like SIFT) can produce vastly different results depending on their specific implementation and parameters, underscoring the need for meticulous reporting in research.

Key Results

The paper's primary "result" is the creation and release of the curated dataset. While the paper doesn't present extensive experimental results using this dataset (as it's intended for future evaluation), it uses it to illustrate the challenges. The authors demonstrate that:

  • The dataset comprises 100 distinct coin issues, 44 issuing authorities, and over 650 specimens across F, VF, and XF grades.
  • They illustrate the visual differences in coin appearance due to grade using figures, showing how wear affects detail.
  • They discuss the limitations of previous approaches, such as local feature-based methods (e.g., SIFT), which struggle with the variability introduced by coin condition and other real-world factors.

Limitations and Future Work

  • Limited Grades: The dataset focuses on Fine, Very Fine, and Extremely Fine grades. Lower grades (Fair, Good) were excluded due to their rarity and the difficulty in extracting meaningful information, but their exclusion might limit evaluation for certain applications.
  • Focus on Denarii: While denarii were chosen for good reasons, future work could expand to other denominations to capture a broader range of material properties and wear patterns.
  • Need for Broader Adoption: The authors explicitly encourage the research community to adopt their dataset as a standard benchmark to foster more comparable and reproducible research.
  • Further Algorithmic Development: The paper sets the stage for future research by identifying key challenges. This includes developing algorithms that are robust to variations in coin condition, exploring semantic analysis for issue recognition, and investigating novel approaches for forgery detection and die matching.
  • Caveat Scolasticus: The authors strongly advocate for researchers to meticulously report implementation details and parameters of algorithms used, a crucial step for reproducibility and understanding of results.

Summary updated

Abstract

Automatic ancient Roman coin analysis only recently emerged as a topic of computer science research. Nevertheless, owing to its ever-increasing popularity, the field is already reaching a certain degree of maturity, as witnessed by a substantial publication output in the last decade. At the same time, it is becoming evident that research progress is being limited by a somewhat veering direction of effort and the lack of a coherent framework which facilitates the acquisition and dissemination of robust, repeatable, and rigorous evidence. Thus, in the present article, we seek to address several associated challenges. To start with, (i) we provide a first overview and discussion of different challenges in the field, some of which have been scarcely investigated to date, and others which have hitherto been unrecognized and unaddressed. Secondly, (ii) we introduce the first data set, carefully curated and collected for the purpose of facilitating methodological evaluation of algorithms and, specifically, the effects of coin preservation grades on the performance of automatic methods. Indeed, until now, only one published work at all recognized the need for this kind of analysis, which, to any numismatist, would be a trivially obvious fact. We also discuss a wide range of considerations which had to be taken into account in collecting this corpus, explain our decisions, and describe its content in detail. Briefly, the data set comprises 100 different coin issues, all with multiple examples in Fine, Very Fine, and Extremely Fine conditions, giving a total of over 650 different specimens. These correspond to 44 issuing authorities and span the time period of approximately 300 years (from 27 BC until 244 AD). In summary, the present article should be an invaluable resource to researchers in the field, and we encourage the community to adopt the collected corpus, freely available for research purposes, as a standard evaluation benchmark.
BibTeX
@inproceedings{arandjelovic2020images,
  author = {Ognjen Arandjelović and Marios Zachariou},
  title = {{Images of Roman Imperial Denarii: A Curated Data Set for the Evaluation of Computer Vision Algorithms Applied to Ancient Numismatics, and an Overview of Challenges in the Field}},
  booktitle = {Sci},
  year = {2020},
  doi = {10.3390/sci2030065},
}

Loading related content...

Figure