Introduction
Attributing medieval panel paintings to specific artists or workshops is a complex task traditionally rooted in qualitative analysis. However, art historians have long recognized a more objective "fingerprint" within these works: punches. These are small, decorative shapes—such as flowers, stars, or geometric patterns—impressed into the gold leaf using handmade mechanical tools. Because these tools were unique to specific workshops and difficult to replicate, they provide a quantitative basis for attribution.
Despite their value, the manual cataloging of these marks is an immense undertaking. For example, historian Erling Skaug spent over three decades manually documenting these patterns. This research introduces an automated machine learning pipeline designed to detect and classify these tiny features within ultra-high-resolution digital images of 13th and 14th-century art.
Technical Challenges
The primary obstacle in automating punch detection is the extreme scale disparity. Medieval paintings are often captured at ultra-high resolutions, sometimes exceeding 50,000 pixels per side. In contrast, the punches themselves are minute, often measuring less than 100 pixels. Standard object detection models cannot ingest these massive images directly, and the high density of these "needles in a haystack" requires a specialized approach to maintain both accuracy and computational efficiency.
Methodology
The researchers developed a pipeline based on YOLOv10, a state-of-the-art one-stage object detector. To bridge the gap between image size and model constraints, they implemented several specialized strategies:
1. Sliding Window Inference
The system decomposes large paintings into 1088x1088 pixel frames. To prevent punches from being bisected at the frame edges, the authors utilized a 324-pixel overlap between windows. This specific value was determined by the maximum dimensions of the largest punch in the dataset, ensuring every mark is captured in its entirety in at least one frame.
2. Custom NMS with Intersection-over-Minimum (IoM)
Standard Non-Maximal Suppression (NMS) typically relies on Intersection-over-Union (IoU) to eliminate redundant detections. However, the sliding window approach often generates "nested" predictions where one bounding box is contained within another. The authors replaced IoU with Intersection-over-Minimum (IoM). This metric is significantly more effective at collapsing redundant, overlapping predictions of varying sizes into a single, accurate detection.
3. Dataset Balancing
To account for the varying frequency of different punch shapes, the team used undersampling techniques. This prevents the model from developing a bias toward more common patterns, ensuring robust performance across all 27 identified categories.
Dataset and Experimental Setup
The study utilized a specialized dataset of 8 ultra-high-resolution images from the Museo Nazionale di Pisa.
- Annotations: 3,475 individual punchmarks were manually labeled by experts.
- Categories: 27 distinct punch shape classes.
- Evaluation: Seven paintings were used for training and validation, while the eighth was reserved as a completely held-out test set to evaluate the model's performance on unseen data.
Key Results and Insights
The pipeline demonstrated high reliability, particularly in its ability to avoid false attributions:
- Precision and F1-Score: The YOLOv10n (nano) model achieved a 94% precision rate and a 90% F1-score. In the context of art history, high precision is critical; a "hallucinated" punch (False Positive) could lead to an incorrect workshop attribution, whereas a missed punch (False Negative) is a more acceptable loss of data.
- Model Scaling: Surprisingly, the smallest model variant (YOLOv10n) outperformed the larger "l" (large) variant. This suggests that for fine-grained, repetitive patterns, extreme overparameterization may lead to increased false negatives rather than better accuracy.
- Impact of IoM: The custom IoM-based suppression strategy was transformative, boosting the precision of the nano model by nearly 19% compared to standard merging techniques.
Conclusion and Future Work
This research provides art historians with a powerful, quantitative tool for the study of medieval workshops. By combining modern object detection with specialized suppression strategies, the system achieves the high precision necessary for scholarly attribution.
Future iterations of this work will focus on testing the model's resilience against damaged or poorly preserved paintings and integrating the detection pipeline with existing historical databases. This will allow for the automatic cross-referencing of new detections with known artist chronologies, further bridging the gap between computer science and art history.
Summary updated