Abstract
Human pose estimation (HPE) is a central part of understanding the visual narration and body movements of characters depicted in artwork collections, such as Greek vase paintings. Unfortunately, existing HPE methods do not generalise well across domains resulting in poorly recognised poses. Therefore, we propose a two-step approach: (1) adapting a dataset of natural images of known person and pose annotations to the style of Greek vase paintings by means of image style-transfer. We introduce a perceptually-grounded style transfer training to enforce perceptual consistency. Then, we fine-tune the base model with this newly created dataset. We show that using style-transfer learning significantly improves the SOTA performance on unlabelled data by more than 6% mean average precision (mAP) as well as mean average recall (mAR). (2) To improve the already strong results further, we created a small dataset (ClassArch) consisting of ancient Greek vase paintings from the 6–5th century BCE with person and pose annotations. We show that fine-tuning on this data with a style-transferred model improves the performance further. In a thorough ablation study, we give a targeted analysis of the influence of style intensities, revealing that the model learns generic domain styles. Additionally, we provide a pose-based image retrieval to demonstrate the effectiveness of our method.
The Challenge of Artistic Domains
State-of-the-art Human Pose Estimation (HPE) models are typically trained on massive datasets of natural photographs, such as COCO. However, when these models are applied to non-photorealistic domains like ancient artwork, they perform poorly. In Greek vase paintings, figures are often flat, highly stylized, and depicted with anatomical conventions that differ significantly from real-world physics.
For archaeologists and art historians, automatically detecting and analyzing these poses is crucial for understanding visual narratives—like "divine pursuit" or "wrestling" scenes—which are often defined by specific body postures. However, manually annotating enough artwork to train a deep learning model from scratch is prohibitively expensive and requires significant domain expertise.
Methodology: Bridging Photos and Vases
We propose a two-step transfer learning approach to bridge the gap between photos and vases:
- Synthetic Domain Adaptation: To adapt natural images to the unique aesthetic of Greek vases, we utilize Adaptive Instance Normalization (AdaIN) style transfer. We "repaint" the COCO dataset using the textures and color palettes of Greek vases, creating a synthetic dataset called Styled-COCO-Persons (SCP). Because the underlying geometry is preserved, the original ground-truth pose annotations remain valid.
- Perceptually-Grounded Training: During the style transfer process, we introduce a perceptual loss as a regularizer. This ensures that while the visual style changes, the human structure (the content) remains consistent, preventing the model from learning visual artifacts that could degrade pose detection.
- Fine-tuning: We then fine-tune these "styled" models on a small, high-quality dataset of real vase paintings using a Faster R-CNN for person detection and HRNet-W32 for keypoint localization.
The ClassArch Dataset
To evaluate our approach, we introduced ClassArch, a new benchmark consisting of 1,513 images from 6th–5th century BCE Greek vases. These images were manually annotated with person bounding boxes and pose keypoints, serving as the "gold standard" for our experiments. We found that training on styled data makes the model a "faster learner"; a model pre-trained on SCP and then fine-tuned on only 50% of the ClassArch dataset performed as well as a standard model trained on 100% of the data.
Performance and Results
Our final "Styled-Tuned" model reached 67.2 mAP on the ClassArch dataset, outperforming models tuned directly from standard photo-based weights (65.6 mAP). Qualitatively, the improvement is stark, as the model correctly identifies keypoints that baseline methods like OpenPose miss entirely due to the stylized rendering of the figures.
Beyond detection, we demonstrated the effectiveness of our method through a pose-based image retrieval system. By measuring the similarity between detected postures, we successfully retrieved similar narrative scenes (e.g., finding all "fleeing" characters) across a large collection of unlabelled vases, providing a powerful tool for digital humanities research.
Limitations and Future Work
While pose estimation has seen significant gains, person detection remains challenging when figures are heavily intertwined, such as in Greek wrestling (Agonal) scenes. Future work will explore incorporating geometric priors or semantic segmentation to better separate characters from decorative backgrounds and improve detection in complex compositions.
Summary updated