Abstract
This survey examines the integration of geometric information—from 2D skeletons to 3D implicit models—as a means to bridge the domain gap between photorealistic data and the abstract nature of artistic images. By leveraging geometric features as structural invariants, researchers can improve AI's ability to extract, analyze, and synthesize art, overcoming challenges such as high inter-class variation and limited annotated datasets.
Introduction: The Challenge of Artistic Domain Gap
Artificial Intelligence has achieved remarkable success in photorealistic computer vision. However, artistic images present a unique set of challenges that standard models often fail to address. These include high inter-class variation—where a single concept like "human" can be represented as a sketch, a Cubist painting, or a realistic portrait—and the complex entanglement of style and content.
Standard models trained on real-world photos rely heavily on texture and color. When applied to art, these models often lose structural integrity due to exaggerated geometries and abstract forms. This survey explores how geometric features serve as an invariant bridge, allowing AI to maintain structural consistency across wildly different artistic styles.
Methodology: Extraction, Analysis, and Synthesis
The research landscape is categorized into three primary pillars:
- Extraction: Deriving geometric labels from 2D art. This encompasses object-centric features (bounding boxes, masks), human-centric features (pose skeletons, facial landmarks), and 3D features (meshes, point clouds, and Neural Radiance Fields).
- Analysis: Utilizing these features for discriminative tasks such as style classification, artist identification, and object detection in non-standard scenes.
- Synthesis: Creating or manipulating art. Geometric priors are crucial here to prevent "texture bleeding" and ensure structural consistency in generative models like diffusion networks.
The Art-AI Data Landscape
A significant hurdle in this field is the scarcity of high-quality, annotated data. Key datasets include StyleObject7K and Watercolor2K for object detection, and People-Art for human-centric tasks. Unlike massive photorealistic datasets like ImageNet, artistic datasets are often smaller and require weakly supervised learning techniques to handle less accurate labeling.
Key Insights: Geometry as an Invariant
The core insight of this survey is that while colors and textures vary drastically across art movements, underlying geometry—such as the human skeleton—remains relatively stable. By using geometry as a "proxy," models can generalize across styles. Furthermore, 3D proxies (like NeRFs) allow for novel applications, such as "walking through" a 2D historical painting via view synthesis.
The paper also highlights the need for AI to distinguish between technical error and "intentional distortion," such as the perspective shifts found in Piranesi’s drawings, which are artistic choices rather than failures of representation.
Quantitative Results
The integration of geometric constraints has led to measurable improvements across various tasks:
- Object Detection: Using Class Activation Maps (CAM) as pseudo-ground truth improved detection on IconArt by 14%.
- Pose Estimation: Models pre-trained on stylized content saw a 36.7 mAP improvement.
- Synthesis Quality: Adding geometric constraints resulted in a 10% improvement in structural consistency during topology transfer.
Limitations and Future Directions
Current bottlenecks include a lack of standardized benchmarks and a heavy geographic bias toward Western art history. Future research is expected to focus on automatic data annotation (e.g., using Segment Anything Models) and attention-based cross-correspondences to better align textual descriptions with abstract visual forms.
Summary updated