Abstract
This research introduces a creative framework for visualizing the "perception gap" between biological and artificial vision systems. By utilizing Voronoi-tessellated diptychs, the authors demonstrate how task-oriented AI focuses on discriminative features like silhouettes and edges, while human attention is captured by semantic context, faces, and artistic composition.
Introduction: Visualizing the Perception Gap
As deep learning models become increasingly integrated into creative and analytical fields, understanding the "black box" of machine perception is essential. While mathematical tools like saliency maps provide a view of what a neural network "attends" to, they are often too abstract for direct comparison with human experience. This paper addresses the lack of qualitative, intuitive comparisons between task-oriented AI (trained to categorize) and the contemplative human mind (driven by evolution and semantics) by turning these disparate modes of seeing into a shared visual language: artistic diptychs.
Methodology: Picture Tessellation
The authors developed a pipeline to transform raw images into "perception maps" using a technique called picture tessellation:
- Saliency Mapping: Machine saliency is calculated using the gradient of the network’s output with respect to the input image. Human saliency is collected via gaze focus data from eye trackers and existing fixation datasets.
- Point Sampling: Saliency maps are treated as probability density functions. High-interest areas receive a high density of sampled points, while "boring" areas receive few.
- Voronoi Tessellation: These points act as seeds for a Voronoi diagram. This creates a mosaic where high-attention areas are rendered with tiny, sharp tiles (high detail) and low-attention areas are rendered with large, blurry tiles (low detail).
- Coloring: Each tile is colored based on the RGB mean of the pixels it covers in the original image.
Comparative Insights: Human vs. Machine
The resulting diptychs reveal fundamental differences in how biological and artificial systems prioritize information. A primary example is the "flower" arrangement, where the two systems diverge significantly in their focus.
In contrast, the machine's perception is driven by the need for classification. It often ignores the internal substance of an object in favor of the discriminative edges that define its category.
Key Findings
- Silhouette vs. Substance: AI is hyper-focused on outlines for identification, often ignoring the internal colors and textures that humans find captivating.
- Semantic Blindness: AI trained for classification frequently misses "narrative" details. For instance, in street scenes, the human eye is drawn to people, while the AI may ignore them if they do not contribute to its specific classification score (e.g., finding buildings or cars).
- Compositional Awareness: Humans naturally follow vanishing points and perspective lines. AI perception is largely indifferent to these structural aesthetics unless they correlate with a learned feature.
- The Adversarial Hint: In some cases, AI saliency is uniform across the frame, suggesting the classifier is sensitive to changes anywhere in the image—a property linked to the vulnerability of neural networks to adversarial attacks.
Artistic Utility and Future Directions
Beyond analysis, these perception maps serve as "computational catalysts." The authors demonstrated that by projecting these distorted maps onto a canvas, an artist can use the machine's unique "view" as a starting point for a new, hybrid painting.
While effective, the authors acknowledge that "saliency" remains a subjective and task-dependent metric. Future work could explore other rendering styles, such as pointillism or impressionism, to further refine how we visualize the divide between human and machine vision, ultimately working toward AI models that perceive the world in a more human-centric, semantic way.
Summary updated