Abstract
This study investigates the alignment between human visual attention and Explainable AI (XAI) in the context of urban safety perception. Using eye-tracking data from 127 participants and Street View Imagery (SVI), the research compares human gaze patterns with seven Class Activation Mapping (CAM) techniques. Findings reveal that while humans focus on micro-scale urban features like signboards and stairways, AI models often exhibit broader, more diffuse attention. XGradCAM and EigenCAM emerged as the most human-like models, providing a benchmark for developing more interpretable and trustworthy urban AI systems.
Introduction
Urban planners are increasingly turning to deep learning and Street View Imagery (SVI) to quantify "perceived safety"—a psychological metric that significantly impacts property values, public health, and urban policy. However, these deep learning models often operate as "black boxes." While they can predict safety scores with high statistical accuracy, the underlying features driving these predictions remain opaque.
Traditional computer vision metrics often rely on "Object Ratios"—the percentage of an image covered by specific categories like trees or roads. This approach fails to account for the psychological principle of selective attention: the human tendency to prioritize specific, often small, visual cues over large, irrelevant areas. To address this, this research bridges the gap between human cognition and machine learning by using eye-tracking technology to validate XAI heatmaps, ensuring that AI models are "looking" at the same things humans do when judging safety.
Methodology
The researchers developed a computational framework to compare human gaze with AI "attention" through four primary stages:
- Human Eye-Tracking: 127 volunteers in Helsingborg, Sweden, participated in a pairwise comparison task using 300 street view images. Using Tobii eye-tracking systems, the study captured where participants looked when answering: "Which place looks safer?"
- Semantic Segmentation: To understand what was being looked at, the team employed the Dense Prediction Transformer (DPT) to segment images into 150 object categories based on the ADE20K dataset.
- Novel Attention Metrics: Two new metrics were introduced to quantify attention:
- Mean Object Ratio in Highlighted Regions (MoRH): Measures which objects appear most frequently in the areas humans actually look at.
- Mean Object Hue (MoH): Uses the color gradient of heatmaps to weight the relative importance of specific objects.
- XAI Benchmarking: Seven Class Activation Mapping (CAM) techniques—including GradCAM, ScoreCAM, and EigenCAM—were tested to determine which generated heatmaps most closely aligned with the human "ground truth" established by eye-tracking.
Key Insights: The "Sky Fallacy" and Micro-scale Features
A significant finding was the "Sky Fallacy." In standard image analysis, the sky and road occupy the largest pixel area. However, eye-tracking data revealed that humans almost entirely ignore the sky when assessing safety. Instead, human attention is highly selective, focusing on micro-scale features such as stairways, signboards, flags, and chairs.
These elements often serve as "territorial cues" or indicators of "eyes on the street," aligning with classic urban design theories like Jane Jacobs’ work or Crime Prevention Through Environmental Design (CPTED). Conversely, the study found that vehicles (trucks, vans) were strongly associated with "unsafe" perceptions in the Swedish context, while infrastructure like railings and well-maintained floors boosted safety feelings.
Results: Identifying Human-Like AI
The study identified a clear discrepancy between human and AI attention: AI models tend to produce diffuse attention patterns, while humans exhibit highly focused, selective attention. Despite this, certain models performed significantly better than others in mimicking human gaze:
- XGradCAM achieved the highest Cosine Similarity (0.0161) and the lowest L2 Loss (0.3285) compared to human gaze patterns.
- EigenCAM was found to be the most perceptually aligned with human heatmaps, achieving a Learned Perceptual Image Patch Similarity (LPIPS) score of 0.5478.
These results suggest that XGradCAM and EigenCAM are currently the most reliable tools for interpreting how AI perceives urban environments in a way that mirrors human experience.
Limitations and Future Directions
While the study successfully demonstrated that eye-tracking can serve as a "ground truth" for AI validation, several limitations remain:
- Demographic Bias: 93% of the participants were White, and the study was limited to one city in Sweden. Safety perception is highly subjective and varies across cultures and genders.
- Static vs. Dynamic: Street view images are static. Future work could involve 360-degree videos or Virtual Reality (VR) to capture how movement and sound influence safety.
- Temporal Data: This study used aggregated spatial heatmaps but did not analyze gaze trajectories (the sequence of how people look at an image).
By scaling up these "human-in-the-loop" studies, researchers can refine AI models to be more interpretable, ensuring they are trusted for sensitive urban design and policy decisions.
Summary updated