A Complete Reinforcement-Learning-Based Framework for Urban-Safety Perception

Yaxuan Wang, Zhixin Zeng, Qiushan Li, Yingrui DengISPRS Int. J. Geo Inf 2022 · OpenReview

TL;DR

This paper introduces a novel reinforcement learning (RL) framework for urban safety perception that goes beyond traditional classification or regression models. By framing safety perception as a decision-making problem, their RL agent learns to interpret street-view images, achieving human-like perception abilities and offering improved interpretability.

What Problem Does This Paper Address?

Understanding urban safety perception is vital for urban planning and influencing pedestrian behavior. While deep learning methods, particularly CNNs, have been applied to predict safety scores from street-view images, these methods often rely on large annotated datasets and lack interpretability. They typically produce a single score without explaining why an image is perceived as safe or unsafe. This paper addresses the need for a more complete and interpretable system that can mimic human decision-making processes in assessing urban safety.

Methods

The core of this work is a reinforcement learning (RL) framework designed to predict urban safety perception. The authors model urban safety perception as a Markov Decision Process (MDP). This involves:

  • Feature Extraction: A novel module extracts visual semantic and functional features from street-view images. This includes features like the proportion of sky, greenery, walls, sidewalks, buildings, traffic lights, and traffic signs (visual features), as well as features related to the functional area of the city (e.g., residential, business). They also incorporate safety scores obtained from crowdsourcing.
  • RL Environment: An MDP-based environment is created where an agent interacts with street-view images. The state is defined by the extracted visual and functional features, and the action is to predict whether the image is safe or unsafe.
  • Reward Function: A custom reward function is designed based on expert knowledge. This function assigns rewards or penalties to the agent's actions based on the visual and functional features, aiming to guide the agent towards human-like safety judgments.
  • DRL Algorithm: A Double Deep Q-Network (DDQN) variant, specifically D3QN, is employed to train the RL agent. D3QN is chosen for its ability to handle discrete action spaces and reduce overestimation bias in Q-value estimation.

Datasets

  • Dujiangyan Street Images: The primary dataset consists of street-view images from Dujiangyan City, obtained via the Baidu street-view API. Initially, 11,584 panoramas were collected, which were then processed into individual images.
  • Crowdsourced Safety Scores: A crowdsourcing website was developed to collect pairwise comparisons of images. Users were asked to choose which image in a pair felt safer. These comparisons were then used with the TrueSkill and LambdaRank algorithms to derive normalized safety scores for the images.
  • Expert-Annotated Data: To refine the dataset and incorporate expert knowledge, 15 researchers and professors in relevant fields were involved. They removed low-quality images, selected representative features, and helped define criteria for functional areas. Their input was used to amend safety scores and establish thresholds for safe/unsafe labels. Ultimately, 651 images were used for further analysis after expert rating.

Key Insights and Ideas

  • Safety Perception as Decision-Making: Reframing urban safety perception not just as a classification task but as a sequential decision-making problem is a key conceptual shift. This allows for the application of RL, which is adept at learning complex strategies through interaction.
  • Leveraging Expert Knowledge: The framework effectively integrates domain expertise not only in feature selection but also in designing the reward function, which is crucial for guiding the RL agent towards human-like judgments.
  • Bridging Simulation and Reality: The paper highlights the challenge of applying RL to real-world, complex problems like urban perception. Their work aims to bridge this gap by adapting existing DRL algorithms and developing a tailored reward function for this specific domain.
  • Interpretability through RL: Unlike black-box CNNs, the RL approach, particularly with its defined states, actions, and rewards, offers a more transparent view into how safety judgments are made, allowing for visualization of feature contributions.

Key Results

  • Superior Performance: The RL-based framework (D3QN) achieved the highest Area Under the ROC Curve (AUC) of 0.686 for urban-safety perception prediction, outperforming traditional methods like SVM (0.617), MLP (0.611, 0.540), and a 5-layer CNN (0.550) on their crowdsourced dataset.
  • Visual Perceptual Pattern Recovery: The D3QN method also demonstrated strong performance in recovering visual perceptual patterns, achieving a cosine similarity of 0.369 and a KLD of 0.244, indicating its ability to capture nuanced visual aspects related to safety.
  • Visual Interpretability: The paper presents visualizations (Figure 7) showing how different visual features contribute to the agent's reward, offering insights into the decision-making process and demonstrating the framework's interpretability.

Limitations and Future Work

  • Dataset Size: The authors acknowledge that the small size of their crowdsourced dataset can lead to overfitting in neural network-based methods and may affect model generalization.
  • Noise in Labels: Despite expert correction, noise can still exist in crowdsourced labels, which can impact model performance.
  • Reward Function Fine-tuning: For broader applicability to other cities, the authors suggest that the reward function may need fine-tuning.
  • Future Work: Future research should focus on scaling up the dataset and exploring other embedding measures for the reward function, potentially incorporating more expert knowledge to enhance accuracy and interpretability.

Summary updated

Abstract

Urban-safety perception is crucial for urban planning and pedestrian street preference studies. With the development of deep learning and the availability of high-resolution street images, the use of artificial intelligence methods to deal with urban-safety perception has been considered adequate by many researchers. However, most current methods are based on the feature-extraction capability of convolutional neural networks (CNNs) with large-scale annotated data for training, mainly aimed at providing a regression or classification model. There remains a lack of interpretable and complete evaluation systems for urban-safety perception. To improve the interpretability of evaluation models and achieve human-like safety perception, we proposed a complete decision-making framework based on reinforcement learning (RL). We developed a novel feature-extraction module, a scalable visual computational model based on visual semantic and functional features that could fully exploit the knowledge of domain experts. Furthermore, we designed the RL module—comprising a combination of a Markov decision process (MDP)-based street-view observation environment and an intelligent agent trained using a deep reinforcement-learning (DRL) algorithm—to achieve human-level perception abilities. Experimental results using our crowdsourced dataset showed that the framework achieved satisfactory prediction performance and excellent visual interpretability.
BibTeX
@article{wang2022complete,
  author = {Yaxuan Wang and Zhixin Zeng and Qiushan Li and Yingrui Deng},
  title = {{A Complete Reinforcement-Learning-Based Framework for Urban-Safety Perception}},
  journal = {ISPRS Int. J. Geo Inf.},
  year = {2022},
  url = {https://openreview.net/forum?id=Vr35W1EKaK},
}
Figure