Use of Tencent Street View Imagery for Visual Perception of Streets

Liang Cheng, Sensen Chu, Wenwen Zong, Shuyi Li, Jie Wu, Manchun LiISPRS Int. J. Geo Inf 2017 · DOI

TL;DR

This paper introduces a novel automated method to quantify the visual perception of urban streets using Tencent Street View imagery. By developing four key indices—Salient Region Saturation (SRS), Visual Entropy (VE), Green View Index (GVI), and Sky-Openness Index (SOI)—the authors provide a more comprehensive and objective way to assess streetscape aesthetics, aiding urban planning and design.

What Problem Does This Paper Address?

Evaluating the visual perception of urban streets is crucial for urban planning and enhancing residents' quality of life. However, traditional methods have been hampered by a lack of detailed, large-scale data and inadequate analytical techniques. While the advent of street view services like Google Street View and Tencent Street View has provided a wealth of street-level imagery, extracting meaningful insights into visual perception from this data has remained a challenge. Existing approaches often rely on a limited number of viewpoints or treat visual perception as an abstract concept, failing to capture its multifaceted nature. This paper aims to bridge this gap by proposing a robust, automated framework to quantify various aspects of visual perception from street view imagery.

Methods

The core of this research lies in an automated technical procedure that transforms raw street view images into quantifiable metrics of visual perception. The process involves several key steps:

  • Panoramic Image Generation: Eight street view images captured at different headings (0°, 45°, 90°, etc.) with a 0° pitch angle are stitched together to create a 360° panoramic view, simulating human visual experience.
  • Image Processing: Techniques such as color space conversion (RGB to Lab* and HSV) and image segmentation are employed.
  • Salient Region Detection: The paper utilizes a saliency detection algorithm (SDSP) to identify regions within the panoramic image that are most likely to attract human attention.
  • Index Calculation: Four novel indices are proposed and calculated:
  • Salient Region Saturation (SRS): Measures the saturation of identified salient regions, reflecting the visual brilliance and vividness of the streetscape.
  • Visual Entropy (VE): Quantifies the amount of information and complexity within the streetscape, reflecting its richness and layeredness.
  • Green View Index (GVI): Calculates the proportion of green areas within the street view, indicating the level of greenery.
  • Sky-Openness Index (SOI): Measures the proportion of the sky visible in the street view, reflecting the sense of openness.

Datasets

The study utilized Tencent Street View imagery for the Jianye District of Nanjing City, China. A total of 1669 sampling points were established at 30-meter intervals, from which panoramic images were generated. For evaluation, 84 panoramic images were selected for a questionnaire-based study involving 100 volunteers.

Key Insights and Ideas

  • Street View as a Rich Data Source: The paper strongly advocates for the utility of street view imagery as a powerful and accessible data source for urban landscape analysis, overcoming previous data limitations.
  • Multifaceted Visual Perception: The proposal of four distinct indices (SRS, VE, GVI, SOI) highlights that visual perception is not a monolithic concept but rather a combination of various attributes like brilliance, richness, greenery, and openness.
  • Automated Assessment is Feasible: The research demonstrates that complex visual perception metrics can be reliably extracted and quantified automatically from street view data, paving the way for large-scale urban analysis.
  • Correlation with Human Perception: A significant finding is the strong correlation between the calculated indices and human perception scores obtained from questionnaires, validating the effectiveness of the proposed metrics.
  • Context Matters: The study implicitly shows how different urban forms (e.g., elevated roads, residential streets, main roads) result in distinct index values, underscoring the importance of context in visual perception.

Key Results

  • The proposed four indices (SRS, VE, GVI, SOI) were shown to effectively characterize different aspects of visual perception: brilliance, richness, greenery, and sky openness, respectively.
  • Strong correlations were found between the calculated indices and human perception scores:
  • SRS: R² = 0.7522
    • VE: R² = 0.8268
    • GVI: R² = 0.8727
    • SOI: R² = 0.8577
  • The study area (Jianye District, Nanjing) generally exhibited high levels of visual entropy, suggesting richly layered streetscapes.
  • Streets with higher GVI (more greenery) and SOI (more sky visibility) generally correlated with higher general landscape scores, though the relationship with GVI and SOI showed a "V" distribution, indicating that other factors can also contribute to high scores.

Limitations and Future Work

  • Transferability: The authors acknowledge that the transferability of these indices to other cities might be affected by factors such as location, time of image capture, sunlight, and local history, culture, and climate. Further research is needed to determine effective ways to combine indices and adjust weights for different contexts.
  • GVI Limitation: The GVI, while effective, cannot differentiate between vegetation and other green elements like signs or buildings. The authors suggest using multi-temporal data to address this in future work.
  • Algorithm Extension: The authors propose extending this automated algorithm to other street view platforms like Google Street View and Baidu Street View, and integrating it with other urban data sources.
  • Combination of Indices: Future work should focus on determining optimal combinations or weightings of the four indices to best suit specific urban planning requirements and visual perception goals.

Summary updated

BibTeX
@article{cheng2017use,
  author = {Liang Cheng and Sensen Chu and Wenwen Zong and Shuyi Li and Jie Wu and Manchun Li},
  title = {{Use of Tencent Street View Imagery for Visual Perception of Streets}},
  journal = {ISPRS Int. J. Geo Inf.},
  year = {2017},
  doi = {10.3390/ijgi6090265},
}
Figure