Landslide Susceptibility Mapping by Fusing Convolutional Neural Networks and Vision Transformer

Shuai Bao, Jiping Liu, Liang Wang, Milan Konecný, Xianghong Che, Shenghua Xu, Pengpeng Li2023 · DOI

TL;DR

This paper tackles the challenge of accurately predicting landslide susceptibility by fusing two powerful deep learning architectures: Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). By combining their strengths, the authors developed novel models (BoTNet and ConViT) that outperform individual models, offering a more robust tool for landslide hazard assessment and disaster prevention.

What Problem Does This Paper Address?

Landslide susceptibility mapping (LSM) is crucial for regional planning and disaster management. Traditional methods often struggle with capturing the complex spatial relationships and features inherent in landslide-prone areas. While Convolutional Neural Networks (CNNs) excel at extracting local features, they can miss broader spatial context. Vision Transformers (ViTs), on the other hand, are adept at capturing global dependencies but typically require vast amounts of training data, which can be a limitation in specialized domains like LSM. This paper aims to bridge this gap by creating hybrid models that leverage the best of both worlds, addressing the limitations of individual CNN and ViT approaches for LSM.

Methods

The core of this research lies in fusing CNN and ViT architectures. Specifically, the authors explored two fusion strategies:

  • BoTNet (Bottleneck Transformer Network): This approach integrates Multi-Head Self-Attention (MHSA) modules from ViT into a CNN (ResNet50) backbone. The MHSA modules replace some of the convolutional layers in the later stages of the ResNet architecture. This aims to enhance the model's ability to capture long-range dependencies while retaining the local feature extraction capabilities of CNNs.
  • ConViT (Convolutional Vision Transformer Network): This model builds upon the ViT architecture by incorporating "soft convolutional inductive biases." This is achieved through a novel "Gated Positional Self-Attention" (GPSA) mechanism, which allows the model to learn how to balance content-based self-attention with convolutional operations. This aims to improve ViT's performance with smaller datasets and better leverage local information.

The study also involved a thorough analysis of landslide conditioning factors using GeoDetector to assess their interactions and identify potential collinearity, ensuring the selected factors were suitable for modeling.

Datasets

The study utilized a landslide inventory dataset for Pingwu County, Sichuan Province, China. This dataset was augmented with 11 landslide conditioning factors derived from various sources:

  • Topographic factors: Elevation, slope, aspect, plan curvature, and profile curvature.
  • Geological factors: Distance to fault and lithology.
  • Hydrological factors: Distance to river and rainfall.
  • Ecological factors: Landcover and Normalized Difference Vegetation Index (NDVI).

The landslide conditioning factor data had a spatial resolution of 30m, except for topographic data which was 10m. The landslide dataset was split into training (70%) and testing (30%) sets.

Key Insights and Ideas

  • Synergy of CNNs and ViTs: The fusion of CNNs and ViTs is not just an additive process; it creates a synergistic effect that leads to improved landslide susceptibility mapping performance.
  • Addressing Data Scarcity: ConViT's approach of incorporating convolutional inductive biases into ViT shows promise in mitigating the high data requirements typically associated with pure Transformer models.
  • Local vs. Global Feature Importance: The study suggests that ViT models might be more sensitive to the "shape" of landslide units, while CNNs (like ResNet) focus more on "texture." The fused models aim to capture both.
  • Smoother Loss Landscapes: The inclusion of attention mechanisms (MHSA in BoTNet, GPSA in ConViT) leads to smoother loss landscapes, which can make model optimization more challenging but also potentially lead to better generalization.

Key Results

  • Improved Accuracy: Both BoTNet and ConViT achieved the highest accuracy on the test dataset at 87.78%, outperforming individual ResNet (86.00%) and ViT (86.67%) models.
  • Enhanced F1-Score: ConViT achieved the highest F1-score at 87.64%, an improvement over the other models.
  • Robustness: The models showed minimal differences in accuracy and F1-score between training and testing datasets, indicating a lack of overfitting.
  • Spatial Prediction: The landslide susceptibility maps generated by all four models showed a high probability of landslides in the southeastern part of Pingwu County, consistent with the presence of high elevation relief, proximity to rivers, and fault lines. The predictions were also consistent with actual landslide occurrences.

Limitations and Future Work

  • Specificity of Study Area: The authors acknowledge that the chosen study area (Pingwu County) has unique topographical characteristics, and future experiments should be conducted across diverse geographical regions.
  • Data Resolution Mismatch: The study noted a difference in spatial resolution between some raw data (30m) and topographic data (10m), which could potentially impact prediction accuracy.
  • Deeper Fusion: The current fusion approaches are considered "shallow." The authors suggest exploring "deeper" fusion strategies, such as using CNNs in the initial layers and ViTs in later layers, or vice-versa, to further exploit the advantages of both architectures. This could lead to even better suppression of high-frequency noise and a more comprehensive understanding of landslide units.

Summary updated

BibTeX
@misc{bao2023landslide,
  author = {Shuai Bao and Jiping Liu and Liang Wang and Milan Konecný and Xianghong Che and Shenghua Xu and Pengpeng Li},
  title = {{Landslide Susceptibility Mapping by Fusing Convolutional Neural Networks and Vision Transformer}},
  year = {2023},
  doi = {10.3390/s23010088},
}
Figure