Research on improved SegFormer with multi-module fusion for landslide remote sensing image recognition

Minghua Luo, Canming Yuan, Rui Ma, Bibo Dai, Jinxin Huang, Xin Pan, Xu Wu, Zhixin ZhangResearch Square 2026 · DOI

TL;DR

Mapping landslides from satellite imagery is notoriously difficult because they often look identical to roads or bare soil. This paper introduces an upgraded SegFormer architecture that integrates squeeze-and-excitation modules and a dedicated edge-fusion branch to achieve a 0.932 F1-score, significantly outperforming traditional CNN-based methods like U-Net.

What Problem Does This Paper Address?

Rapid landslide identification is critical for disaster response, yet traditional manual interpretation is too slow for large-scale events. While deep learning has automated much of this, standard Convolutional Neural Networks (CNNs) suffer from a "receptive field bottleneck"—they are great at seeing local textures but struggle to understand the broader geographical context.

Furthermore, landslides present two specific technical hurdles:

  1. Spectral Similarity: Landslides, mountain roads, and bare rock often share the same color and texture.
  2. Boundary Precision: CNNs often produce "fuzzy" or jagged edges, which leads to inaccurate calculations of the affected area and volume.

Methods

The authors build upon SegFormer, a transformer-based vision model, and introduce three primary structural innovations:

  • Squeeze-and-Excitation (SE) Modules: These are embedded into the encoder to adaptively weight feature channels. This helps the model "ignore" background noise and focus on the specific spectral signatures of landslide regions.
  • Auxiliary Edge-Fusion Branch: Instead of relying on the main network to figure out boundaries, the authors added a dedicated branch that extracts edge cues from shallow layers. This branch is supervised by a gradient of the ground-truth masks.
  • Adaptive Feature Gating: A mathematical gating mechanism (using a control coefficient $\alpha$) fuses the edge map with the main probability map. This sharpens the final prediction, effectively "cleaning up" the borders of the landslide.
  • Hybrid Loss & Post-Processing: The model uses Focal Loss to handle the massive class imbalance (where landslides occupy only a tiny fraction of the pixels). After inference, the results undergo "black region suppression" to remove shadows and a Dense Conditional Random Field (DenseCRF) pass to ensure the final mask aligns perfectly with the original image's contours.

Datasets

The study utilized a dedicated landslide remote sensing dataset:

  • Quantity: 100 high-resolution optical satellite images.
  • Split: 70 images for training, 20 for validation/tuning, and 10 for independent testing.
  • Annotation: Every image was manually annotated with pixel-level binary masks by experienced interpreters to provide a high-fidelity ground truth.

Key Insights and Ideas

  • Global Context is King: The authors argue that a landslide isn't just a "brown patch"; its identity is defined by its relationship to the surrounding slope and vegetation. Transformers, which use self-attention to model long-range dependencies, are inherently better at this than CNNs.
  • The "Road" Problem: One of the most interesting findings was the model's ability to distinguish between linear man-made structures (roads) and the "fan-shaped" or irregular geometry of landslides. The auxiliary edge branch was pivotal here.
  • Shadow Mitigation: Deep learning models often mistake mountain shadows for landslides due to low contrast. The inclusion of a specific "black region suppression" step in the post-processing pipeline shows a pragmatic approach to real-world sensor limitations.

Key Results

The proposed method consistently outperformed both the classical U-Net and traditional Otsu thresholding across all metrics:

Metric Proposed Method U-Net Otsu + Filtering
Accuracy 0.976 0.960 0.954
Precision 0.918 0.894 0.821
Recall 0.947 0.869 0.940
F1-Score 0.932 0.873 0.868

Qualitatively, the model produced much cleaner masks with fewer "burrs" or fragmented patches, demonstrating superior robustness in complex terrains.

Limitations and Future Work

While the results are impressive, the authors note a few areas for growth:

  • Data Scarcity: A dataset of 100 images is relatively small for deep learning. Future work aims to expand this to more diverse geographic regions.
  • Dimensionality: The current model relies on 2D imagery. The authors suggest that integrating 3D elevation data (DEM) could further help filter out non-slope artifacts like roads.
  • Multi-Tasking: The next step is a multi-task framework that doesn't just identify the landslide but simultaneously estimates its volume and assesses future susceptibility.

Summary updated

Abstract

<title>Abstract</title> Landslides pose severe threats to life and property, necessitating rapid and accurate identification for effective hazard assessment and emergency response. This study proposes an improved SegFormer model designed for precise landslide extraction from high-resolution remote sensing imagery. To address the limitations of existing segmentation methods in handling complex backgrounds and irregular boundaries, the proposed framework integrates several structural enhancements. These include a squeeze-and-excitation module to suppress background noise, an auxiliary edge-fusion branch to capture explicit boundary details, and an adaptive feature gating mechanism to refine feature representation. The model is trained using focal loss to mitigate class imbalance and employs a three-stage recognition process, culminating in post-processing via dense conditional random fields for boundary refinement. Experimental results on a dataset of 100 high-resolution satellite images demonstrate that this approach significantly outperforms the classical U-Net architecture and traditional thresholding techniques. The model achieved an accuracy of 0.976, a precision of 0.918, a recall of 0.947, and an F1-score of 0.932. These findings confirm that the proposed method offers exceptional accuracy and robustness, providing an effective automated tool for large-scale landslide detection in complex terrain.
BibTeX
@inproceedings{luo2026research,
  author = {Minghua Luo and Canming Yuan and Rui Ma and Bibo Dai and Jinxin Huang and Xin Pan and Xu Wu and Zhixin Zhang},
  title = {{Research on improved SegFormer with multi-module fusion for landslide remote sensing image recognition}},
  booktitle = {Research Square},
  year = {2026},
  doi = {10.21203/rs.3.rs-8711252/v1},
}
Figure