THE LAND COVER CLASSIFICATION USING A FEATURE PYRAMID NETWORKS ARCHITECTURE FROM SATELLITE IMAGERY

Qin Zhang, Yilin Zhang, Peipei Yang, Yanda Meng, Shenglong Zhuo, Zhen Yang2020 · DOI

Abstract

Extracting land cover information from satellite imagery is critical for automated monitoring in remote sensing. Deep convolutional neural networks (DCNNs) are often limited by small annotated datasets. This paper presents FPN-VGG, a fully convolutional architecture combining Feature Pyramid Networks (FPN) and VGG16. By employing transfer learning and a variant Dice loss function, FPN-VGG addresses challenges like multi-scale features and class imbalance. Experimental results on a custom pixel-wise annotated dataset show an overall accuracy of 82.9%, outperforming state-of-the-art models like U-Net and SegNet.

Introduction: The Data Bottleneck in Remote Sensing

High-resolution satellite imagery is abundant, but the availability of high-quality, pixel-wise annotated data remains a significant bottleneck for automated environmental monitoring. Beyond dataset size, two primary technical hurdles persist:

  1. Multi-scale Feature Extraction: Objects such as narrow roads and expansive forests exist at vastly different scales, requiring architectures that can process fine-grained details and high-level semantic context simultaneously.
  2. Class Imbalance: Natural landscapes are rarely uniform. As shown in Figure 3, classes like "Tree" and "Low Vegetation" dominate the pixel count, while critical infrastructure like "Roads" represents a tiny fraction of the data. Standard loss functions often ignore these minority classes to maximize global accuracy.
Pixel distribution across the six land cover classes, highlighting the severe class imbalance.

Methodology: FPN-VGG Architecture

The proposed FPN-VGG architecture utilizes a VGG16 backbone as an encoder to extract hierarchical features. The decoder is based on a Feature Pyramid Network (FPN) that creates a pyramid of features at four distinct levels. These features are up-sampled and concatenated to preserve spatial resolution while incorporating deep semantic information.

Transfer Learning and Optimization

To mitigate the limitations of a small dataset, the VGG16 encoder is initialized with weights pre-trained on the ImageNet dataset. This "Parameter Transfer" (illustrated in Figure 2) allows the model to leverage general visual features (edges, textures) before fine-tuning on remote sensing data.

The FPN-VGG workflow, showing parameter transfer from ImageNet-trained VGG to the satellite imagery task.

Furthermore, the authors introduced a hybrid loss function, $L_{cce\_dice}$, which combines Categorical Cross-Entropy with Dice Loss. While Cross-Entropy handles general classification, the Dice Loss component specifically penalizes poor overlap in small areas, forcing the network to improve its performance on underrepresented classes like roads.

Dataset and Experimental Setup

The study utilized a custom dataset derived from ZY-3 satellite Digital Orthophoto Maps (DOM) covering 3,400 $km^2$ in East Asia. The images have a 5.8m ground resolution and include Red, Green, Blue (RGB), and Near-Infrared (N) bands. Six categories were manually annotated: Background, Low Vegetation, Tree, Building, Road, and Water.

A key finding during experimentation was the superiority of NRG (Near-Infrared, Red, Green) band combinations over standard RGB. The Near-Infrared band significantly improved the distinction between water and vegetation, leading to a 5.4% increase in mean Intersection over Union (mIoU).

Results and Discussion

FPN-VGG was benchmarked against FCN-8s, SegNet, and U-Net. The proposed model achieved the highest performance across all primary metrics:

  • Overall Accuracy (OA): 82.9% (vs. U-Net's 79.2%).
  • Mean F1 Score: 66.0% (vs. U-Net's 56.8%).
  • Road Extraction: The "Road" class, which SegNet failed to identify (0% F1), reached a 51.2% F1 score with FPN-VGG using the hybrid loss function.

Conclusion

The FPN-VGG architecture effectively addresses the core challenges of land cover classification in satellite imagery. By combining multi-scale feature pyramids with transfer learning and a class-balanced loss function, the model achieves superior performance on small, unbalanced datasets. Future work will focus on testing the model's robustness across diverse geographical regions and integrating deeper backbones like ResNet.

Summary updated

Abstract

Extracting land cover information from satellite imagery is of great importance for the task of automated monitoring in various remote sensing applications. Deep convolutional neural networks make this task more feasible, but they are limited by the small dataset of annotated images. In this paper, we present a fully convolutional networks architecture, FPN-VGG, that combines Feature Pyramid Networks and VGG. In order to accomplish the task of land cover classification, we create a land cover dataset of pixel-wise annotated images, and employ a transfer learning step and the variant dice loss function to promote the performance of FPN-VGG. The results indicate that FPN-VGG shows more competence for land cover classification comparing with other state-of-the-art fully convolutional networks. The transfer learning and dice loss function are beneficial to improve the performance of on the small and unbalanced dataset. Our best model on the dataset gets an overall accuracy of 82.9%, an average F1 score of 66.0% and an average IoU of 52.7%.
BibTeX
@inproceedings{zhang2020land,
  author = {Qin Zhang and Yilin Zhang and Peipei Yang and Yanda Meng and Shenglong Zhuo and Zhen Yang},
  title = {{THE LAND COVER CLASSIFICATION USING A FEATURE PYRAMID NETWORKS ARCHITECTURE FROM SATELLITE IMAGERY}},
  booktitle = {˜The},
  year = {2020},
  doi = {10.5194/isprs-archives-xliii-b3-2020-241-2020},
}
Figure