Abstract
That most deep learning models are purely data driven is both a strength and a weakness. Given sufficient training data, the optimal model for a particular problem can be learned. However, this is usually not the case and so instead the model is either learned from scratch from a limited amount of training data or pre-trained on a different problem and then fine-tuned. Both of these situations are potentially suboptimal and limit the generalizability of the model. Inspired by this, we investigate methods to inform or guide deep learning models for geospatial image analysis to increase their performance when a limited amount of training data is available or when they are applied to scenarios other than which they were trained on. In particular, we exploit the fact that there are certain fundamental rules as to how things are distributed on the surface of the Earth and these rules do not vary substantially between locations. Based on this, we develop a novel feature pooling method for convolutional neural networks using Getis-Ord Gi* analysis from geostatistics. Experimental results show our proposed pooling function has significantly better generalization performance compared to a standard data-driven approach when applied to overhead image segmentation.
Introduction: The Challenge of Cross-Location Generalization
Deep learning models for remote sensing are notoriously "data-hungry" and often overfit the specific characteristics of their training environments. A significant hurdle in this field is cross-location generalization: a model trained to identify urban features in one city (e.g., Potsdam) frequently fails when applied to another (e.g., Vaihingen) due to variations in architecture, vegetation, sun angle, or sensor specifications.
Standard computer vision architectures rely on purely data-driven pooling mechanisms, such as Max Pooling, which treat pixels as independent values. However, these methods ignore the "First Law of Geography," which states that "everything is related to everything else, but near things are more related than distant things." By incorporating this "geospatial common sense" into the network architecture, we can develop models that are more robust to location shifts.
Methods: Getis-Ord $G_i^*$ Pooling
The core innovation of this work is the $\mathcal{G}$-pooling layer, which replaces or augments standard Max Pooling in a Fully Convolutional Network (FCN). This layer leverages the Getis-Ord $G_i^*$ Statistic, a tool from geostatistics used for "hotspot analysis" to determine if a high value in a feature map represents an isolated outlier or a statistically significant spatial cluster.
As illustrated in the logic above, the pooling process differs from standard methods:
- Statistic Calculation: For each sliding window, the $G_i^*$ statistic is calculated for the center pixel.
- Thresholding: If the $G_i^*$ value exceeds a predefined threshold (indicating a significant spatial cluster), the network propagates the value at that location.
- Fallback: If the value is below the threshold, the system defaults to standard Max Pooling.
This approach acts as a form of structural regularization, forcing the model to prioritize spatial clusters during training and preventing it from focusing on isolated, high-frequency noise that may not generalize across different geographic regions.
Key Results and Evaluation
The method was evaluated using the ISPRS 2D Semantic Labeling Challenge datasets for Vaihingen (a village with detached buildings) and Potsdam (a historic city with dense settlement). These locations offer distinct urban morphologies, making them ideal for testing generalization.
Cross-Location Performance
The most significant improvements were observed in cross-location experiments, where the model was trained on one city and tested on the other:
- Potsdam $\to$ Vaihingen: $\mathcal{G}$-pooling improved the Mean Intersection over Union (mIoU) from 35.06% (Max Pooling) to 37.65%.
- Vaihingen $\to$ Potsdam: The improvement was even more pronounced, with mIoU jumping from 15.48% to 23.02%.
Notably, $\mathcal{G}$-pooling performed competitively with AdaptSegNet, an unsupervised domain adaptation method, despite requiring no data from the target location during training.
Qualitative Analysis
Qualitative segmentation results on Potsdam and Vaihingen datasets comparing different pooling methods against the ground truth. Source, p. 8
Visual analysis highlights how $\mathcal{G}$-pooling produces significantly "cleaner" segmentations. It effectively reduces "pepper-and-salt" noise within large objects like buildings and roads, which is a common artifact in standard data-driven pooling methods.
Conclusion and Future Directions
$\mathcal{G}$-pooling demonstrates that incorporating domain-specific geographic priors can significantly enhance the generalization of deep models without the need for massive unlabeled datasets. While the current implementation focuses on spatial clusters, future work could incorporate other geographic rules, such as linear feature priors for roads. Additionally, optimizing the threshold sensitivity and exploring class-specific priors remain important avenues for further research in robust geospatial image analysis.
Summary updated