Abstract
Traditionally, land-cover mapping from remote sensing images is performed by classifying each atomic region in the image in isolation and by enforcing simple smoothing priors via random fields models as two independent steps. In this paper, we propose to model the segmentation problem by a discriminatively trained Conditional Random Field (CRF). To this end, we employ Structured Support Vector Machines (SSVM) to learn the weights of an informative set of appearance descriptors jointly with local class interactions. We propose a principled strategy to learn pairwise potentials encoding local class preferences from sparsely annotated ground truth. We show that this approach outperform standard baselines and more expressive CRF models, improving by 4-6 points the average class accuracy on a challenging dataset involving urban high resolution satellite imagery.
Introduction
Semantic segmentation in remote sensing involves assigning a thematic label (e.g., "road," "building," "water") to every pixel or region in a satellite image. In Very High Resolution (VHR) imagery, this task is particularly challenging due to high spectral variability; for example, different roofing materials can look spectrally similar to road surfaces. Traditional approaches often rely on a two-step process: an initial pixel-wise classification followed by a smoothing filter (like a Potts model) to ensure spatial consistency. However, simple smoothing often fails to capture the complex contextual relationships inherent in urban environments—such as the fact that pools are typically adjacent to buildings, while roads rarely intersect forests. Furthermore, obtaining dense ground-truth annotations for training is labor-intensive, necessitating models that can learn effectively from sparse data.
Methodology
The authors introduce a framework based on Conditional Random Fields (CRF) where parameters are learned jointly using Structured Support Vector Machines (SSVM). The core innovation is the modeling of spatial context through "ring-based" potentials:
- Single Ring Potentials (SRP): Instead of limiting interactions to immediate neighbors, the model considers all pixels within a specific radius $\rho$. It learns weights for every possible pair of classes within this distance.
- Multiple Ring Potentials (MRP): To capture multi-scale interactions, the model employs several concentric rings. This allows it to learn complex spatial rules, such as "repulsion" at short distances and "attraction" at longer ones.
The learning process utilizes a max-margin approach (SSVM) to ensure the energy of the correct labeling is lower than alternatives. For inference, the model uses Graph-Cuts (specifically QPBO with $\alpha$-expansion) to find the optimal label configuration efficiently.
Experimental Results
The model was evaluated on the Zurich dataset, which contains 20 multispectral VHR images (0.61m resolution) with 8 urban classes. Notably, only 47% of the superpixels were annotated, providing a rigorous test for learning from sparse data.
The learned models significantly outperformed standard baselines. The LEARNED SRP 40m model achieved a Mean Average Accuracy (AA) of 78.35%, a substantial improvement over the 72.19% achieved by a standard Random Forest and 73.71% by a traditional contrast-sensitive Potts model. The gains were most pronounced in rare classes; for instance, "Rails" accuracy increased from 2.11% to 21.35%.
Qualitatively, the learned potentials effectively reduced classification noise without the "blobby" over-smoothing characteristic of traditional CRFs. The MRP model revealed interesting geographic regularities, such as the structural layout of cities where buildings and roads are more likely to co-occur at specific distances than buildings and buildings.
Conclusion
By moving beyond simple smoothing to learned class-specific interactions, this research demonstrates that spatial context can be a powerful discriminative feature in urban scene segmentation. While selecting the optimal radius remains a computational challenge, the use of multiple rings offers a promising path toward more automated and accurate land-cover mapping. Future work will focus on improving inference efficiency and scaling the approach to larger, pixel-level datasets.
Summary updated