SPMK AND GRABCUT BASED TARGET EXTRACTION FROM HIGH RESOLUTION REMOTE SENSING IMAGES

Weihong Cui, Guofeng Wang, Chenyi Feng, Yiwei Zheng, Jonathan Li, Yi Zhang2016 · DOI

Abstract

Target detection and extraction from high-resolution remote sensing images is a fundamental application in urban planning and disaster management. In this paper, we propose a semi-automatic target extraction method that combines machine learning-based detection with interactive segmentation to improve image interpretation efficiency. We introduce the dense transform color scale-invariant feature transform (TC-SIFT) descriptor and a combined histogram of oriented gradients (HOG) & HSV descriptor to characterize the spatial structure and color information of targets. Using k-means clustering, we generate a bag of visual words and employ a three-level spatial pyramid (SP) to represent target patches. An ensemble of SVM classifiers is constructed using TC-SIFT-SP and multi-scale HOG & HSV features to detect targets, specifically buildings. Experimental results demonstrate a detection accuracy exceeding 90%. Following detection, we utilize the resulting rectangle regions as foreground candidates for a GrabCut-based, boundary-regularized semi-automatic segmentation algorithm to extract accurate target boundaries.

Introduction

Extracting specific objects like buildings from Very High Resolution (VHR) remote sensing imagery is critical for modern geospatial analysis. However, this task often presents a "Goldilocks" problem: manual interpretation is highly accurate but labor-intensive, while fully automatic methods frequently struggle with spectral similarities between different land covers—such as grey rooftops and asphalt roads—or fail to identify the optimal scale for segmentation. This research proposes a semi-automatic workflow designed to bridge this gap, leveraging computational power for initial detection and boundary suggestion while allowing human operators to refine results rapidly.

Methodology

Feature Representation and Detection

The proposed workflow begins with a robust feature extraction phase designed to capture both local texture and global spatial arrangements.

  1. TC-SIFT-SP: Standard SIFT descriptors often ignore color information. We utilize "Transform Color SIFT" (TC-SIFT) to incorporate color awareness, making the model more robust to illumination changes and shadows. To solve the spatial limitations of the standard Bag of Visual Words (BoVW) model, we apply a Spatial Pyramid (SP) approach. This divides image patches into increasingly fine grids (1x1, 2x2, and 4x4) to encode the spatial layout of features.
Spatial Pyramid representation of a building target patch showing visual word distribution across a 4x4 grid.
Spatial Pyramid representation of a non-target field patch, illustrating the distinct feature distribution compared to buildings.
  1. MS-HOG & HSV: In addition to TC-SIFT, we employ Multi-Scale Histograms of Oriented Gradients (HOG) combined with HSV color histograms. While TC-SIFT excels at local texture, MS-HOG captures the global structural integrity of the target.
  2. Ensemble SVM Classification: Two separate Support Vector Machine (SVM) classifiers are trained on these feature sets. The final detection is determined by the intersection of both results. This conservative approach prioritizes high precision, effectively reducing false positives by ensuring both structural and color-spatial criteria are met.

Interactive Segmentation and Boundary Regularization

The bounding boxes generated during the detection phase serve as initial seeds for the GrabCut algorithm. GrabCut is an iterative graph-cut method that estimates foreground and background distributions. To move from "blobby" pixel-based segments to map-ready vector data, we apply the Douglas-Peucker algorithm for contour simplification. Finally, geometric constraints—specifically parallelism and orthogonality—are enforced to produce regularized polygons that match the architectural reality of building footprints.

Results and Discussion

The system was tested on 0.2m resolution aerial imagery from the Huizhou district in Guangdong, China. The training set included 300 positive building samples and 500 negative samples (forest, water, roads).

  • Detection Performance: The ensemble method achieved a detection accuracy for buildings of over 90%. Notably, in specific test scenarios, the intersection method reached a precision of 0.97. While this high precision resulted in a lower recall (0.62), it ensures that the extracted targets are highly reliable, minimizing the need for human correction of false positives.
  • Feature Optimization: Through empirical testing, a vocabulary of 500 visual words was identified as the optimal balance between classification accuracy and computational efficiency for the TC-SIFT-SPMK method.
  • Qualitative Refinement: The boundary regularization successfully transformed irregular GrabCut outputs into clean polygons, demonstrating the effectiveness of combining statistical segmentation with geometric priors.

Conclusion

This research demonstrates that a combined SPMK and GrabCut approach provides an efficient, semi-automatic solution for building extraction from high-resolution UAV imagery. While highly effective, future work will focus on improving boundary regularization in dense urban environments where buildings are closely clustered. Additionally, we aim to implement "self-organized" scale fitting to allow the system to adapt automatically to varying image resolutions and sensor types.

Summary updated

Abstract

Target detection and extraction from high resolution remote sensing images is a basic and wide needed application. In this paper, to improve the efficiency of image interpretation, we propose a detection and segmentation combined method to realize semi-automatic target extraction. We introduce the dense transform color scale invariant feature transform (TC-SIFT) descriptor and the histogram of oriented gradients (HOG) & HSV descriptor to characterize the spatial structure and color information of the targets. With the k-means cluster method, we get the bag of visual words, and then, we adopt three levels’ spatial pyramid (SP) to represent the target patch. After gathering lots of different kinds of target image patches from many high resolution UAV images, and using the TC-SIFT-SP and the multi-scale HOG & HSV feature, we constructed the SVM classifier to detect the target. In this paper, we take buildings as the targets. Experiment results show that the target detection accuracy of buildings can reach to above 90%. Based on the detection results which are a series of rectangle regions of the targets. We select the rectangle regions as candidates for foreground and adopt the GrabCut based and boundary regularized semi-auto interactive segmentation algorithm to get the accurate boundary of the target. Experiment results show its accuracy and efficiency. It can be an effective way for some special targets extraction.
BibTeX
@inproceedings{cui2016spmk,
  author = {Weihong Cui and Guofeng Wang and Chenyi Feng and Yiwei Zheng and Jonathan Li and Yi Zhang},
  title = {{SPMK AND GRABCUT BASED TARGET EXTRACTION FROM HIGH RESOLUTION REMOTE SENSING IMAGES}},
  booktitle = {˜The},
  year = {2016},
  doi = {10.5194/isprs-archives-xli-b7-195-2016},
}
Figure