Abstract
Target detection and extraction from high-resolution remote sensing images is a fundamental application in urban planning and disaster management. In this paper, we propose a semi-automatic target extraction method that combines machine learning-based detection with interactive segmentation to improve image interpretation efficiency. We introduce the dense transform color scale-invariant feature transform (TC-SIFT) descriptor and a combined histogram of oriented gradients (HOG) & HSV descriptor to characterize the spatial structure and color information of targets. Using k-means clustering, we generate a bag of visual words and employ a three-level spatial pyramid (SP) to represent target patches. An ensemble of SVM classifiers is constructed using TC-SIFT-SP and multi-scale HOG & HSV features to detect targets, specifically buildings. Experimental results demonstrate a detection accuracy exceeding 90%. Following detection, we utilize the resulting rectangle regions as foreground candidates for a GrabCut-based, boundary-regularized semi-automatic segmentation algorithm to extract accurate target boundaries.
Introduction
Extracting specific objects like buildings from Very High Resolution (VHR) remote sensing imagery is critical for modern geospatial analysis. However, this task often presents a "Goldilocks" problem: manual interpretation is highly accurate but labor-intensive, while fully automatic methods frequently struggle with spectral similarities between different land covers—such as grey rooftops and asphalt roads—or fail to identify the optimal scale for segmentation. This research proposes a semi-automatic workflow designed to bridge this gap, leveraging computational power for initial detection and boundary suggestion while allowing human operators to refine results rapidly.
Methodology
Feature Representation and Detection
The proposed workflow begins with a robust feature extraction phase designed to capture both local texture and global spatial arrangements.
- TC-SIFT-SP: Standard SIFT descriptors often ignore color information. We utilize "Transform Color SIFT" (TC-SIFT) to incorporate color awareness, making the model more robust to illumination changes and shadows. To solve the spatial limitations of the standard Bag of Visual Words (BoVW) model, we apply a Spatial Pyramid (SP) approach. This divides image patches into increasingly fine grids (1x1, 2x2, and 4x4) to encode the spatial layout of features.
- MS-HOG & HSV: In addition to TC-SIFT, we employ Multi-Scale Histograms of Oriented Gradients (HOG) combined with HSV color histograms. While TC-SIFT excels at local texture, MS-HOG captures the global structural integrity of the target.
- Ensemble SVM Classification: Two separate Support Vector Machine (SVM) classifiers are trained on these feature sets. The final detection is determined by the intersection of both results. This conservative approach prioritizes high precision, effectively reducing false positives by ensuring both structural and color-spatial criteria are met.
Interactive Segmentation and Boundary Regularization
The bounding boxes generated during the detection phase serve as initial seeds for the GrabCut algorithm. GrabCut is an iterative graph-cut method that estimates foreground and background distributions. To move from "blobby" pixel-based segments to map-ready vector data, we apply the Douglas-Peucker algorithm for contour simplification. Finally, geometric constraints—specifically parallelism and orthogonality—are enforced to produce regularized polygons that match the architectural reality of building footprints.
Results and Discussion
The system was tested on 0.2m resolution aerial imagery from the Huizhou district in Guangdong, China. The training set included 300 positive building samples and 500 negative samples (forest, water, roads).
- Detection Performance: The ensemble method achieved a detection accuracy for buildings of over 90%. Notably, in specific test scenarios, the intersection method reached a precision of 0.97. While this high precision resulted in a lower recall (0.62), it ensures that the extracted targets are highly reliable, minimizing the need for human correction of false positives.
- Feature Optimization: Through empirical testing, a vocabulary of 500 visual words was identified as the optimal balance between classification accuracy and computational efficiency for the TC-SIFT-SPMK method.
- Qualitative Refinement: The boundary regularization successfully transformed irregular GrabCut outputs into clean polygons, demonstrating the effectiveness of combining statistical segmentation with geometric priors.
Conclusion
This research demonstrates that a combined SPMK and GrabCut approach provides an efficient, semi-automatic solution for building extraction from high-resolution UAV imagery. While highly effective, future work will focus on improving boundary regularization in dense urban environments where buildings are closely clustered. Additionally, we aim to implement "self-organized" scale fitting to allow the system to adapt automatically to varying image resolutions and sensor types.
Summary updated