BISVP: Building Footprint Extraction Via Bidirectional Serialized Vertex Prediction

Mingming Zhang, Ye Du, Zhenghui Hu, Qingjie Liu, Yunhong Wang

Introduction

Extracting accurate building footprints from satellite imagery is a fundamental task in modern cartography, urban planning, and disaster response. Historically, this has been approached as a semantic segmentation problem—essentially a "coloring book" task where models assign a building or non-building label to every pixel. While effective at identifying structures, this approach often results in "jittery" or rounded edges that require significant post-processing to convert into the clean, geometric polygons required for Geographic Information Systems (GIS).

In the research paper "BiSVP: Building Footprint Extraction via Bidirectional Serialized Vertex Prediction," a team from Beihang University proposes a paradigm shift. Rather than refining a raster mask, BiSVP treats footprint extraction as a geometry problem, teaching the model to "draw" buildings by predicting their vertices in a specific sequence. This end-to-end, refinement-free framework produces clean polygons directly from raw imagery.

Methodology

The BiSVP framework is designed to overcome the limitations of traditional vertex-based methods, which often struggle with vertex ordering and complex, non-convex building shapes. The architecture introduces three primary innovations:

1. Bidirectional Serialized Vertex Prediction (A-Bi-BP)

A polygon can be mathematically described by a sequence of vertices starting from any point and moving in either a clockwise or counter-clockwise direction. BiSVP leverages this symmetry using an LSTM-based module that predicts the vertex sequence in both directions simultaneously. By integrating information from these two perspectives, the model achieves a more robust understanding of the building's geometry, effectively regularizing the output and reducing errors in vertex placement.

2. Cross-Scale Feature Fusion (CSFF)

Accurate vertex placement requires a balance of high-level semantic context (to identify the building) and low-level spatial resolution (to pinpoint the corner). The CSFF module utilizes a Transformer-based cross-attention mechanism to aggregate features across different scales. This coarse-to-fine approach ensures that the vertex predictor has access to both the global shape of the structure and the fine-grained details of its boundaries.

3. Gaussian Constrained Attention

Predicting long sequences for complex buildings is computationally challenging and prone to drift. To mitigate this, the authors integrated a Gaussian constrained attention mechanism into the vertex prediction module. This helps the model maintain focus on the most relevant portions of the feature map as it iterates through the sequence of corners, ensuring that each predicted vertex is grounded in the local image evidence.

Experimental Evaluation

The model was rigorously evaluated on three diverse benchmarks representing different geographic and architectural challenges:

  • SpaceNet (Las Vegas): High-resolution imagery focusing on urban environments.
  • 5M-Building: A large-scale dataset with varying resolutions and diverse building types.
  • CNData: A challenging dataset featuring dense, small-scale buildings in both rural and urban Chinese provinces.

Quantitative Results

BiSVP demonstrated significant performance gains over both standard segmentation baselines (such as Mask R-CNN) and specialized polygonal methods (such as PolyMapper). On the SpaceNet dataset, BiSVP achieved a 13.34% improvement in the $F1_{75}$ score, a metric that rewards high-accuracy delineation. Furthermore, the Average Precision at a strict overlap threshold ($AP_{75}$) saw an increase of 15.7% on SpaceNet and 4% on the complex CNData set.

Qualitative Advantages

Beyond numerical metrics, the qualitative superiority of BiSVP is evident in the sharpness of the extracted footprints. While segmentation-based models often produce "noisy" or irregular contours, BiSVP generates geometrically plausible polygons with sharp corners that closely align with human-annotated ground truths. This "refinement-free" nature allows for immediate integration into mapping pipelines without the need for secondary simplification algorithms.

Conclusion and Future Work

BiSVP represents a significant step forward in automated cartography by successfully formulating building extraction as a bidirectional sequence prediction problem. By combining multi-scale feature fusion with bidirectional LSTMs, the model captures the underlying "grammar" of building shapes more effectively than previous methods.

While the current framework is sensitive to extreme occlusions, such as heavy tree cover or deep shadows, it provides a robust foundation for future research. The authors suggest that integrating these bidirectional vertex predictors into modern Transformer-based detectors, such as DETR, could further enhance the speed and accuracy of large-scale urban mapping.

Summary updated

BibTeX
@inproceedings{zhang2023bisvp,
  author = {Mingming Zhang and Ye Du and Zhenghui Hu and Qingjie Liu and Yunhong Wang},
  title = {{BISVP: Building Footprint Extraction Via Bidirectional Serialized Vertex Prediction}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
  doi = {10.1109/icassp49357.2023.10096880},
}
Figure