Detecting buildings in aerial images

Andres Huertas, Ramakant NevatiaComput. Vis. Graph. Image Process 1988 · DOI

TL;DR

This paper presents a novel method for automatically detecting rectangular buildings in aerial images by leveraging the geometric properties of shadows. By analyzing the interplay between building edges and their cast shadows, the system can overcome limitations of traditional edge detection and infer 3D structure, significantly advancing the state-of-the-art in aerial image analysis.

What Problem Does This Paper Address?

Automatically generating maps from aerial imagery is a crucial task for many applications. A major hurdle in this process is accurately extracting and describing man-made structures, particularly buildings. Existing low-level segmentation techniques often produce fragmented or incomplete outlines of buildings. Furthermore, aerial images lack direct 3D information, making it challenging to distinguish raised structures from ground features (like parking lots) and to estimate their height. This paper tackles these challenges by developing a system that can reliably detect rectangular buildings in medium-density suburban aerial images, even when segmentation is imperfect, by intelligently using shadow information.

Methods

The core of the proposed method lies in a multi-stage process that combines geometric modeling with shadow analysis:

  • Line and Corner Detection: The system begins by applying an edge detection algorithm (USC "LINEAR") to extract line segments from the image. These segments are then used to identify corners, defined as near-orthogonal L-junctions.
  • Corner Interpretation and Shadow Correspondence: Corners are initially classified as "bright" or "dark" based on the intensity of the regions around them. This classification, combined with the known direction of illumination, allows the system to label corners as either "object" or "shadow" corners. The crucial step here is establishing "shadow correspondence" between an object corner and a shadow corner that are geometrically aligned with the illumination direction. This correspondence is a strong indicator of a 3D structure.
  • Boundary Tracing and Shape Verification: The system then traces continuous boundaries by grouping strongly compatible corners (sharing a segment) into chains. These chains are then analyzed to see if they conform to a rectangular model. The method accounts for fragmented boundaries by looking for weaker compatibility between corners and attempting to close gaps.
  • Hypothesis Verification and 3D Reconstruction: A closed outline is considered a potential building if it contains a corner with shadow correspondence. If not, the system actively searches for shadow evidence. The width of the shadow, in conjunction with the illumination direction, is used to estimate the building's height and construct a basic 3D model.

Datasets

The paper primarily uses aerial images of the Los Angeles International Airport (LAX) for demonstration and testing. Figure 1 shows a 512x512 resolution image of LAX. The authors do not explicitly mention creating new benchmarks but rather test their method on this real-world dataset.

Key Insights and Ideas

  • Shadows as an Asset, Not a Hindrance: Traditionally, shadows in images are treated as a nuisance that can obscure features. This paper ingeniously reframes shadows as a powerful cue for disambiguating 3D structures and estimating their height.
  • Geometric Model for Robustness: By assuming a generic rectangular model for buildings, the system can overcome the inherent inaccuracies and fragmentation of low-level image segmentation. This model-based approach provides a strong prior that guides the interpretation of noisy data.
  • Shadow Correspondence for 3D Inference: The concept of "shadow correspondence" is a key innovation. By matching object corners with their corresponding shadow corners, the system gains confidence in its building hypothesis and obtains crucial information for inferring depth and height.
  • Handling Imperfect Data: The method is designed to be robust to image quality issues, fragmented segments, and partial outlines, which are common in real-world aerial imagery.

Key Results

The paper demonstrates the effectiveness of its approach on aerial images of LAX. Figure 15 shows detected closed boundaries (boxes) representing buildings. Figure 17c shows verified boxes, highlighting that even distorted and incomplete outlines can be correctly identified if shadow correspondence is found. The system successfully identifies seven buildings in a challenging section of the LAX image, providing estimates of their perimeter, area, height, orientation, centroid, and number of sides/corners.

Limitations and Future Work

The authors acknowledge several limitations and suggest avenues for future research:

  • Poor Contrast and Texture: Buildings with low contrast or fragmented edges due to nearby objects (like vehicles) are difficult to detect. Future work will focus on improving low-level edge detection and line-linking methods, potentially by detecting weak edges when line direction is known.
  • Small Building Size: Some buildings are missed simply due to their small size. Improving image resolution or processing at higher resolutions could address this.
  • Complex Roof Structures: The current model assumes simple, flat roofs. More complex buildings with intricate roof structures require more sophisticated models. Stereo vision and improved shadow analysis are suggested for such cases.
  • Building Density and Shadow Overlap: In dense urban areas, shadows from one building can fall on others, complicating segmentation. More complex models and stereo vision are proposed to handle these scenarios.
  • Limited 3D Model: The current 3D reconstruction is basic, primarily estimating height. More detailed 3D modeling would require more advanced techniques.

Summary updated

BibTeX
@article{huertas1988detecting,
  author = {Andres Huertas and Ramakant Nevatia},
  title = {{Detecting buildings in aerial images}},
  journal = {Comput. Vis. Graph. Image Process},
  year = {1988},
  doi = {10.1016/0734-189x(88)90016-3},
}
Figure