Building Block Extraction from Historical Maps Using Deep Object Attention Networks

Jian Yang, Yao Zhao, Guangxia Wang, Jian Yang 0034, Lantian Zhang, Xiaofei QiIsprs Int. J. geo inf 2022 · DOI

TL;DR

This paper introduces DOANet, a novel deep learning architecture for automatically extracting building blocks from historical maps. By combining a multi-scale feature extraction network with an object attention mechanism and leveraging transfer learning, DOANet significantly improves accuracy, especially in scenarios with limited annotated data, addressing a key bottleneck in historical map analysis.

What Problem Does This Paper Address?

Historical maps are invaluable resources for understanding past landscapes and human activity, but extracting precise geographic information from them is challenging. Traditional methods often lack automation and require extensive manual effort. While deep learning has shown promise, applying it to historical maps faces unique hurdles: building blocks vary greatly in size and style, can overlap, and are often obscured by other map elements or damaged by time. Furthermore, the high cost of annotating these maps for training deep learning models severely limits their practical application. This paper aims to bridge this gap by developing an automated, accurate, and data-efficient method for building block extraction from historical maps.

Methods

The core of this work is the DOANet (Deep Object Attention Network), built upon the HRNet and OCRNet frameworks.

  • Feature Extraction Module: Inspired by HRNet, this module uses parallel high-resolution and low-resolution sub-networks with multi-scale feature fusion. This design helps maintain fine-grained details and robustly capture local information, mitigating issues caused by down-sampling in traditional CNNs.
  • Object Attention Module: This module is the key innovation. It comprises:
  • Criss-Cross Attention: This mechanism allows each pixel to attend to all other pixels in the same row and column, capturing long-range spatial dependencies and contextual information.
  • Object Context Module: This module fuses features from different receptive fields to capture detailed contextual information. Together, these attention mechanisms enhance the network's ability to learn building block features by focusing on relevant global and local context, while suppressing irrelevant features.

  • Transfer Learning: To overcome the challenge of limited annotated historical map data, the researchers employed transfer learning. They pre-trained the DOANet on a large, publicly available remote sensing dataset (WHU building dataset) and then fine-tuned it on a smaller, specific historical map dataset. This approach effectively transfers learned features and reduces the need for extensive manual annotation on the target historical maps.

Datasets

  • Source Domain Dataset: WHU building dataset (used for pre-training).
  • Target Domain Dataset: ICDAR2021 Competition on Historical Map Segmentation dataset. This dataset includes training, validation, and three test images of historical maps of Paris (1860-1940). The building blocks in the training and validation images were manually annotated. For experimentation, images were cropped into 512x512 patches.

Key Insights and Ideas

  • Attention is Crucial for Historical Maps: The paper strongly suggests that attention mechanisms are vital for capturing the complex spatial relationships and contextual cues present in historical maps, which are often lost in standard CNN architectures.
  • Multi-Scale Fusion is Key: The HRNet-inspired parallel multi-resolution approach effectively preserves both fine details and broader context, which is essential for distinguishing buildings of various sizes and complexities.
  • Transfer Learning is a Game-Changer for Scarce Data: The significant performance boost achieved by using transfer learning highlights its critical role in making deep learning practical for historical map analysis, where annotated data is inherently limited and expensive to create.
  • Local vs. Global Context: The DOANet effectively balances the need for fine-grained local feature extraction with the understanding of global spatial context, a common challenge in image segmentation.

Key Results

The proposed DOANet achieved state-of-the-art results on the ICDAR2021 historical map segmentation task.

  • Performance Improvement: DOANet's PQ (Performance Quality) score was higher than other methods for all three test images, with an overall increase of at least 7.2% compared to existing algorithms.
  • Data Efficiency: The study demonstrated that DOANet requires significantly fewer training samples (around 600) to achieve a high performance score (>75%) compared to a model without transfer learning (which needed ~1200 samples). This directly translates to reduced annotation costs.
  • Qualitative Improvements: Visual results showed that DOANet effectively aggregated local features and reduced the impact of blurred boundaries, leading to more accurate building block extraction.

Limitations and Future Work

  • Overlapping Buildings: The authors acknowledge that the method still struggles with accurately separating overlapping or nested building blocks, as illustrated by a qualitative example where several distinct buildings were incorrectly grouped.
  • Generalizability: Future work will focus on improving the handling of overlapping buildings and extending the method's applicability to different map styles, languages, and time periods to enhance its generalizability.

Summary updated

BibTeX
@article{yang2022building,
  author = {Jian Yang and Yao Zhao and Guangxia Wang and Jian Yang and Lantian Zhang and Xiaofei Qi},
  title = {{Building Block Extraction from Historical Maps Using Deep Object Attention Networks}},
  journal = {Isprs Int. J. geo inf},
  year = {2022},
  doi = {10.3390/ijgi11110572},
}
Figure