DCV2I: A Practical Approach for Supporting Geographers' Visual Interpretation in Dune Segmentation with Deep Vision Models

Anqi Lu, Zifeng Wu, Zheng Jiang, Wei Wang, Eerdun Hasi, Yi WangAAAI 2024 · DOI

TL;DR

This paper introduces DCV2I, a system that seamlessly integrates deep learning-based dune segmentation models into geographers' existing GIS workflows, specifically ArcGIS. By automating a tedious manual task and allowing for easy refinement within familiar tools, DCV2I significantly reduces geographers' workload and accelerates research on desertification.

What Problem Does This Paper Address?

Visual interpretation of aerial and satellite imagery is a cornerstone of human geography, enabling the identification, classification, and quantification of geographic features. However, this process is notoriously time-consuming and labor-intensive, requiring extensive manual effort from professional geographers. For tasks like dune segmentation, where images can contain hundreds of objects with diverse morphological traits, the manual workload becomes overwhelming, often leading to only a small fraction of collected data being analyzed. Existing automated tools often lack the sophistication to handle the nuances of such data or are difficult for geographers to integrate into their established workflows. This gap hinders comprehensive analysis and the discovery of valuable insights into phenomena like desertification.

Methods

The core of DCV2I is a novel deep dune segmentation model built upon the Attention U-Net architecture. To enhance its ability to capture multi-scale features crucial for segmenting dunes of varying sizes and complexities, the researchers integrated an Atrous Spatial Pyramid Pooling (ASPP) module into the network. This model is then encapsulated within a user-friendly tool that connects directly to ArcGIS, a widely used GIS platform.

The workflow is designed to be non-invasive:

  1. Geographers load an image into ArcGIS.
  2. They initiate the DCV2I tool, which sends the image data to the deep learning model.
  3. The model automatically segments the dunes.
  4. The editable segmentation results are returned to ArcGIS.
  5. Geographers can then refine these results using their familiar GIS tools, without needing to learn deep learning concepts or leave their existing environment.

This approach prioritizes interoperability with existing tools, full encapsulation of the deep learning model (making it invisible to the user), and stability of performance across different dune types.

Datasets

The researchers utilized a newly constructed sand image dataset for training and evaluation. This dataset was created from Pleiades satellite imagery (0.5-meter resolution) of the Mu Us Sandy Land in China. Geographers manually annotated dune boundaries and types, and the images were then cropped into 512x512 pixel tiles for efficient processing. Data augmentation techniques were applied to enhance the dataset. The paper also mentions the use of the VOC dataset for pre-training the VGG16 encoder.

Key Insights and Ideas

  • Geographer-Centric Design is Crucial: The most significant insight is the paramount importance of respecting and integrating with users' existing workflows and tools. Geographers preferred refining automatic results within ArcGIS over learning new interactive segmentation tools like SAM, highlighting that "non-invasive" integration is key to adoption.
  • Encapsulation Over Interactivity: The decision to fully encapsulate the deep learning model, rather than making it interactive, proved to be a critical design choice. This minimized the learning curve and cognitive overhead for geographers.
  • ASPP and Attention for Dune Nuances: The combination of ASPP for multi-scale feature extraction and the Attention mechanism within the U-Net architecture was essential for accurately segmenting the diverse morphological traits of dunes, outperforming standard U-Net.
  • Practical Value Trumps Pure Performance: While model performance metrics are important, the real value of DCV2I lies in its practical utility, demonstrated by the high percentage of automatically generated segments used as-is and the significant time savings for geographers.

Key Results

  • Model Performance: The proposed DCV2I model achieved state-of-the-art results on their dune segmentation dataset, with accuracy of 96.72%, F1 Score of 72.96%, mPA of 80.67%, and MIoU of 75.09%. This significantly outperformed baseline models like U-Net and DeepLabV3+.
  • Time Savings: DCV2I reduced the time required for dune segmentation from several days (manual) to less than an hour, representing a substantial efficiency gain.
  • User Acceptance: Professional geographers found DCV2I easy to use with minimal training, were highly satisfied with the results, and expressed strong interest in continuing its use. Over 30% of the Mu Us Sandy Land area (over 13,000 sq km) was segmented using DCV2I by August 2023, far exceeding manual efforts over the previous five years.
  • Qualitative Improvements: Visual comparisons showed DCV2I produced more accurate dune boundaries and fewer misclassifications compared to U-Net.

Limitations and Future Work

  • Waiting Time: While faster than native GIS tools, the model still requires a waiting period (10-15 minutes for a 200M pixel image on a high-end laptop) for processing. Accelerating inference speed on personal computing devices is a key area for future work.
  • Generalization: While the model showed stable performance, further validation on diverse dune types and geographic regions would be beneficial.
  • Broader Adoption: The authors plan to further promote the adoption of DCV2I within the geography community.
  • Expanding Scope: The success of DCV2I in dune segmentation suggests potential for applying similar integrated approaches to other geographic interpretation tasks.

Summary updated

BibTeX
@inproceedings{lu2024dcv,
  author = {Anqi Lu and Zifeng Wu and Zheng Jiang and Wei Wang and Eerdun Hasi and Yi Wang},
  title = {{DCV2I: A Practical Approach for Supporting Geographers' Visual Interpretation in Dune Segmentation with Deep Vision Models}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30313},
}
Figure