Extracting urban water by combining deep learning and Google Earth Engine

Yudie Wang, Zhiwei Li, Chao Zeng, Gui-Song Xia, Huanfeng ShenarXiv 2019 · arXiv · OpenReview

TL;DR

This paper introduces the Offline Training and Online Prediction (OTOP) method, which combines a deep learning model (MSCNN) trained offline with the Google Earth Engine (GEE) platform for efficient and accurate urban water extraction from Landsat imagery. OTOP leverages GEE's massive data processing capabilities and CNN's feature extraction power, outperforming traditional methods like MNDWI and Random Forest, especially in complex urban environments.

What Problem Does This Paper Address?

Urban water bodies are crucial for urban ecosystems and development, but their distribution is rapidly changing due to urbanization and climate change. Accurately mapping these water bodies is essential for effective urban management and planning. Traditional remote sensing methods, like water indices (e.g., MNDWI), struggle with distinguishing water from similar spectral signatures (e.g., shadows, asphalt) and require manual threshold adjustments. Machine learning methods (e.g., Random Forest) offer improvements but still depend heavily on feature engineering and high-quality training data. While deep learning, specifically Convolutional Neural Networks (CNNs), shows promise, applying it to large-scale, long-term urban water extraction faces challenges with data download, storage, and computational demands. This paper aims to bridge this gap by proposing a method that combines the strengths of deep learning with the cloud-based processing power of Google Earth Engine for efficient and robust urban water extraction.

Methods

The core innovation is the Offline Training and Online Prediction (OTOP) framework.

  • Offline Training: A Multiscale Convolutional Neural Network (MSCNN) is trained offline. This MSCNN is a simplified version of a previously proposed Multiscale Convolutional Feature Fusion (MSCFF) model, adapted for urban water extraction. It uses six different scales of convolutional operations to capture multiscale and multilevel features, enhancing its ability to distinguish water from complex urban backgrounds. The input to the network consists of six Landsat bands (visible, near-infrared, and two short-infrared bands). The model is trained using a large dataset of Landsat images and manually created water masks.
  • Online Prediction on GEE: Once trained, the MSCNN model's parameters are uploaded to Google Earth Engine. GEE then simulates the MSCNN's processing flow (convolution, pooling, up-sampling) using its own JavaScript API and parallel processing capabilities. This allows for urban water extraction directly on GEE without downloading massive datasets, leveraging GEE's access to global Landsat archives and its computational power.

Datasets

  • Training Data: 36 Landsat images covering 36 major cities in China, acquired at different times and from different sensors (TM, ETM+, OLI). These images were selected to represent diverse climates, topographies, and development levels. Cloudy images were also included to improve robustness.
  • Water Masks: Manually created reference water masks for the training data, following specific criteria (e.g., minimum pixel size, handling of slender water bodies and mixed pixels).
  • Testing Data: Central urban areas of four specific cities: Changchun, Wuhan, Kunming, and Guangzhou. These were chosen to represent diverse water types and urban features.
  • Extended Validation Data: Landsat images from an additional 32 major cities across China, used to assess the method's universality.

Key Insights and Ideas

  • Synergy of GEE and Deep Learning: The OTOP framework effectively combines the strengths of GEE (massive data access, cloud processing) and CNNs (powerful feature extraction) to overcome the limitations of each individually. This is a significant step towards making deep learning more practical for large-scale geospatial analysis.
  • Robustness to Urban Complexity: The MSCNN's multiscale feature extraction is crucial for accurately distinguishing urban water from challenging features like building shadows and low-albedo surfaces, which often confuse traditional methods.
  • Eliminating Manual Thresholding: By using a trained deep learning model, OTOP removes the need for subjective and time-consuming manual threshold selection, a major drawback of index-based methods.
  • Efficiency Gains: Processing directly on GEE eliminates the need for data download and local storage, drastically reducing processing time and computational burden for large-scale, long-term studies.

Key Results

  • High Accuracy: In the four test cities (Changchun, Wuhan, Kunming, Guangzhou), OTOP achieved mean Kappa of 0.924, mean F1-score of 0.930, and mean IoU of 0.869. These metrics were consistently higher than those achieved by MNDWI and Random Forest.
  • Improved Discrimination: OTOP demonstrated superior ability to differentiate urban water from building shadows and dark surfaces compared to MNDWI and RF, especially in dense urban areas.
  • Extended Validation Success: In an extended validation across 32 additional Chinese cities, OTOP maintained high accuracy (Kappa and F1-score generally above 0.9, IoU above 0.82), indicating strong universality and robustness across diverse urban environments and sensors.
  • Balanced Errors: OTOP achieved a better balance between omission errors (missing water) and commission errors (misclassifying non-water as water) compared to the other methods.

Limitations and Future Work

  • Landsat Resolution: The paper acknowledges that the 30m spatial resolution of Landsat images limits the detection of very small water bodies or mixed pixels at edges, leading to some misdetections.
  • Pixel-Based vs. Edge Detection: While OTOP is effective, it's noted that it's slightly inferior to pixel-based methods in terms of the fineness of water detection at edges.
  • Training Data Bias: The model was trained on manually drawn water masks from Landsat images in China. For global applicability, more diverse training data from different urban environments worldwide would be beneficial.
  • Future Work: The authors suggest that adding more accurate edge pixel samples to the training data could improve detection of fine water features. They also propose that the OTOP framework could be extended to other land use classification tasks.

Summary updated

Abstract

Urban water is important for the urban ecosystem. Accurate and efficient detection of urban water with remote sensing data is of great significance for urban management and planning. In this paper, we proposed a new method to combine Google Earth Engine (GEE) with multiscale convolutional neural network (MSCNN) to extract urban water from Landsat images, which is summarized as offline training and online prediction (OTOP). That is, the training of MSCNN was completed offline, and the process of urban water extraction was implemented on GEE with the trained parameters of MSCNN. The OTOP can give full play to the respective advantages of GEE and CNN, and make the use of deep learning method on GEE more flexible. It can process available satellite images with high performance without data download and storage, and the overall performance of urban water extraction is also higher than that of the modified normalized difference water index (MNDWI) and random forest. The mean kappa, F1-score and intersection over union (IoU) of urban water extraction with the OTOP in Changchun, Wuhan, Kunming and Guangzhou reached 0.924, 0.930 and 0.869, respectively. The results of the extended validation in the other major cities of China also show that the OTOP is robust and can be used to extract different types of urban water, which benefits from the structural design and training of the MSCNN. Therefore, the OTOP is especially suitable for the study of large-scale and long-term urban water change detection in the background of urbanization.
BibTeX
@misc{wang2019extracting,
  author = {Yudie Wang and Zhiwei Li and Chao Zeng and Gui-Song Xia and Huanfeng Shen},
  title = {{Extracting urban water by combining deep learning and Google Earth Engine}},
  year = {2019},
  eprint = {1912.10726},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1912.10726},
}
Figure