Sparkpr: An Efficient Parallel Inversion of Forest Canopy Closure

Guangsheng Chen, Tongtong Lou, Weipeng Jing, Zeyu WangIEEE Access 2019 · OpenReview

Abstract

Forest canopy closure (FCC) is a vital parameter for monitoring forest ecosystems and managing resources. As remote sensing data enters the "big data" era, traditional serial processing methods struggle to keep pace with the sheer volume of satellite imagery. This paper presents Sparkpr, a parallel forest canopy density inversion algorithm based on the Apache Spark framework. Utilizing GF-1 remote sensing images and ground-truth data from the Maoershan Laoshan Experimental Forest Farm, the study employs a multi-element linear regression model for parallel inversion. Experimental results demonstrate that Sparkpr maintains high accuracy (87.65%) while achieving significant speedups—up to 4.63x on a 9-node cluster—compared to serial processing.

Introduction

Forest Canopy Closure (FCC) represents the ratio of the vertical projection area of tree crowns to the total woodland area. It is a fundamental metric for assessing forest health, carbon sequestration capacity, and urban environmental quality. Historically, FCC estimation relied on manual field surveys or serial processing of satellite imagery. However, the rapid advancement of remote sensing technology has led to a data explosion. High-resolution imagery from satellites like GF-1 generates massive datasets that exceed the processing capabilities of single-machine serial algorithms. While distributed frameworks like MapReduce offered initial solutions, their reliance on disk I/O for intermediate data storage limits efficiency. Sparkpr addresses these challenges by leveraging Spark's in-memory computing capabilities.

Methodology

The Sparkpr algorithm is designed to parallelize the inversion process using the Apache Spark framework. The core of the approach involves several key stages:

  1. Distributed Storage and Indexing: Remote sensing images are stored in the Hadoop Distributed File System (HDFS). To manage the large-scale data efficiently, the system utilizes a structured mapping between data blocks, indices, and storage nodes.
Data indexing and storage node mapping in the Sparkpr framework.
  1. Feature Selection: From an initial pool of 32 spectral combinations, correlation analysis identified five critical indices for the regression model: NDVI (Normalized Difference Vegetation Index), RVI (Ratio Vegetation Index), DVI (Difference Vegetation Index), ARVI (Atmospherically Resistant Vegetation Index), and a specific spectral ratio ($X_{17}$).
  2. Parallel Inversion: The algorithm converts image data into Resilient Distributed Datasets (RDDs). The inversion function, based on a multivariate linear regression model, is mapped across the cluster nodes. This allows simultaneous calculation of FCC values for different image segments, with results aggregated on the master node.

Experimental Results

The performance of Sparkpr was evaluated using GF-1 imagery of the Laoshan Experimental Forest Farm and 80 ground-truth sample points.

  • Accuracy: The multivariate linear regression model achieved an $R^2$ of 0.651. The overall estimation accuracy (EA) was 87.65%, meeting the standards for forest resource surveys. Comparison of histograms between serial and parallel outputs confirmed that the parallelization process did not introduce data errors.
  • Performance Scaling: The study compared serial processing against Spark Standalone and Spark on YARN modes. While serial processing took 363 seconds, the 9-node Spark on YARN cluster reduced this to 78 seconds, a 4.63x speedup. The efficiency gains were most pronounced when transitioning from a single node to a small cluster, with performance continuing to improve as more nodes were added.

Conclusion

Sparkpr provides a robust and efficient solution for large-scale forest canopy monitoring. By utilizing Spark's in-memory parallel processing, it overcomes the bottlenecks of traditional serial methods without compromising on accuracy. Future research will explore the integration of non-linear machine learning models and real-time processing via Spark Streaming to further enhance the system's capabilities.

Summary updated

Abstract

Forest canopy closure is an important parameter to study forest ecosystem and understand the status of forest resources. With the development of remote sensing big data, the amount of remote sensing data has increased sharply, which makes the existing serial processing of remote sensing data face severe challenges.In order to satisfy the requirements of efficient remote sensing data processing, Spark open source framework is applied to the parallel processing of remote sensing images, and a parallel forest canopy density inversion algorithm based on Spark is proposed. We call this algorithm Sparkpr. Based on the GF-1 remote sensing images and 80 actual measured sample points obtained by Maoershan Laoshan Experimental Forest Farm of Northeast Forestry University in 2016. In this paper, a multi-element linear regression algorithm is used to carry out parallel inversion of the forest canopy density in the Laoshan Experimental Forest Farm of the Maoershan. The comparison experiment between single machine mode and spark standalone and spark on yarn mode is carried out. The experimental results show that the serial and parallel inversion results of forest depression density based on the model are consistent, and the parallel inversion results are accurate and credible. With the increase of computing nodes, the efficiency of parallel inversion is also improving.
BibTeX
@inproceedings{chen2019sparkpr,
  author = {Guangsheng Chen and Tongtong Lou and Weipeng Jing and Zeyu Wang},
  title = {{Sparkpr: An Efficient Parallel Inversion of Forest Canopy Closure}},
  booktitle = {IEEE Access},
  year = {2019},
  url = {https://openreview.net/forum?id=X6Z5Ab25Z4L},
}
Figure