Abstract
Forest canopy closure (FCC) is a vital parameter for monitoring forest ecosystems and managing resources. As remote sensing data enters the "big data" era, traditional serial processing methods struggle to keep pace with the sheer volume of satellite imagery. This paper presents Sparkpr, a parallel forest canopy density inversion algorithm based on the Apache Spark framework. Utilizing GF-1 remote sensing images and ground-truth data from the Maoershan Laoshan Experimental Forest Farm, the study employs a multi-element linear regression model for parallel inversion. Experimental results demonstrate that Sparkpr maintains high accuracy (87.65%) while achieving significant speedups—up to 4.63x on a 9-node cluster—compared to serial processing.
Introduction
Forest Canopy Closure (FCC) represents the ratio of the vertical projection area of tree crowns to the total woodland area. It is a fundamental metric for assessing forest health, carbon sequestration capacity, and urban environmental quality. Historically, FCC estimation relied on manual field surveys or serial processing of satellite imagery. However, the rapid advancement of remote sensing technology has led to a data explosion. High-resolution imagery from satellites like GF-1 generates massive datasets that exceed the processing capabilities of single-machine serial algorithms. While distributed frameworks like MapReduce offered initial solutions, their reliance on disk I/O for intermediate data storage limits efficiency. Sparkpr addresses these challenges by leveraging Spark's in-memory computing capabilities.
Methodology
The Sparkpr algorithm is designed to parallelize the inversion process using the Apache Spark framework. The core of the approach involves several key stages:
- Distributed Storage and Indexing: Remote sensing images are stored in the Hadoop Distributed File System (HDFS). To manage the large-scale data efficiently, the system utilizes a structured mapping between data blocks, indices, and storage nodes.
- Feature Selection: From an initial pool of 32 spectral combinations, correlation analysis identified five critical indices for the regression model: NDVI (Normalized Difference Vegetation Index), RVI (Ratio Vegetation Index), DVI (Difference Vegetation Index), ARVI (Atmospherically Resistant Vegetation Index), and a specific spectral ratio ($X_{17}$).
- Parallel Inversion: The algorithm converts image data into Resilient Distributed Datasets (RDDs). The inversion function, based on a multivariate linear regression model, is mapped across the cluster nodes. This allows simultaneous calculation of FCC values for different image segments, with results aggregated on the master node.
Experimental Results
The performance of Sparkpr was evaluated using GF-1 imagery of the Laoshan Experimental Forest Farm and 80 ground-truth sample points.
- Accuracy: The multivariate linear regression model achieved an $R^2$ of 0.651. The overall estimation accuracy (EA) was 87.65%, meeting the standards for forest resource surveys. Comparison of histograms between serial and parallel outputs confirmed that the parallelization process did not introduce data errors.
- Performance Scaling: The study compared serial processing against Spark Standalone and Spark on YARN modes. While serial processing took 363 seconds, the 9-node Spark on YARN cluster reduced this to 78 seconds, a 4.63x speedup. The efficiency gains were most pronounced when transitioning from a single node to a small cluster, with performance continuing to improve as more nodes were added.
Conclusion
Sparkpr provides a robust and efficient solution for large-scale forest canopy monitoring. By utilizing Spark's in-memory parallel processing, it overcomes the bottlenecks of traditional serial methods without compromising on accuracy. Future research will explore the integration of non-linear machine learning models and real-time processing via Spark Streaming to further enhance the system's capabilities.
Summary updated