Abstract
Airborne LiDAR technology has proven to be a powerful tool for obtaining high-density, high-accuracy surface information. However, processing the resulting massive point cloud data to generate high-quality Digital Elevation Models (DEMs) requires significant time and computing resources. This paper explores the use of Apache Hadoop, a distributed system infrastructure featuring the Hadoop Distributed File System (HDFS) and the MapReduce parallel programming model, to improve the efficiency of DEM generation. Using point cloud data from Dongting Lake, China (approximately 311.6 million points), the researchers implemented a Hadoop-based algorithm and compared its performance against a traditional sequential C++ implementation. The results indicate that while non-Hadoop implementations perform well when memory is sufficient, the Hadoop-based approach offers a superior performance-cost ratio and better stability when handling vast quantities of data.
Introduction
Airborne LiDAR (Light Detection and Ranging) is the gold standard for mapping terrain, capable of capturing millions of data points per second. However, this high density is a double-edged sword: the resulting "point clouds" are often so massive that traditional single-machine algorithms struggle to process them. Generating a Digital Elevation Model (DEM)—a 3D grid representing the earth's surface—requires filtering out non-terrain objects and interpolating millions of points into a regular grid. On standard workstations, this process often hits memory limits and becomes agonizingly slow. The authors sought to leverage Hadoop's distributed framework to make mass DEM production faster and more economical.
Methodology
The researchers developed a workflow that transitions from standard sequential processing to a distributed MapReduce model. The process involves three main stages: data pre-processing (converting raw files to LAS format), terrain filtering (separating ground points), and the core innovation: parallelized interpolation.
The MapReduce Approach
The interpolation step, using the Inverse Distance Weighted (IDW) method, was parallelized to handle massive point sets across a cluster:
- Map Phase: Each LiDAR point is treated as an input. The "Mapper" determines which grid cell the point belongs to and calculates its distance to the center of that cell.
- Reduce Phase: The "Reducer" collects all points assigned to a specific grid cell and calculates the final elevation (Z-value) for that cell based on the weighted average of the surrounding points.
Experimental Results
The study utilized a real-world dataset from the Gong Shuangcha flood diversion area in Dongting Lake, China, consisting of 311.6 million points (5.80 GB). The algorithm's efficiency was tested on Dell PowerEdge R430 servers in various configurations.
As shown in the performance data, processing time increases linearly with the number of points. A key finding was the scalability trade-off: while a highly optimized C++ program can be faster than Hadoop when data fits entirely within a machine's RAM, Hadoop becomes the superior choice as data volume scales beyond the capacity of a single computer. Furthermore, the Hadoop approach demonstrated a higher performance-cost ratio by allowing organizations to combine affordable commodity hardware into a powerful resource pool.
Conclusion
The Hadoop-based algorithm effectively addresses the memory and processing bottlenecks associated with massive LiDAR datasets. While network I/O overhead can impact performance on smaller tasks, the distributed approach is more stable and economical for large-scale geospatial tasks. Future work will focus on parallelizing the initial terrain filtering phase and optimizing network bandwidth to further enhance efficiency in complex topographies.
Summary updated