Abstract
There is much current interest in using multisensor airborne remote sensing to monitor the structure and biodiversity of woodlands. This paper addresses the application of nonparametric (NP) image-registration techniques to precisely align images obtained from multisensor imaging, which is critical for the successful identification of individual trees using object recognition approaches. NP image registration, in particular, the technique of optimizing an objective function containing similarity and regularization terms, provides a flexible approach for image registration. Here, we develop a NP registration approach in which a normalized gradient field is used to quantify similarity and curvature is used for regularization (NGF-Curv method). Using a survey of woodlands in southern Spain as an example, we show that NGF-Curv can be successful at fusing datasets when there is little prior knowledge about how the datasets are interrelated. The validity of NGF-Curv in airborne remote sensing is demonstrated by a series of experiments showing it is capable of aligning images precisely, making it a valuable component of algorithms designed to identify objects within multisensor datasets.
The Challenge of Multisensor Fusion
In environmental monitoring, multisensor fusion is considered the "holy grail." LiDAR provides 3D structural data, hyperspectral sensors reveal chemical properties, and high-resolution photography provides fine-grained visual detail. However, precisely "stacking" these images is notoriously difficult due to three primary factors:
- Modality Mismatch: A tree crown appears fundamentally different in a LiDAR intensity map compared to a color photograph or a specific hyperspectral band.
- Geometric Distortion: Aircraft motion (roll, pitch, yaw) and rugged terrain create non-linear displacements that standard affine transformations (rotation and scaling) cannot resolve.
- Lack of Metadata: Datasets often lack precise ground control points (GCPs) or perfect georeferencing, leading to misalignments that prevent the identification of individual trees.
Methodology: The NGF-Curv Approach
To address these challenges, we propose a nonparametric (NP) registration approach called NGF-Curv. Unlike parametric methods that rely on a few global variables, NP methods allow every pixel to move independently to find the optimal fit. The core of this method involves two mathematical components:
- Normalized Gradient Field (NGF): Rather than comparing raw brightness—which varies wildly between sensors—NGF compares the direction of gradients (edges). If two sensors detect an edge in the same location, they are likely observing the same object.
- Curvature Regularization (Curv): Because NP methods have high degrees of freedom, they can potentially warp images into unrealistic shapes. Curvature regularization acts as a constraint that penalizes oscillations, ensuring the resulting transformation is smooth and physically plausible.
The implementation utilizes a variational formulation and a multilevel optimization scheme via the FAIR toolbox, starting with low-resolution versions of the images to find general alignment before refining at full resolution.
Datasets and Study Area
The study utilized a comprehensive survey of Mediterranean woodlands in Los Alcornocales Natural Park, Spain. The data included LiDAR (1-m resolution), hyperspectral imagery (3-m resolution), and digital photography (0.3-m resolution). The terrain varied from flat valleys to rugged hills with rocky outcrops, providing a rigorous test for the algorithm's flexibility.
Key Results and Discussion
NGF-Curv was compared against four industry standards: Normalized Cross Correlation (NCC), Mutual Information (MI), NGF (without curvature), and SURF (feature-based).
In both flat and rugged terrain, NGF-Curv achieved the highest correlation and the lowest mean error between registered images. Qualitative results demonstrate that features like roads and individual tree crowns join seamlessly across sensor boundaries, whereas other methods often showed visible "jumps" or misalignments.
Performance Trade-offs
The primary trade-off for this precision is computational speed. Because the algorithm optimizes every pixel, it is significantly slower than parametric methods. In one test, NGF-Curv required approximately 380 seconds, while SURF took 197 seconds and NCC required only 10 seconds. However, for applications requiring individual tree identification, this increased accuracy is essential.
Conclusion and Future Work
NGF-Curv is robust enough to align images based purely on their internal content, which is vital for historical data or remote areas where ground surveys are impossible. Future work will focus on improving processing speed through GPU acceleration and automating the selection of "smoothness" and "edge" parameters. Additionally, integrating shade-removal algorithms may further improve precision by reducing the impact of "false edges" created by strong shadows.
Summary updated