Automatic Registration Method for Fusion of ZY-1-02C Satellite Images

Qi Chen, Shugen Wang, Bo Wang, Mingwei SunRemote 2013 · DOI

TL;DR

This paper presents a novel coarse-to-fine automatic image registration (AIR) method specifically designed for fusing ZY-1-02C satellite images. By combining Scale-Invariant Feature Transform (SIFT) for initial coarse matching with Normalized Cross-Correlation (NCC) and Least Squares Matching (LSM) for precise matching, along with a Triangulated Irregular Network (TIN) for robust error elimination, the method achieves high sub-pixel accuracy and efficiency suitable for operational satellite data processing.

What Problem Does This Paper Address?

Accurate image registration is crucial for fusing satellite imagery from different sensors, especially for applications like enhancing resolution or combining spectral information. However, ZY-1-02C satellite images present significant challenges: their large size makes direct processing computationally expensive, and inherent inaccuracies in positioning (due to sensor limitations, focal plane distortion, and unrecorded spacecraft jitter) lead to substantial differences between images. Traditional methods struggle to find reliable corresponding points efficiently, either being too slow (area-based matching) or failing to distribute points evenly across large, complex scenes (feature-based matching alone). This paper tackles the problem of achieving both high accuracy and operational efficiency in registering these challenging satellite images for fusion purposes.

Methods

The proposed method employs a coarse-to-fine strategy to overcome the limitations of individual approaches:

  • Geometric Correction: Initial geometric correction is performed using Rational Function Models (RFM) and elevation data (SRTM-DEM) to align images to a common ground coordinate system.
  • Coarse Matching (SIFT on Subsampled Images): To manage large image sizes, images are subsampled once using an interval sampling method. Scale-Invariant Feature Transform (SIFT) is then applied to these smaller images to find an initial set of corresponding points. This step provides a rough global estimation of the transformation.
  • Feature Extraction and Initial Precise Matching: Harris feature points are extracted from the MUX image, and their corresponding points on the HR/PAN image are predicted using the global affine transformation estimated from SIFT. Precise matching is then conducted using Normalized Cross-Correlation (NCC) for pixel-level accuracy and refined with Least Squares Matching (LSM) for sub-pixel precision.
  • Error Elimination (TIN-based Local Estimation): To handle complex, non-uniform distortions that global models can't fully capture, a Triangulated Irregular Network (TIN) is constructed from the matched points. Local affine transformations are estimated within each triangle of the TIN. By analyzing the residual errors of points within these local transformations, false matches are identified and eliminated. This local approach is more robust to complex distortions than global methods.
  • Image Resampling: Finally, image resampling is performed using local affine transformations derived from the optimized TIN structure to achieve high-precision registration. Bilinear interpolation is used for sub-pixel resampling.

Datasets

The experiments were conducted using three datasets from the ZY-1-02C satellite:

  • Dataset 1: Mohe, China (mountainous terrain)
  • Dataset 2: Hangzhou, China (urban and partly mountainous terrain)
  • Dataset 3: Tokyo, Japan (urban and sea area)

These datasets consist of pairs of PAN/HR and MUX images with varying resolutions (5.0m/10.0m for PAN/MUX, and 2.36m/10.0m for HR/MUX). The paper does not introduce new benchmark datasets but uses existing ZY-1-02C data to validate the proposed method.

Key Insights and Ideas

  • Subsampling for Efficiency: The key to handling large satellite images efficiently is subsampling once for coarse matching, rather than using complex multi-level pyramids. This significantly reduces memory consumption and processing time.
  • SIFT for Global Roughness: SIFT, despite its computational cost on full-size images, is effective for initial coarse matching on subsampled data, providing a robust global estimation that drastically reduces the search space for precise matching.
  • TIN for Local Robustness: Complex distortions in satellite imagery necessitate local modeling. The TIN structure provides an efficient way to define local neighborhoods and apply local affine transformations for accurate error elimination, outperforming global models.
  • Coarse-to-Fine Synergy: The true power lies in the integration: coarse SIFT matching guides precise NCC/LSM matching, and the resulting dense, accurate matches are then refined by local TIN-based error elimination. This layered approach ensures both accuracy and robustness.

Key Results

  • High Accuracy: The proposed method achieved sub-pixel registration accuracy across all three datasets, with RMSE values as low as 0.37 pixels (Dataset 1), 0.43 pixels (Dataset 2), and 0.66 pixels (Dataset 3).
  • Efficiency: The registration times for the three datasets were 68.2 s, 83.2 s, and 561.8 s, respectively. These times are deemed suitable for the commercial operation of the Automatic Satellite Data Processing System (ASDPS).
  • Superiority over ENVI: Comparative experiments showed that the proposed method significantly outperformed standard methods available in ENVI (affine, quadratic polynomial, and triangulation) in terms of both accuracy and efficiency. The ENVI methods, particularly affine and quadratic polynomial, suffered from misalignment and "edge phenomena" in fused images.
  • Robustness to Distortion: The method demonstrated effectiveness in handling significant differences in positioning accuracy and complex distortions present in ZY-1-02C imagery.

Limitations and Future Work

  • Limitations:
    • The method relies on Rational Polynomial Coefficients (RPCs) for initial geometric correction. It cannot directly handle images with rotation and scale differences if RPCs are unavailable.
  • For images with very small positioning differences, the coarse-to-fine strategy might be unnecessary, and direct NCC matching could suffice.
  • The accuracy of local affine estimation can be affected in areas with sparse matching point distribution.
  • Bilinear interpolation used in resampling might introduce minor errors.
  • Future Work:
    • Explore more efficient feature-based matching algorithms (e.g., SURF, Gradient Location-Orientation Histogram) to potentially reduce computation intensity.
  • Investigate methods to minimize gray value deviations during geometric correction and resampling, which currently involve double bilinear interpolation.
  • Further research is needed to determine if the approach can handle images with even higher resolution differences.

Summary updated

Abstract

Automatic image registration (AIR) has been widely studied in the fields of medical imaging, computer vision, and remote sensing. In various cases, such as image fusion, high registration accuracy should be achieved to meet application requirements. For satellite images, the large image size and unstable positioning accuracy resulting from the limited manufacturing technology of charge-coupled device, focal plane distortion, and unrecorded spacecraft jitter lead to difficulty in obtaining agreeable corresponding points for registration using only area-based matching or feature-based matching. In this situation, a coarse-to-fine matching strategy integrating two types of algorithms is proven feasible and effective. In this paper, an AIR method for application to the fusion of ZY-1-02C satellite imagery is proposed. First, the images are geometrically corrected. Coarse matching, based on scale invariant feature transform, is performed for the subsampled corrected images, and a rough global estimation is made with the matching results. Harris feature points are then extracted, and the coordinates of the corresponding points are calculated according to the global estimation results. Precise matching is conducted, based on normalized cross correlation and least squares matching. As complex image distortion cannot be precisely estimated, a local estimation using the structure of triangulated irregular network is applied to eliminate the false matches. Finally, image resampling is conducted, based on local affine transformation, to achieve high-precision registration. Experiments with ZY-1-02C datasets demonstrate that the accuracy of the proposed method meets the requirements of fusion application, and its efficiency is also suitable for the commercial operation of the automatic satellite data process system.
BibTeX
@inproceedings{chen2013automatic,
  author = {Qi Chen and Shugen Wang and Bo Wang and Mingwei Sun},
  title = {{Automatic Registration Method for Fusion of ZY-1-02C Satellite Images}},
  booktitle = {Remote},
  year = {2013},
  doi = {10.3390/rs6010157},
}
Figure