Uncertainty Quantification for Hyperspectral Image Denoising Frameworks based on Low-rank Matrix Approximation

Shaobo Xia, Jingwei Song, Dong Chen, Jun Wang, Chen, DongarXiv 2020 · DOI · arXiv

Abstract: Sliding-window based low-rank matrix approximation (LRMA) is a technique widely used in hyperspectral images (HSIs) denoising or completion. However, the uncertainty quantification of the restored HSI has not been addressed to date. Accurate uncertainty quantification of the denoised HSI facilitates applications such as multi-source or multi-scale data fusion, data assimilation, and product uncertainty quantification, since these applications require an accurate approach to describe the statistical distributions of the input data. Therefore, we propose a prior-free closed-form element-wise uncertainty quantification method for LRMA-based HSI restoration. Our closed-form algorithm overcomes the difficulty of the HSI patch mixing problem caused by the sliding-window strategy used in the conventional LRMA process. The proposed approach only requires the uncertainty of the observed HSI and provides the uncertainty result relatively rapidly and with similar computational complexity as the LRMA technique. We conduct extensive experiments to validate the estimation accuracy of the proposed closed-form uncertainty approach. The method is robust to at least 10% random impulse noise at the cost of 10-20% of additional processing time compared to the LRMA. The experiments indicate that the proposed closed-form uncertainty quantification method is more applicable to real-world applications than the baseline Monte Carlo test, which is computationally expensive.

Introduction

Hyperspectral images (HSIs), which capture hundreds of narrow spectral bands, are essential for remote sensing but are notoriously susceptible to noise. Low-Rank Matrix Approximation (LRMA) is a state-of-the-art technique for cleaning this data by assuming the "true" signal lives in a much lower-dimensional space than the raw noise.

However, until now, LRMA has been a "black box" regarding reliability. While we get a denoised image, we lack a measure of how much to trust specific pixels. This is a significant hurdle for high-stakes applications like data fusion (combining satellite data with drone data) or data assimilation in climate models, where knowing the input's statistical distribution is mandatory. The challenge is twofold: the mathematical complexity of non-linear LRMA functions and the "patch mixing" effect caused by the sliding-window strategy used to process massive HSI data.

Methodology

The proposed method introduces a prior-free closed-form uncertainty quantification approach. Instead of running a denoising algorithm hundreds of times (the computationally expensive Monte Carlo approach), the researchers derived a direct mathematical relationship between input noise and output variance.

The proposed framework integrates the LRMA restoration process with a parallel closed-form uncertainty quantification path.

Key components include:

  • Patch-wise Propagation: Using Singular Value Decomposition (SVD), the method relates the variance of the observed noisy patch to the variance of the estimated low-rank components.
  • Solving "Uncertainty Mixing": This is the paper's most novel contribution. Recognizing that overlapping patches are not independent, the authors used a weighted average formulation based on mutual information, where the correlation between patches is determined by their spatial overlap ratio.
  • Solver Independence: The method is "plug-and-play," requiring only that the chosen solver (like GoDec or ALM) reaches a near-global minimum.

Experimental Validation

The method was evaluated using three standard benchmarks: Indian Pines (AVIRIS), University of Pavia (ROSIS), and Washington D.C. Mall. These datasets provided a mix of simulated and real-world urban environments with complex spectral signatures.

Key Insights

  • Gaussian Preservation: Through Shapiro-Wilk tests and Q-Q plots, the authors proved that the noise in the restored HSI still follows a Gaussian distribution. This justifies using variance as a sufficient metric for uncertainty.
Q-Q plot for the Washington D.C. dataset demonstrating that the restored noise follows a standard normal distribution.
  • Efficiency: On the Washington D.C. dataset, a Monte Carlo test took 12,437 seconds, while the proposed closed-form method (LRMA + Uncertainty) took only 1,590 seconds—a massive speedup with only 10–20% overhead compared to standard denoising.
  • Accuracy: The "coverage rate" (how often the true value falls within the predicted 95% confidence interval) remained consistently between 94% and 97% for moderate noise levels.

Conclusion

This work provides a mathematically rigorous confidence measure for every pixel in a denoised HSI. By overcoming the patch-mixing problem and maintaining low computational complexity, the method makes uncertainty quantification practical for large-scale remote sensing pipelines. Future work will focus on improving sensitivity in extremely high-noise conditions and accounting for structured noise like sensor striping.

Summary updated

Abstract

Sliding-window based low-rank matrix approximation (LRMA) is a technique widely used in hyperspectral images (HSIs) denoising or completion. However, the uncertainty quantification of the restored HSI has not been addressed to date. Accurate uncertainty quantification of the denoised HSI facilitates to applications such as multi-source or multi-scale data fusion, data assimilation, and product uncertainty quantification, since these applications require an accurate approach to describe the statistical distributions of the input data. Therefore, we propose a prior-free closed-form element-wise uncertainty quantification method for LRMA-based HSI restoration. Our closed-form algorithm overcomes the difficulty of the HSI patch mixing problem caused by the sliding-window strategy used in the conventional LRMA process. The proposed approach only requires the uncertainty of the observed HSI and provides the uncertainty result relatively rapidly and with similar computational complexity as the LRMA technique. We conduct extensive experiments to validate the estimation accuracy of the proposed closed-form uncertainty approach. The method is robust to at least 10% random impulse noise at the cost of 10-20% of additional processing time compared to the LRMA. The experiments indicate that the proposed closed-form uncertainty quantification method is more applicable to real-world applications than the baseline Monte Carlo test, which is computationally expensive. The code is available in the attachment and will be released after the acceptance of this paper.
BibTeX
@misc{xia2020uncertainty,
  author = {Shaobo Xia and Jingwei Song and Dong Chen and Jun Wang and Chen, Dong},
  title = {{Uncertainty Quantification for Hyperspectral Image Denoising Frameworks based on Low-rank Matrix Approximation}},
  year = {2020},
  eprint = {2004.10959},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2004.10959},
}
Figure