Abstract
We investigate the extension of the recently proposed weighted Fourier burst accumulation (FBA) method into the wavelet domain. The purpose of FBA is to reconstruct a clean and sharp image from a sequence of blurred frames. This concept lies in the construction of weights to amplify dominant frequencies in the Fourier spectrum of each frame. The reconstructed image is then obtained by taking the inverse Fourier transform of the average of all processed spectra. We first suggest replacing the rigid registration step used in the original algorithm with a nonrigid registration in order to process sequences acquired through atmospheric turbulence. Second, we propose to work in a wavelet domain instead of the Fourier one. This leads us to the construction of two types of algorithms. Finally, we propose an alternative approach to replace the weighting idea by an approach promoting the sparsity in the used space. Several experiments are provided to illustrate the efficiency of the proposed methods.
Introduction
Imaging over long distances—such as through telescopes or across expansive landscapes—is frequently hampered by atmospheric turbulence. This phenomenon introduces two primary degradations: spatially varying blur and complex geometric distortions (warping). While the Fourier Burst Accumulation (FBA) method was previously developed to mitigate "hand-shake" blur by selecting and averaging the best frequencies from a sequence of images, it struggles with the non-rigid nature of atmospheric turbulence. Furthermore, the global nature of the Fourier transform is often suboptimal for capturing the localized, sharp features of objects obscured by shimmer.
This research proposes a significant evolution of the FBA framework by transitioning the process into the wavelet domain and incorporating non-rigid registration. By leveraging the localized representation of wavelets and the efficiency of sparsity-promoting techniques, the authors demonstrate a more robust pipeline for reconstructing stable, high-contrast images from turbulent sequences.
Methodology
The proposed framework introduces three major technical upgrades to the standard burst accumulation pipeline:
1. Non-Rigid Registration
Standard FBA assumes rigid motion, where the entire frame shifts uniformly. In contrast, atmospheric turbulence causes different parts of an image to warp in different directions simultaneously. To correct this, the authors implement a multiscale Lucas-Kanade optical flow algorithm. This step "un-warps" the local geometric distortions before the deblurring process begins, ensuring that subsequent averaging does not result in a "wobbly" or ghosted final image.
2. Transition to the Wavelet Domain
Rather than analyzing global frequencies, the researchers move the mathematical operations to the wavelet domain. Wavelets, specifically Framelets and Curvelets, are better suited for representing edges and directional textures. Two specific architectures were developed:
- Weighted Wavelet Burst Accumulation (WWBA): Weights are computed and applied directly to wavelet coefficients.
- Weighted Wavelet Fourier Burst Accumulation (WWFBA): A hybrid approach where FBA is applied to individual wavelet subbands.
3. Sparse Burst Accumulation (SDBA)
Moving beyond simple weighted averaging, the authors introduce a sparsity-promoting approach. By applying L1-norm soft-thresholding, the algorithm identifies and retains only the most dominant coefficients across the frame sequence. This effectively discards noise and blur-induced artifacts, leading to a reconstruction that is often sharper and higher in contrast than traditional weighted methods.
Experimental Setup and Results
The algorithms were tested using 50-frame sequences from the "Barchart" datasets, which include real-world data from field trials in New Mexico. These sequences represent a "torture test" for image restoration due to varying levels of heat shimmer and atmospheric instability.
Key Findings
- The Necessity of Registration: Experiments confirmed that non-rigid registration is non-negotiable. Without it, even the most advanced deblurring algorithms fail to produce a stable image because they cannot account for the underlying geometric warping.
- Sparsity vs. Weighting: The Sparse Burst Accumulation (SDBA) method frequently outperformed traditional weighting. By keeping only the "strongest" parts of each frame, the algorithm naturally enhances contrast and edge definition.
- Curvelet Efficacy: Curvelets proved particularly effective for man-made targets. Their design allows for superior representation of curved edges and directional structures compared to standard wavelets.
- Computational Efficiency: While the original FBA remains the fastest (~2.2 seconds for 50 frames), the Curvelet-based Sparse (C-SWBA) method provides a high-quality result in approximately 4.5 seconds. This makes it a viable candidate for near-real-time applications, whereas Framelet-based methods (Fr-WWBA) were significantly slower due to unoptimized code structures.
Discussion and Future Work
The study identifies a stable parameter set for these methods, finding that a sparsity threshold of $\lambda = 0.001$ and a weighting power of $p = 11$ provide an excellent balance between detail retention and noise suppression. However, the authors note that the optimal value of $p$ currently lacks a rigorous theoretical proof.
Future research will focus on optimizing the computational overhead of Framelet-based methods through pre-compiled functions and exploring more complex learned representations. The shift from simple averaging to more sophisticated operator selection—such as those found in deep learning—remains a promising frontier for further mitigating the effects of atmospheric turbulence.
Summary updated