Images are traditionally represented as grids of spatial pixels. While intuitive, this spatial representation couples two fundamentally distinct types of visual information: global geometric structure and fine-grained local texture. When a neural network processes raw pixels, it must simultaneously untangle macro-level shapes from micro-level noise, lighting variations, or environmental degradations.
Wavelet-based frequency decomposition acts as a mathematical prism to solve this problem. By applying a Discrete Wavelet Transform (DWT)—frequently utilizing Haar filters—an image is cleanly separated into distinct frequency sub-bands. The low-frequency components capture the smooth, global structural layout (the "approximation"), while the high-frequency components isolate sharp transitions, fine textures, and edge details along horizontal, vertical, and diagonal axes. This explicit separation allows machine learning models to process, manipulate, or protect different visual characteristics independently, leading to significant gains in robustness, efficiency, and restoration fidelity.
Decoupling Environmental Degradations from Scene Content
In real-world computer vision, adverse weather and lighting conditions pose a severe challenge because they corrupt images unevenly across the frequency spectrum. As noted by Dengxin Dai et al. in Guest Editorial: Special Issue on "Computer Vision for All Seasons: Adverse Weather and Lighting Conditions", analyzing images in the frequency domain allows models to isolate environmental noise from underlying scene content. Different degradations target different frequency bands:
- Rain and Noise: Rain streaks manifest as sharp, high-frequency directional edges. In Deep Leaning Based Frequency-Aware Single Image Deraining by Extracting Knowledge from Rain and Background, Ye Xiong et al. exploit this by treating low-frequency sub-bands as pure background information, while dedicating specialized neural branches to strip rain streaks out of the high-frequency bands without destroying fine background details.
- Haze and Fog: Conversely, haze acts as a low-frequency atmospheric blanket that alters global contrast and occludes object semantics. Jie Sun et al. leverage this insight in WDMamba: When Wavelet Degradation Prior Meets Vision Mamba for Image Dehazing to split image dehazing into a coarse-to-fine process, prioritizing global structure restoration in the low-frequency components where haze degradation is most concentrated. Similarly, Qi Bi et al. demonstrate in Learning Generalized Segmentation for Foggy-Scenes by Bi-directional Wavelet Guidance that separating object semantics from foggy noise allows for robust semantic segmentation in adverse weather.
- Illumination and Contrast: Low-light conditions primarily suppress low-frequency luminance and contrast. In LoLI-Street: Benchmarking Low-Light Image Enhancement and Beyond, Md Tanvir Islam et al. decompose low-light images to process global illumination and fine details separately. Earlier classical approaches, such as the work by Jianye Peng in A Histogram Equalization Algorithm about Second Generation Wavelet Integer Transform, targeted only the low-frequency coefficients for histogram equalization, boosting structural visibility while naturally preventing the amplification of high-frequency sensor noise. In modern architectures, the authors of From Darkness to Detail: Frequency-Aware SSMs for Low-Light Vision show that decoupling frequency components allows State Space Models (SSMs) to enhance contrast and illumination rapidly without introducing spatial artifacts.
Accelerating and Refining Generative Models
Generative models like Diffusion Models and GANs often struggle with high-frequency details, exhibiting a "low-frequency bias" that leads to blurry outputs or identity loss in face restoration. Moreover, running diffusion steps across full-resolution pixel spaces is computationally prohibitive. Wavelet decomposition provides an elegant solution to both efficiency and fidelity.
Targeted Restoration Guidance
In blind face restoration, Jiankang Deng et al. introduce WaveFace: Authentic Face Restoration with Efficient Frequency Recovery, which shifts the restoration process into the wavelet domain to handle structural recovery and fine-grained texture separately. This achieves a 10x acceleration while preserving facial identity. For general blind image restoration, Jun Xiao 0010 et al. develop Frequency-Aware Guidance for Blind Image Restoration via Diffusion Models, using a 2D DWT to isolate low-frequency shapes from high-frequency directional details, providing targeted guidance to diffusion models during the denoising process.
Computational Efficiency and Compression
Instead of running expensive diffusion steps across all pixels, Ge Meng et al. propose Accelerated Diffusion via High-Low Frequency Decomposition for Pan-Sharpening, which applies the diffusion process exclusively to low-frequency wavelet coefficients, using a lighter, dedicated module to restore high-frequency spatial textures. This dramatically reduces inference time and memory usage.
Input Image ───► Discrete Wavelet Transform (DWT)
│
├──► Low-Frequency Sub-band ──► Heavy Generative Modeling (e.g., Diffusion)
│
└──► High-Frequency Sub-bands ──► Light Texture/Detail Restoration
When compressing diffusion models via post-training quantization, Cuong Pham et al. show in Beyond Uniformity: Sample and Frequency Meta Weighting for Post-Training Quantization of Diffusion Models that decomposing calibration images allows the quantization process to apply distinct weights to different frequency sub-bands based on their reconstruction importance at various denoising stages. Meanwhile, in GAN inversion, Chaewon Kim et al. introduce WINE: Wavelet-Guided GAN Inversion and Editing for High-Fidelity Refinement, utilizing a wavelet loss and fusion technique to explicitly force the network to reconstruct high-frequency details that standard pixel-wise losses ignore.
Correcting Inductive Biases in Recognition
Standard deep neural networks suffer from an unintended inductive bias: they tend to overfit to high-frequency local textures rather than learning robust, low-frequency global structures. This shortcut learning causes models to fail when encountering out-of-distribution data or subtle category shifts. By explicitly separating these frequencies, researchers can manipulate what the network learns.
In open-set recognition, when presented with unknown classes, networks often make overconfident predictions based purely on familiar high-frequency textures. To combat this, Lijun Liu et al. develop Frequency Shuffling and Enhancement for Open Set Recognition. Their framework shuffles high-frequency details while enhancing low-frequency global structures, forcing the model to rely on robust semantic shapes and significantly improving out-of-distribution detection.
Similarly, in tasks requiring highly specific classification under concept drift, Shuo Ye et al. present Concept Drift and Long-Tailed Distribution in Fine-Grained Visual Categorization: Benchmark and Method, which decomposes images into low-frequency structural and high-frequency texture components via Haar wavelets. This allows for targeted data augmentation on specific frequency bands, preventing the network from overfitting to transient texture features.
For multi-source image fusion in high-accuracy mapping applications, Chunxiang Wang et al. demonstrate in A New Approach to High-accuracy Road Orthophoto Mapping Based on Wavelet Transform that wavelet transforms can merge overlapping visual data by applying frequency-specific filtering to maintain sharp road markings (high frequency) while smoothing out background sensor noise. In remote sensing imagery, Shuang Xu et al. introduce Haar Nuclear Norms With Applications to Remote Sensing Imagery Restoration to perform low-rank modeling on approximation and detail coefficients simultaneously, preserving both coarse structures and fine textures.
Key Foundational Work
An Uneven Illumination Correction Algorithm for Optical Remote Sensing Images Covered with Thin Clouds (2015) — contributed 2 concept elements to this direction.
A New Approach to High-accuracy Road Orthophoto Mapping Based on Wavelet Transform (2011) — contributed 1 concept elements to this direction.