An Integrated Framework of Frequency-Domain Denoising with Learnable Parameters in Variational Autoencoders

Hui Li, Hongtian Zhao, Peng LiResearch Square 2026 · DOI

TL;DR

This paper introduces Learnable FFT-VAE, a framework that embeds a differentiable, frequency-domain denoising module directly into the Variational Autoencoder pipeline. By using a dedicated sub-network to predict per-pixel filtering parameters, the model learns to dynamically suppress noise while preserving sharp structural details, outperforming both standard VAEs and fixed-filter preprocessing.

What Problem Does This Paper Address?

Variational Autoencoders (VAEs) are powerful tools for learning compressed representations of data, but they have a "noise problem." When input images are corrupted by noise, VAEs often struggle to distinguish between meaningful features and random artifacts.

Traditional solutions usually fall into two camps:

  1. Spatial Denoising: Using filters like Gaussian blur, which often destroys sharp edges and fine textures.
  2. Fixed Frequency Filtering: Using the Fast Fourier Transform (FFT) to remove high-frequency noise. However, because these filters are usually "one-size-fits-all," they fail to adapt to images where noise levels vary across different regions (spatially heterogeneous noise).

The authors aim to bridge this gap by creating a denoising mechanism that is both frequency-aware and data-adaptive.

Methods

The core innovation is the integration of three primary modules into an end-to-end trainable architecture:

  1. Hyperparameter Learning Net (HLN): A CNN that takes the noisy image and predicts three per-pixel parameters:
  2. $\alpha$ (Alpha): Controls the blending of high-frequency residuals.
  3. $\sigma$ (Sigma): Determines the bandwidth of the local low-pass filter.
  4. $m$ (Mask): Acts as a gate for high-frequency components, representing spatial confidence.
  5. Denoising-Correction Module (DCM): This module performs the heavy lifting in the spectral domain. It transforms the image via FFT, applies masks generated by the learned $\sigma$, and uses $\alpha$ and $m$ to selectively re-incorporate important high-frequency details (like edges) while discarding noise.
  6. VAE Backbone: A standard encoder-decoder structure that operates on the denoised output of the DCM.

Because the entire pipeline is differentiable, the VAE’s reconstruction loss backpropagates through the FFT operations to "teach" the HLN how to filter more effectively.

Datasets

The authors evaluated their framework on two main datasets:

  • FashionMNIST: Used for controlled testing. The researchers added synthetic Gaussian noise at varying levels (standard deviations from 0.1 to 0.4) to create a benchmark for reconstruction fidelity.
  • MedMNIST (pneumoniamnis subset): A real-world dataset of chest X-ray images. This dataset is significantly more complex due to inherent medical imaging artifacts and the critical need to preserve fine anatomical structures for diagnosis.

Key Insights and Ideas

  • Denoising as a Learnable Pre-layer: Instead of treating denoising as a separate preprocessing step, the authors treat it as the "first layer" of the VAE. This ensures the latent space is built on "clean" features rather than noise-polluted ones.
  • The Power of Per-Pixel Parameters: By predicting $\alpha, \sigma,$ and $m$ for every pixel, the model can be "aggressive" with filtering in flat, noisy areas (like backgrounds) while being "conservative" near edges and textures to prevent blurring.
  • Interpretability: The learned parameter maps act as a form of "attention." Visualizing the $\sigma$ map, for instance, reveals exactly which parts of the image the model considers "high-detail" versus "noisy."

Key Results

  • Quantitative Superiority: On FashionMNIST with a noise level of 0.25, the Learnable FFT-VAE achieved a PSNR of 17.25 dB and SSIM of 0.6152, outperforming the "Direct VAE" (no denoising) and significantly beating the "Fixed FFT-VAE" (which suffered from over-smoothing).
  • Robustness to Noise Intensity: As Gaussian noise increased from 0.1 to 0.4, the learnable framework maintained a higher Structural Similarity Index (SSIM) than baselines, proving its adaptability.
  • Medical Utility: In the MedMNIST tests, the model successfully preserved fine lung textures and rib edges in X-rays that were blurred or distorted by standard VAE approaches. This suggests the method is viable for downstream diagnostic tasks where detail is paramount.

Limitations and Future Work

  • Noise Type Specificity: The study primarily focuses on Gaussian noise. It remains to be seen how well this frequency-domain approach handles non-additive noise, such as Poisson noise or adversarial perturbations.
  • Computational Complexity: While FFTs are efficient ($O(N \log N)$), adding an HLN and FFT/IFFT steps to every forward pass increases the computational overhead compared to a vanilla VAE.
  • Hyperparameter Regularization: The authors noted the need for a specific regularization term ($L_{reg}$) to keep the learned parameters stable. Tuning this regularization itself might require domain expertise, potentially re-introducing a bit of the manual tuning the paper seeks to avoid.

Summary updated

Abstract

<title>Abstract</title> Variational Autoencoders (VAEs) have demonstrated substantial utility in generative modeling, however their performance is often compromised when inputs are corrupted by noise or measurement artifacts. Traditional approaches to preprocessing either fail to adapt to spatially heterogeneous noise or result in excessive smoothing of high-frequency features. In this study, we propose an automatic learnable FFT-VAE framework that integrates learnable frequency-domain denoising and correction parameters with generative modeling. The method predicts per-pixel parameters (\((\alpha)\),\((\sigma)\) , and\((m)\)), which dynamically control the contribution of high-frequency components, the bandwidth of the low-pass filter, and spatial confidence, respectively. This adaptive approach allows the network to suppress noise while preserving structural details in a data-driven manner, eliminating the need for manual hyperparameter tuning. We tested the method on synthetic FashionMNIST datasets under multiple Gaussian noise levels and demonstrate the efficacy of the proposed framework. Compared to baseline configurations, the automatic learnable FFT-VAE method achieves higher reconstruction fidelity, improved PSNR, SSIM, and SNR, and produces more coherent latent representations. We also tested the method in the MedMNIST dataset as a real-world application. The results indicate that the method not only enhances denoising performance but also facilitates the extraction of meaningful features for downstream analytical or diagnostic tasks.
BibTeX
@inproceedings{li2026integrated,
  author = {Hui Li and Hongtian Zhao and Peng Li},
  title = {{An Integrated Framework of Frequency-Domain Denoising with Learnable Parameters in Variational Autoencoders}},
  booktitle = {Research Square},
  year = {2026},
  doi = {10.21203/rs.3.rs-8209010/v1},
}
Figure