WaveletLoRA: Frequency-Aware Content-Style Decomposition for Personalized Image Generation

Peiyao Wang, Jiahui Sun, Weining Wang, Jing LiuarXiv 2026 · arXiv

TL;DR

WaveletLoRA is a frequency-aware framework that achieves state-of-the-art content-style disentanglement from a single reference image. By leveraging the discrete wavelet transform (DWT) to assign dedicated LoRA branches to different frequency subbands, the model cleanly separates structural identity from artistic style, enabling high-fidelity personalized image generation.

What Problem Does This Paper Address?

Personalized image generation aims to capture specific concepts—like a particular pet or a unique art style—from reference images to create new content. While methods like DreamBooth or StyleDrop are effective, they typically require multiple images or focus on either content or style in isolation.

Disentangling both factors simultaneously from a single image remains a "holy grail" challenge. Existing spatial-domain methods often suffer from "style leakage" (where the style bleeds into content-only generations) or "subject residue" (where the original subject's shape haunts a style transfer). The authors argue this happens because these methods ignore a fundamental signal-processing truth: content and style naturally live in different frequency bands.

Methods

The core innovation of WaveletLoRA is using frequency decomposition as a principled inductive bias for disentanglement.

  1. Wavelet-based Decomposition: The researchers apply a two-level Discrete Wavelet Transform (DWT) to the reference image. This breaks the image into one low-frequency subband ($LL_2$), which captures global appearance and color (style), and six high-frequency subbands that capture edges, textures, and structural details (content).
  2. Frequency-Specific LoRA Branches: Instead of training a single LoRA, WaveletLoRA instantiates seven independent branches. The low-frequency branch is optimized for style, while the high-frequency branches focus on content.
  3. Dense Mixture-of-Experts (MoE): To integrate the high-frequency information, the authors use a dense MoE module. A gating network dynamically weights the different high-frequency experts based on the input, allowing the model to prioritize specific structural cues (like horizontal edges or fine textures) depending on the subject.
  4. Disentanglement Objectives: The training is guided by a composite loss function:
  5. Weight-level frequency separation: Penalizes high-frequency energy in style weights and low-frequency energy in content weights.
  6. AdaIN-based style alignment: Ensures the generated style matches the global statistics (color palette, contrast) of the reference.
  7. Orthogonality regularization: Encourages the content and style LoRA masks to be column-wise orthogonal to prevent overlap.
Figure 2: Overview of WaveletLoRA and WaveBench. (Top) Given a reference image, a two-level DWT decomposes the image into one low-frequency subband and multiple high-frequency subbands. The low-frequency branch learns global appearance attributes for style modeling, while the high- frequency branches preserve multi-scale structural details for content representation. An adaptive aggregation MoE module fuses high-frequency representations into a unified content LoRA, followed by frequency-domain disentanglement supervision and AdaIN-based style alignment. (Bottom) Construction pipeline of WaveBench, including prompt generation, manual verification, and reference image synthesis.

Datasets

The authors introduced WaveBench, a large-scale benchmark specifically designed for single-image content-style decomposition.

  • Scale: 1,000 diverse examples spanning animals, humans, objects, and scenery.
  • Diversity: Includes styles ranging from traditional Chinese painting to futuristic digital art.
  • Construction: Prompts were expanded using ChatGPT-4o, manually filtered for quality, and reference images were synthesized using Gemini-3 Pro to ensure high fidelity.
Figure 4: Diversity of WaveBench.

Key Insights and Ideas

  • Frequency as a Natural Prior: Unlike Fourier transforms, which provide global spectral data, wavelets offer localized multi-scale decomposition. This makes them uniquely suited for separating "what" an object is (high-freq structure) from "how" it looks (low-freq style).
  • The MoE Advantage: The visualization of MoE gating weights (Figure 7 in the paper) reveals that the model adaptively chooses experts. For a pencil sketch, it prioritizes edge-heavy subbands; for pixel art, it triggers diagonal-heavy subbands. This avoids "representation collapse" where the model treats all structural details the same.
  • VLM-based Evaluation: The authors found that standard metrics like DINOv2 can be misleading because they measure holistic similarity rather than purity of separation. They developed a VLM-based protocol (using Qwen2.5-VL) that explicitly rates "Separation" and "Similarity," which correlates much more closely with human judgment.

Key Results

WaveletLoRA consistently outperformed existing SOTA methods like B-LoRA, UnZipLoRA, and USO.

  • Quantitative: On WaveBench, WaveletLoRA achieved the highest VLM-based scores for both content preservation (0.8563) and style fidelity (0.8730).
  • Qualitative: As shown in the comparison figures, WaveletLoRA produces much "cleaner" results. For instance, when generating a subject in a new context, it successfully strips away the original artistic medium (like watercolor textures) that other models fail to remove.
  • User Preference: In a user study, WaveletLoRA's combined score (0.7750) significantly surpassed the next best specialized method, USO (0.5825).
Figure 1: Given a single input image (middle), WaveletLoRA learns a disentangled content or subject LoRA (left) and style LoRA (right) that can be used to generate new images with the learned concepts.

Limitations and Future Work

  • Computational Overhead: Training seven LoRA branches and an MoE module increases the parameter count and training time. WaveletLoRA takes ~320 minutes to train for 600 steps, compared to ~230 minutes for a standard dual-LoRA setup.
  • Inference Speed: Inference is slightly slower (~10s vs ~6s per image) due to the additional frequency-specific processing.
  • Future Directions: The authors suggest exploring even deeper wavelet decompositions or applying this frequency-aware logic to video generation, where temporal consistency adds another layer of complexity to content-style separation.

Summary updated

Abstract

Content-style decomposition from a single image is a fundamental challenge in personalized image generation, aiming to separate subject identity from visual style for flexible recomposition. Existing approaches primarily operate in the spatial domain, without explicitly exploiting the distinct frequency characteristics naturally associated with content and style. In this paper, we propose WaveletLoRA, a novel frequency-aware framework for explicit content-style decomposition through wavelet-domain supervision. Specifically, we first apply a multi-level discrete wavelet transform (DWT) to decompose the reference image into low-frequency and high-frequency subbands. We then assign a dedicated LoRA branch to each subband, where the low-frequency branch models global appearance for style acquisition, while the high-frequency branches capture structural details for content preservation. To further improve disentanglement quality, we introduce a dense Mixture-of-Experts aggregation module together with a frequency-domain regularization objective, which reduces style leakage and enhance content fidelity. In addition, we establish WaveBench and develop a VLM-based evaluation protocol that explicitly measures both attribute similarity and attribute separation for disentanglement assessment. Extensive experiments demonstrate that WaveletLoRA consistently outperforms existing methods and produces results that better align with human preference.
BibTeX
@misc{wang2026waveletlora,
  author = {Peiyao Wang and Jiahui Sun and Weining Wang and Jing Liu},
  title = {{WaveletLoRA: Frequency-Aware Content-Style Decomposition for Personalized Image Generation}},
  year = {2026},
  eprint = {2611.15617},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.15617},
}
Figure