Abstract
VibrantLeaves (VL) is a novel synthetic image generator designed to train deep learning models for image restoration tasks without relying on real-world photography. By mathematically modeling complex geometry, multi-scale textures, and physical depth, VL generates abstract training data that enables models to achieve performance comparable to those trained on natural datasets while offering improved robustness and interpretability.
Introduction
Deep Neural Networks (DNNs) are the state-of-the-art for image restoration tasks like denoising, deblurring, and super-resolution. However, they typically require massive datasets of natural images, which are often biased, contain private information, and act as "black boxes." Previous attempts to use synthetic data, such as the "Dead Leaves" model (layering simple disks), resulted in a significant performance gap because they failed to capture the complex textures and depth cues of the real world.
VibrantLeaves addresses this by asking: Can we build a parametric generator that captures the essential statistical "DNA" of natural images well enough to replace them entirely during training?
Methodology
VibrantLeaves builds upon the classical Dead Leaves framework—an occlusion-based model where shapes are "dropped" onto a canvas—but introduces three major principled upgrades:
1. Complex Geometry
Instead of simple circles, VL uses concave hulls (or $\alpha$-shapes). By sampling random points and removing specific triangles from their Delaunay triangulation, the generator creates shapes with arbitrary curvatures, sharp corners, and holes, mimicking the structural variety of real objects.
2. Two-Scale Textures
VL incorporates a parametric texture generator that combines:
- Pseudo-periodic patterns: Using warped sinusoids to create structures like bricks, fur, or grids.
- Micro-textures: Using Gaussian noise with a power spectrum that decays at a $1/|\nu|^\gamma$ rate, a statistical property of natural scenes representing fine details like sand or grass.
3. Physical Depth
VL simulates a 3D-like environment by stacking foreground, middle, and background layers. It applies a depth-of-field simulator with varying blur kernels and perspective tilting to recreate the vanishing points and focal blurs encountered in real photography.
Key Results and Insights
- Closing the Performance Gap: For image denoising, models trained on VibrantLeaves came within 0.72 dB PSNR of models trained on real natural images, a massive improvement over the ~2.2 dB gap seen with previous synthetic models.
- Statistics Over Realism: A striking insight is that a network doesn't need to see semantic objects (like cats or trees) to learn restoration. It only needs data sharing the same second-order statistics (gradient distributions and power spectra) as natural images.
- The Power of Invariance: Because the VL generator is mathematically defined to be invariant to rotation and scaling, the resulting networks are naturally more robust to tilted or zoomed images than those trained on natural datasets, which often have a horizontal/vertical bias.
- Plug-and-Play (PnP) Effectiveness: VL-trained denoisers proved highly effective as "priors" for other tasks like deblurring and inpainting, demonstrating a high-quality learned representation of clean image structures.
Limitations and Future Work
VL still struggles slightly with the Urban100 dataset, which contains perfectly repetitive, rigid architectural patterns that VL's current random distortions do not perfectly match. Future work may involve "back-propagating" through the generator's parameters to automatically find the optimal synthetic dataset for specific restoration tasks, potentially closing the final performance gap.
Summary updated