Abstract: Generative models like diffusion and flow matching traditionally rely on iterative numerical integration, creating a massive computational bottleneck. Primal-Spectral Generative Modeling introduces a novel analytical solver, the Pseudoinverse Lévy Inversion (PiLI), which replaces iterative ODE/SDE integration with a single-step, closed-form solution. By training in the spectral domain using Characteristic Functions, the proposed PriSpecNet achieves a 170× reduction in Gflops on ImageNet-256 while maintaining state-of-the-art generation quality and forecasting accuracy.
1. The Challenge: Iterative Bottlenecks and Topological Blindness
Most modern generative models operate in the "primal" Euclidean space, learning to transport a simple noise distribution to a complex data distribution by following a vector field. This standard approach suffers from two major drawbacks:
- The Iterative Bottleneck: Generating a single sample requires dozens or hundreds of neural function evaluations (NFEs) to numerically integrate an ODE or SDE. This makes them prohibitively slow for real-time applications.
- Topological Blindness: Because these models learn from sparse samples, they often struggle to capture the global topology of continuous manifolds. In low-density regions, gradients can vanish, leading to "local stagnation" during training and poor mode coverage.
2. Methodology: Bridging Primal and Spectral Domains
To overcome these limitations, the authors propose the Primal-Spectral Generative Model, which bridges the gap between primal state space and the spectral domain.
Forward Spectral Modulation
Instead of operating solely on raw data points, the method maps data into the spectral domain by evaluating the Characteristic Function (CF)—essentially a complex exponential basis. This transforms the linear interpolation of data into a multiplicative phase modulation in the spectral domain.
Reverse Spectral Reconstruction
The authors prove that minimizing the difference between the learned and true spectral densities effectively "squeezes" the Fisher divergence bounds. This provides persistent gradient signals across the entire domain, even where data is sparse, preventing the vanishing gradient problem seen in primal-only models.
The PiLI Solver: 1-NFE Generation
The core breakthrough is the Pseudoinverse Lévy Inversion (PiLI) solver. Rather than using an ODE solver to step through time, PiLI formulates the recovery of the data ($x_0$) as a weighted least squares problem. By applying Tikhonov regularization to filter high-frequency noise, the authors derive a closed-form analytical inversion. This enables 1-NFE generation: a single pass through the network predicts the phase, followed by a fast matrix operation to recover the pixels.
PriSpecNet Architecture
To implement this, the authors design PriSpecNet, a hybrid architecture featuring a primal backbone for local feature extraction fused with a complex-valued spectral decoder to capture global topology.
3. Key Insights and Theoretical Guarantees
- Algebraic vs. Temporal Generation: The fundamental shift in this work is moving from temporal integration (moving along a path over time) to algebraic inversion (solving a system of equations). This paradigm shift is what enables the massive jump in efficiency.
- Persistent Gradients & Phase Consistency: In the primal domain, gradients vanish in the "empty" spaces between data clusters. The CF provides a continuous global representation, ensuring valid guidance throughout the entire domain. The spectral approach preserves phase information across the continuous surface, acting as a robust "information buffer" against sparse sampling.
- Theoretical Grounding: The method is rigorously grounded in Lévy’s Continuity Theorem, guaranteeing that minimizing spectral error leads to convergence to the true data distribution.
4. Experimental Results
The model was rigorously evaluated across synthetic, time series, and image datasets, demonstrating exceptional performance and efficiency.
- ImageNet $256 \times 256$ Efficiency: The 1-NFE PiLI solver requires only 26 Gflops, compared to the 4,436 Gflops required by a 25-NFE DPM-Solver++. This represents a staggering 170× improvement in computational cost.
- Image Quality: Despite the massive speedup, PiLI-XL achieves a highly competitive FID of 1.66, outperforming previous training-from-scratch baselines like iMF (1.72) and various distillation-based methods.
- Time Series State-of-the-Art: PriSpecNet unifies generation and forecasting, establishing new SOTA results. It reduced Context-FID by 50.0%, 41.5%, 80.6%, and 63.1% on Sines, Solar, ETTh, and Stock benchmarks, respectively. It also decreased forecasting Mean Squared Error (MSE) by 29.8% on Solar and 23.8% on Stock.
- Synthetic GMMs: On Gaussian Mixture Models (1, 3, and 5 modes), the spectral method resolved complex multimodal distributions in just 50 epochs, whereas primal-only methods exhibited significant over-smoothing even after 200 epochs.
5. Limitations and Future Directions
While highly effective, the authors note a few constraints and areas for future exploration:
- Spectral Bottlenecks: The choice of the spectral dimension $K$ is critical. A $K$ that is too small lacks expressivity, while a $K$ that is too large can slow down training convergence.
- Phase Wrapping: To avoid numerical instability, data must be normalized to a compact support (e.g., $[-1, 1]$). While standard in many ML tasks, it remains a strict requirement for the PiLI solver's stability.
- Primal Redundancy: Currently, PriSpecNet maintains both primal and spectral paths. Future architectures could potentially be "Spectral-Only," eliminating primal redundancy for even greater computational efficiency.
Summary updated