Abstract
This paper introduces efficient Projected Gradient Descent (PGD) algorithms for signal recovery from noisy, nonlinear measurements by leveraging deep generative models as priors. We provide the first theoretical guarantees for linear convergence to optimal statistical error rates in this setting, despite the non-convexity of the generative prior. Empirical results on MNIST and CelebA datasets demonstrate that our PGD-based approach significantly outperforms traditional sparsity-based methods and existing generative prior techniques.
1. Introduction
Inverse problems involve reconstructing an original signal $x \in \mathbb{R}^p$ from a set of $n$ measurements $y$. Traditional Compressed Sensing (CS) relies on the assumption that $x$ is sparse in some basis. However, CS faces challenges when dealing with nonlinear measurement processes and when more complex priors are required to capture the structure of high-dimensional data like natural images. Deep generative models, such as Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs), have emerged as powerful alternatives to sparsity priors. This work bridges the gap between these advanced priors and nonlinear measurement models.
2. Methodology
We propose two primary algorithms based on Projected Gradient Descent (PGD). Unlike previous methods that optimize within the latent space of the generator, our approach operates in the ambient signal space and periodically projects the estimate back onto the range of the generative model.
2.1 PGD-GLasso (Unknown Nonlinearity)
In scenarios where the specific nonlinear link function $f$ is unknown, PGD-GLasso utilizes a linear least-squares loss. Theoretical analysis shows that for certain sensing distributions (e.g., Gaussian), the linear estimator can recover the direction of the signal, treating the nonlinearity as a form of unconventional noise.
2.2 PGD-NLasso (Known Nonlinearity)
When the nonlinear function $f$ is known and monotonic, PGD-NLasso employs a nonlinear least-squares loss to achieve higher precision.
3. Theoretical Framework
The primary challenge in analyzing these algorithms is the non-convexity of the range of a neural network. We utilize a "Set-Restricted Eigenvalue Condition" (TS-REC) to prove that the sensing matrix behaves predictably on the subset of signals produced by the generative model. This allows us to guarantee linear convergence to the optimal statistical error rate $O(\sqrt{k \log L / n})$, where $k$ is the latent dimension and $L$ is the Lipschitz constant of the generator.
4. Experimental Results
The proposed algorithms were evaluated on the MNIST and CelebA datasets using pre-trained VAE and DCGAN models, respectively.
As shown in Figure 1, the PGD-based methods produce high-quality reconstructions even at low measurement rates where traditional Lasso fails. Our methods also demonstrate greater robustness to initialization compared to the CSGM baseline.
Quantitative analysis (Figure 2) confirms that the PGD algorithms achieve significantly higher cosine similarity with fewer measurements than Lasso. The results also validate the theoretical prediction of linear convergence, with the reconstruction error decreasing exponentially during initial iterations.
5. Discussion and Limitations
While the PGD framework offers substantial improvements, the computational cost of the projection step $P_K$ remains a bottleneck, as it requires solving a non-convex optimization problem. Additionally, current theoretical guarantees are primarily established for Gaussian sensing matrices. Future research will focus on extending these results to more structured measurement matrices and improving the efficiency of the projection operator.
6. Conclusion
This paper establishes a rigorous algorithmic and theoretical foundation for solving nonlinear inverse problems with generative priors. By shifting the optimization to the ambient space and utilizing PGD, we achieve fast convergence and superior reconstruction performance, paving the way for more robust signal processing in complex, nonlinear environments.
Summary updated