Abstract
Inspired by the recent success of deep neural networks and efforts to develop multi-layer dictionary models, we propose a Deep Analysis dictionary Model (DeepAM) optimized for single image super-resolution. Unlike other multi-layer models, our architecture contains $L$ layers of analysis dictionaries and soft-thresholding operators to extract high-level features, followed by a synthesis dictionary layer for regression. Each analysis dictionary is partitioned into an Information Preserving Analysis Dictionary (IPAD) and a Clustering Analysis Dictionary (CAD). The IPAD passes key information to the next layer, while the CAD produces sparse feature representations. DeepAM utilizes both supervised and unsupervised learning. Results show DeepAM outperforms comparable DNNs on small training datasets and adapts to unseen noise levels via simple threshold rescaling.
Introduction
Single Image Super-Resolution (SISR) is a fundamental task in computer vision. While Deep Neural Networks (DNNs) have achieved state-of-the-art results, they often function as "black boxes" requiring vast amounts of training data. Moreover, DNNs are typically sensitive to noise levels; a model trained on clean data often fails when presented with noisy test images, necessitating costly retraining. This paper introduces DeepAM, an interpretable, data-efficient, and noise-robust alternative that leverages the principles of sparse representation within a deep architecture.
The DeepAM Architecture
DeepAM is structured as an $L$-layer deep analysis dictionary model. Each of the first $L$ layers performs a forward pass consisting of an analysis dictionary multiplication followed by a soft-thresholding operator. This sequence gradually extracts high-level features from the input low-resolution (LR) image. The final layer is a synthesis dictionary ($D$) designed to reconstruct the high-resolution (HR) output ($\hat{y}$).
Unlike synthesis-based dictionary models that require iterative sparse pursuit during inference, DeepAM's analysis-based approach allows for efficient computation via simple matrix multiplications and thresholding. This makes the model significantly faster during both training and deployment.
Dictionary Partitioning: IPAD and CAD
A key innovation of DeepAM is the partitioning of each analysis dictionary $\Omega$ into two functional components: the Information Preserving Analysis Dictionary (IPAD) and the Clustering Analysis Dictionary (CAD).
- IPAD ($\Omega_I$): Learned using unsupervised methods, this component ensures that the essential signal subspace of the input is preserved and passed to the subsequent layer.
- CAD ($\Omega_C$): Learned through supervised methods, this component focuses on identifying discriminative high-frequency features and residuals that are crucial for reconstruction but often lost during downsampling.
This hybrid learning strategy allows the model to maintain a clear functional role for different "neurons," providing a level of interpretability often missing in standard deep learning models.
Experimental Results
The model was evaluated on standard benchmarks such as Set5 and Set14. DeepAM demonstrates a significant advantage in small-data regimes. When trained on only 5,000 patches, DeepAM achieved a PSNR of 35.96 dB on Set5, outperforming comparable DNN architectures with ReLU (35.06 dB) or soft-thresholding (35.62 dB).
Furthermore, DeepAM exhibits unique robustness to noise. Because the soft-thresholding parameters are tied to signal and noise variance, a pre-trained DeepAM can be adapted to new noise levels simply by mathematically rescaling the thresholds of the first layer. In tests with varying noise levels, this rescaled DeepAM maintained high performance, whereas standard DNNs saw their PSNR drop significantly due to the mismatch between training and testing noise.
Conclusion
DeepAM bridges the gap between the interpretability of sparse coding and the representational power of deep learning. Its ability to perform well with limited data and its inherent flexibility in handling noise make it a compelling alternative to standard black-box DNNs for image restoration. Future research will explore extending this framework to fully convolutional architectures and applying it to other domains like medical imaging (CT/MRI), where data efficiency and noise robustness are critical.
Summary updated