Abstract
Single Image Super-Resolution (SISR) remains a challenging ill-posed problem in computer vision. While deep learning has significantly advanced the field, existing models often struggle with high computational costs and optimization difficulties. This paper introduces the Expectation-Maximization Attention Cross Residual Network (EACRN), which utilizes a hierarchical "cross-residual" architecture and an iterative Expectation-Maximization (EM) attention mechanism. EACRN efficiently captures long-range dependencies and multi-scale features, achieving state-of-the-art accuracy with a fraction of the training updates required by previous models.
Introduction
The goal of SISR is to reconstruct a high-resolution (HR) image from a single low-resolution (LR) input. Deep Convolutional Neural Networks (CNNs) have become the standard approach, but they face two primary hurdles:
- Optimization Complexity: Increasing network depth often leads to vanishing gradients, making training difficult.
- Computational Inefficiency: Global context is typically captured via self-attention, which has quadratic complexity $O(N^2)$ relative to pixel count, leading to high memory and compute demands.
Proposed Method: EACRN
The EACRN architecture addresses these challenges through three key innovations:
Cross Residual in Cross Residual (CRICR)
Instead of standard linear residual connections, EACRN employs a hierarchical structure. Multiple Cross Residual Groups (CRG) are linked via global skip connections. Within each CRG, Cross Residual Blocks (CRB) utilize short skip connections. This design creates "highways" for low-frequency information, allowing the network to focus its learning capacity on high-frequency details like edges and textures.
Multi-Scale Feature Extraction
To capture patterns of varying sizes, the residual blocks incorporate convolutions with different kernel sizes (e.g., $3\times3$ and $5\times5$). This allows the model to adaptively process features at multiple scales simultaneously.
Expectation-Maximization Attention (EMA)
The core innovation is the EMA module. Unlike standard self-attention that computes a full pixel-to-pixel similarity matrix, EMA treats attention as an iterative estimation problem. It uses the EM algorithm to find a compact set of representative "bases." This approach is robust to input variance and significantly reduces the computational burden, making it highly efficient for GPU processing.
Experimental Results
The model was evaluated on standard benchmarks including Set5, Set14, BSD100, and Urban100, using the DIV2K dataset for training.
Quantitative and Qualitative Performance
EACRN consistently outperformed established models like RCAN, EDSR, and SAN. On the Set5 benchmark ($2\times$ scale), EACRN achieved a PSNR of 38.42 dB, surpassing RCAN's 38.27 dB. Visually, EACRN produces sharper edges and more accurate textures, particularly in complex urban scenes where other models often introduce blurring or aliasing.
Training Efficiency
A standout feature of EACRN is its convergence speed. As shown in the comparative analysis, while models like RCAN require approximately $4 \times 10^5$ updates to reach peak performance, EACRN achieves superior results in just $1 \times 10^3$ updates—a massive reduction in training time and resource consumption.
Conclusion
EACRN represents a significant step forward in efficient SISR. By combining a hierarchical cross-residual structure with an iterative EM-based attention mechanism, the model achieves high-quality reconstructions with remarkable training efficiency. While the performance gains are most notable on complex textures, future work may focus on optimizing the model for simpler, repetitive geometric patterns and extending the architecture to video super-resolution.
Summary updated