Abstract
Face Super-Resolution (FSR) requires the restoration of high-resolution facial details from low-resolution inputs while maintaining structural identity. Traditional attention mechanisms often suffer from blurriness due to global noise or fixed-window constraints. This paper introduces the LAA-Transformer (Learning Attention from Attention), which employs a self-refinement strategy to adaptively focus on high-frequency facial features. By integrating Region Selection Attention (RSA) and frequency-domain wavelet fusion, the model achieves state-of-the-art performance in reconstructing sharp, realistic facial textures.
Introduction
Face Super-Resolution (FSR) is a specialized task focused on restoring high-resolution (HR) faces from low-resolution (LR) inputs. Unlike general super-resolution, FSR must preserve critical facial landmarks like eyes, nose, and mouth to maintain identity. Existing attention mechanisms face a trade-off: global attention often incorporates irrelevant distant textures leading to blur, while local window attention (e.g., SwinIR) is restricted to fixed grids that may miss irregular high-frequency details in "flat" areas. The LAA-Transformer addresses this by becoming content-aware, automatically identifying and refining the most informative regions.
Methodology
The proposed Efficient Self-Refinement Transformer consists of three primary novel components designed to optimize feature extraction and reconstruction.
Region Selection Attention (RSA)
RSA implements the "Learning Attention from Attention" concept through a two-step process. First, Coarse Self-Attention (CSA) is computed on downsampled features to generate a saliency map of information density. Second, the model selects the top-$k$ most critical regions for fine-grained Multi-Head Self-Attention (MHSA). This allows the model to attend to irregular, feature-rich shapes rather than being confined to square windows.
Feature Integration Module (FIM)
To move beyond treating all feature channels equally, the FIM employs a dual-branch structure. It combines channel attention with depth-wise convolutions to prioritize informative channels, enhancing the representational power of the transformer blocks.
Wavelet Fusion Module (WFM)
The WFM operates in the frequency domain using Haar Wavelets. It merges structural information (low/middle frequencies) from shallow layers with high-frequency details from deep layers. This approach ensures that the overall facial geometry is preserved while textures are realistically refined.
Experimental Results
The LAA-Transformer was evaluated on the CelebA and Helen datasets for an $8\times$ super-resolution task.
Quantitative Performance
On the Helen dataset, the model achieved a PSNR of 28.36 dB, outperforming the previous state-of-the-art Transformer (SwinIR) by 0.32 dB. It also demonstrated superior performance in SSIM and LPIPS metrics. Ablation studies confirmed that the region selection mechanism in RSA is vital, as removing it resulted in a 0.12 dB drop in PSNR.
Qualitative Analysis
Visual comparisons highlight the model's ability to produce sharper pupils and clearer nose structures with fewer artifacts compared to methods like RCAN, DIC, and SwinIR.
Conclusion
The LAA-Transformer introduces an efficient, content-aware approach to Face Super-Resolution. By leveraging self-refinement and frequency-domain fusion, it effectively captures high-frequency details without the computational overhead of global attention. Future work may address performance under extreme poses, occlusions, and scaling to even higher resolution factors.
Summary updated