Abstract
Single Image Super-Resolution (SISR) aims to reconstruct high-resolution (HR) images from low-resolution (LR) inputs. Traditional deep learning approaches often treat the network as a "black box," failing to distinguish between smooth regions and complex textures. This paper introduces the Detail-Fidelity Attention Network (DeFiAN), which utilizes an interpretable attention mechanism based on classical image processing. By explicitly extracting structural information via multi-scale Hessian filtering, DeFiAN adaptively prioritizes high-frequency details, achieving state-of-the-art fidelity and visual sharpness.
Introduction
While deep Convolutional Neural Networks (CNNs) have significantly advanced SISR, many architectures are adapted from visual recognition tasks without considering the specific requirements of detail reconstruction. This often leads to uniform processing where fine textures are blurred or lost. The challenge is to design a model that can adaptively preserve low-frequency "smooth" areas while aggressively reconstructing high-frequency "details" in a divide-and-conquer manner.
Methodology: The Detail-Fidelity Attention Module (DeFiAM)
The core innovation of DeFiAN is the Detail-Fidelity Attention Module (DeFiAM). Unlike standard attention blocks that learn arbitrary features, DeFiAM follows a mathematically grounded pipeline:
Multi-Scale Hessian Filtering (MSHF)
The network uses fixed-weight filters to calculate the Hessian matrix of feature maps. The maximum eigenvalue of this matrix serves as a metric for structural edges. By employing multiple kernel sizes ($3\times3, 5\times5, 7\times7$), the MSHF captures a hierarchy of details, from fine textures to coarser structural boundaries.
Dilated Encoder-Decoder (DiEnDec)
To process Hessian-derived features without losing resolution, the authors designed the Dilated Encoder-Decoder (DiEnDec). Utilizing dilated convolutions, this component progressively expands the receptive field to capture global context while maintaining the spatial dimensions necessary for super-resolution. This process effectively mimics classical morphological operations like erosion and dilation on the detail maps.
Distribution Alignment Cell (DAC)
To ensure effective modulation, the Distribution Alignment Cell (DAC) aligns the statistical distribution of the attention map with the original features. This learnable normalization and scaling process facilitates stable training and faster convergence.
Experimental Results
The DeFiAN model was trained on the DIV2K dataset and evaluated across five standard benchmarks: Set5, Set14, BSD100, Urban100, and Manga109.
Quantitative Performance
DeFiAN consistently outperforms state-of-the-art models such as RCAN and EDSR. Notably, on the challenging Manga109 dataset at $\times4$ upscaling, DeFiAN achieved a PSNR of 31.67 dB, a significant improvement of 0.37 dB over the previous best-performing method (CSFM).
Qualitative Analysis
The "detail-fidelity" approach is most evident in visual comparisons. In architectural scenes from Urban100, DeFiAN reconstructs distinct window frames and lattices where other models produce blurry artifacts. In Manga109, it restores sharp, readable text and fine hatching lines that are typically lost in standard SR pipelines.
Discussion and Limitations
A key insight of this work is that the Hessian matrix's maximum eigenvalue serves as an excellent proxy for image residues—the high-frequency information often missed by standard networks. However, the use of second-order derivatives introduces a theoretical bottleneck: gradient stability. Backpropagation involves third-order partial gradients, which can lead to exploding or vanishing gradients in extremely deep networks (e.g., >600 layers). Future research will focus on stabilizing these higher-order derivatives or exploring alternative interpretable structural priors.
Summary updated