Abstract
Deep CNNs have achieved significant successes in single image super-resolution (SR). However, conventional methods still resort to predetermined integer scaling factors, making them difficult to apply when arbitrary target resolutions or geometric transformations are required. We propose the SRWarp framework to generalize SR tasks toward arbitrary image transformations. By interpreting image warping as a spatially-varying SR problem, we introduce an adaptive warping layer and multiscale blending to reconstruct visually favorable results. Extensive experiments demonstrate the advantage of SRWarp over traditional interpolation and fixed-scale SR methods.
Introduction
Standard Super-Resolution (SR) research typically focuses on uniform upscaling, where a low-resolution (LR) image is mapped to a high-resolution (HR) grid by a fixed integer factor. However, real-world applications like lens distortion correction, perspective shifts, or homographic alignments involve non-uniform stretching and compression across the image surface.
Traditional tools, such as OpenCV’s warpPerspective, rely on bicubic interpolation. While computationally efficient, these methods often produce blurry edges and lose high-frequency textures because they lack the capacity to reconstruct missing details. Conversely, existing SR models are constrained by regular grids and fixed scales. SRWarp addresses this gap by treating arbitrary transformations as a spatially-varying SR task where the magnification factor changes at every pixel.
Methodology
The core conceptual shift in SRWarp is viewing an image warp as a "curved slice" through a scale pyramid. In this model, the LR image resides at the top ($z=1$), and a high-magnification SR image resides at the bottom (e.g., $z=4$). A warped image is reconstructed by sampling from this continuous scale space.
Adaptive Warping Layer (AWL)
To handle local deformations, the AWL utilizes the Jacobian matrix of the transformation to linearize the geometry at each pixel. Instead of a static kernel, the AWL dynamically predicts a resampling kernel that accounts for local rotation and stretching. This ensures that the sampling process is geometrically consistent with the intended transformation.
Multiscale Blending and Partial Convolutions
The framework extracts features at multiple scales (x1, x2, and x4) and blends them based on a "scale feature" derived from the Jacobian determinant. This determinant represents local magnification, allowing the model to prioritize low-resolution features in "shrunk" areas to avoid aliasing and high-resolution features in "stretched" areas to provide detail. Additionally, partial convolutions are employed to handle "void" pixels at image boundaries, preventing artifacts from leaking into the reconstructed regions.
Experiments and Results
The authors introduced the DIV2K-Warping (DIV2KW) dataset, consisting of high-quality images subjected to random projective transformations. SRWarp was evaluated against traditional interpolation and state-of-the-art SR models combined with standard warping.
Quantitative Performance
On the DIV2KW test set, SRWarp achieved a 31.04 dB mPSNR, significantly outperforming the combination of a state-of-the-art SR model (RDN) and OpenCV warping (30.22 dB). Furthermore, SRWarp proved efficient, running approximately 2.5x faster and using 40% less memory than specialized arbitrary-scale models like Meta-RDN.
Qualitative Comparison
Visual results highlight SRWarp's ability to eliminate "staircase" artifacts and blurriness common in traditional perspective warping.
Conclusion
SRWarp successfully generalizes the super-resolution task to arbitrary geometric transformations. By integrating the transformation's Jacobian into the network architecture via the Adaptive Warping Layer and multiscale blending, the model produces high-quality, sharp images that surpass traditional interpolation techniques. While currently focused on homographies, the framework provides a robust foundation for future work in complex, non-rigid image deformations.
Summary updated