Abstract
Arbitrary-scale super-resolution (ASSR) overcomes the limitation of traditional super-resolution (SR) methods that operate only at fixed scales (e.g., 4x), enabling a single model to handle arbitrary magnification. Most existing ASSR approaches rely on implicit neural representation (INR), but its regression-driven feature extraction and aggregation intrinsically limit the ability to synthesize fine details, leading to low realism. Recent diffusion-based realistic image super-resolution (Real-ISR) models leverage powerful pre-trained diffusion priors and show impressive results at the 4x setting. We observe that they can also achieve ASSR because the diffusion prior implicitly adapts to scale by encouraging high-realism generation. However, without explicit scale control, the diffusion process cannot be properly adjusted for different magnification levels, resulting in excessive hallucination or blurry outputs, especially under ultra-high scales. To address these issues, we propose OmniScaleSR, a diffusion-based realistic arbitrary-scale SR framework designed to achieve both high fidelity and high realism. We introduce explicit, diffusion-native scale control mechanisms that work synergistically with implicit scale adaptation, enabling scale-aware and content-aware modulation of the diffusion process. In addition, we incorporate multi-domain fidelity enhancement designs to further improve reconstruction accuracy. Extensive experiments on bicubic degradation benchmarks and real-world datasets show that OmniScaleSR surpasses state-of-the-art methods in both fidelity and perceptual realism, with particularly strong performance at large magnification factors.
Introduction
Traditional Super-Resolution (SR) models are typically constrained to fixed magnification factors, such as 2x or 4x. Arbitrary-Scale Super-Resolution (ASSR) seeks to break this limitation, allowing a single model to upscale images to any factor. Current ASSR methods often utilize Implicit Neural Representations (INR) to handle continuous coordinates. While effective for maintaining structure, INR-based methods often produce blurry, "regression-averaged" results that lack fine-grained textures.
Recent advancements in diffusion models have significantly improved the realism of SR outputs. However, these models are generally "scale-blind," lacking explicit mechanisms to adjust their generative behavior based on the target magnification. This leads to inconsistent results at extreme scales, such as x16 or x24, where models may either over-hallucinate details or fail to provide sufficient sharpening. OmniScaleSR addresses this by making the diffusion process "scale-native."
The OmniScaleSR Framework
OmniScaleSR integrates explicit scale controls directly into a Latent Diffusion Model (LDM). The architecture is built around two primary branches: a Generation Branch that leverages pre-trained diffusion priors for realism, and a Fidelity Branch that ensures the output remains true to the low-resolution (LR) input.
Key Innovations
- Global Scale Injection: Similar to how diffusion models utilize timesteps, OmniScaleSR injects the SR scale factor ($s$) into every ResNet block of the UNet. This provides the model with a global perception of the required detail level for the specific magnification task.
- Local Scale Modulation (RefineNet): A specialized module that dynamically coordinates the generation and fidelity branches. Using a "gain" and "bias" mechanism, it adjusts the balance between generative priors and input features based on both the scale and local image content.
- Multi-Domain Fidelity Enhancement: To improve reconstruction accuracy, the framework incorporates enhancements across three domains:
- Pixel Domain: Initial upscaling is performed by a lightweight pre-trained x4 SR model rather than simple bicubic interpolation.
- Latent Domain: A Latent Quality Loss (LQA) ensures that features in the fidelity branch align with high-quality ground truth features.
- Semantic Domain: By utilizing models like BLIP-2 and RAM, the system generates text prompts and tags (e.g., "a birdhouse") to maintain semantic consistency during the upscaling process.
Experimental Results
OmniScaleSR was evaluated on a variety of benchmarks, including DIV2K, Urban100, and real-world datasets like RealSR and DRealSR. The model was tested across a wide range of scales, from moderate (x5.3) to extreme (x24).
Quantitative Performance
Quantitative results show that OmniScaleSR consistently ranks at the top across perceptual metrics such as LPIPS, FID, and CLIPIQA. Notably, at an extreme scale of x24 on the RealSR dataset, OmniScaleSR outperformed the previous state-of-the-art (XPSR) by 12.9% in LPIPS and 16.6% in CLIPIQA.
Qualitative Comparison
Visual assessments highlight the model's ability to maintain sharp edges and realistic textures (such as individual feathers or architectural details) even at x16 and x24 magnifications. In contrast, competing models often exhibit blocking artifacts or significant blurriness at these extreme factors.
Conclusion
OmniScaleSR represents a significant step forward in arbitrary-scale super-resolution by successfully combining explicit scale control with powerful diffusion priors. By enabling scale-aware and content-aware modulation, the framework achieves a superior balance between fidelity and realism. Future research will explore inference acceleration techniques, such as distillation, to improve the speed of the diffusion-based process.
Summary updated