OmniScaleSR: Unleashing Scale-Controlled Diffusion Prior for Faithful and Realistic Arbitrary-Scale Image Super-Resolution

Xinning Chai, Zhengxue Cheng, Yuhong Zhang, Hengsheng Zhang, Yingsheng Qin, Yucai Yang, Rong Xie, Li Song

TL;DR

OmniScaleSR is a novel diffusion-based framework for Arbitrary-Scale Super-Resolution (ASSR) that can upscale images to any factor (even extreme ones like ×24) using a single model. By introducing explicit "scale-controlled" mechanisms into the diffusion process, it successfully balances image fidelity with high-quality generative realism, outperforming current state-of-the-art methods in both synthetic and real-world scenarios.

What Problem Does This Paper Address?

Traditional Super-Resolution (SR) models are usually "fixed-scale" (e.g., they only do ×4). Arbitrary-Scale SR (ASSR) aims to handle any magnification factor with one model.

Current ASSR methods typically rely on Implicit Neural Representations (INR), which are great at handling continuous coordinates but often produce blurry, "regression-averaged" results that lack fine textures. On the other hand, recent Diffusion-based models produce incredibly realistic images but are often "scale-blind." Without explicit knowledge of the target scale, these models struggle at extreme magnifications, either "hallucinating" too many fake details or failing to sharpen the image enough, leading to inconsistent results.

Methods

The authors propose OmniScaleSR, which integrates explicit scale controls directly into a Latent Diffusion Model (LDM). The architecture consists of two main branches: a Generation Branch (to create realistic details) and a Fidelity Branch (to ensure the output matches the input).

Key innovations include:

  • Global Scale Injection: Much like how diffusion models use a "timestep" to know where they are in the denoising process, OmniScaleSR injects the SR scale factor ($s$) into every ResNet block of the UNet. This gives the model an "overall perception" of how much detail it needs to generate.
  • Local Scale Modulation (RefineNet): A dedicated module that dynamically coordinates the generation and fidelity branches. It uses a "gain" and "bias" mechanism to adjust how much the model should rely on the original low-resolution features versus its own generative priors, based on the specific scale and local image content.
  • Multi-Domain Fidelity Enhancement:
    • Pixel Domain: Uses a lightweight pre-trained ×4 SR model for initial upscaling instead of simple bicubic interpolation.
  • Latent Domain (LQA Loss): A "Latent Quality Loss" ensures the features in the fidelity branch closely match high-quality ground truth features.
  • Semantic Domain (SePR): Uses BLIP-2 and RAM (Recognize Anything Model) to generate text prompts and tags, ensuring the model understands what it is upscaling (e.g., "a birdhouse") to maintain semantic consistency.

Datasets

  • Training: A combination of high-quality datasets including LSDIR, DIV2K, and FFHQ (human faces).
  • Evaluation:
    • Bicubic Benchmarks: DIV2K and Urban100.
  • Real-World Benchmarks: Synthetic DIV2K (with complex degradations), DRealSR, and RealSR (captured with real cameras).
  • Scales Tested: A wide range from moderate (×5.3) to extreme (×24).

Key Insights and Ideas

  • Scale-Aware Generation: The core "aha moment" is that a model's behavior should change based on scale. At low scales (e.g., ×2), the model should focus on fidelity; at high scales (e.g., ×16), it must lean into its generative "imagination" to fill in missing information.
  • Beyond INR: While previous diffusion ASSR models used INR as a pre-processing or post-processing step, OmniScaleSR makes the diffusion process itself "scale-native."
  • RefineNet as a Dynamic Balancer: By modulating features spatially, the model can choose to be more "creative" in textured areas (like grass) while remaining "faithful" in structured areas (like text or edges).

Key Results

  • Quantitative: OmniScaleSR consistently ranks first or second across nearly all perceptual metrics (LPIPS, FID, CLIPIQA). At an extreme scale of ×24, it outperformed the previous best method (XPSR) by 12.9% in LPIPS and 16.6% in CLIPIQA on the RealSR dataset.
  • Qualitative: Visual comparisons show that while other models produce "blocking artifacts" or blurry textures at ×16 and ×24, OmniScaleSR maintains sharp edges and realistic, fine-grained textures (like individual feathers or architectural details).
  • Efficiency: Despite its complexity, the model is more parameter-efficient than some competitors (like FaithDiff) because it doesn't require massive multi-modal LLMs for every inference step.

Limitations and Future Work

  • Inference Speed: Like most diffusion models, OmniScaleSR requires multiple denoising steps, making it slower than simple CNN-based upscalers. The authors suggest exploring one-step acceleration techniques (like distillation) in the future.
  • Pixel-Level Consistency: While the images look highly realistic, the model does not guarantee strict pixel-perfect PSNR (Peak Signal-to-Noise Ratio). It prioritizes how the image looks to a human over mathematical identity to the original.
  • Semantic Bias: If the automated text prompts (from BLIP-2) are incorrect, the model might occasionally "hallucinate" the wrong type of texture (e.g., turning wheat into grass). Improving prompt reliability is a noted area for future research.

Summary updated

Abstract

Arbitrary-scale super-resolution (ASSR) overcomes the limitation of traditional super-resolution (SR) methods that operate only at fixed scales (e.g., 4x), enabling a single model to handle arbitrary magnification. Most existing ASSR approaches rely on implicit neural representation (INR), but its regression-driven feature extraction and aggregation intrinsically limit the ability to synthesize fine details, leading to low realism. Recent diffusion-based realistic image super-resolution (Real-ISR) models leverage powerful pre-trained diffusion priors and show impressive results at the 4x setting. We observe that they can also achieve ASSR because the diffusion prior implicitly adapts to scale by encouraging high-realism generation. However, without explicit scale control, the diffusion process cannot be properly adjusted for different magnification levels, resulting in excessive hallucination or blurry outputs, especially under ultra-high scales. To address these issues, we propose OmniScaleSR, a diffusion-based realistic arbitrary-scale SR framework designed to achieve both high fidelity and high realism. We introduce explicit, diffusion-native scale control mechanisms that work synergistically with implicit scale adaptation, enabling scale-aware and content-aware modulation of the diffusion process. In addition, we incorporate multi-domain fidelity enhancement designs to further improve reconstruction accuracy. Extensive experiments on bicubic degradation benchmarks and real-world datasets show that OmniScaleSR surpasses state-of-the-art methods in both fidelity and perceptual realism, with particularly strong performance at large magnification factors. Code will be released at https://github.com/chaixinning/OmniScaleSR.
BibTeX
@article{chai2026omniscalesr,
  author = {Xinning Chai and Zhengxue Cheng and Yuhong Zhang and Hengsheng Zhang and Yingsheng Qin and Yucai Yang and Rong Xie and Li Song},
  title = {{OmniScaleSR: Unleashing Scale-Controlled Diffusion Prior for Faithful and Realistic Arbitrary-Scale Image Super-Resolution}},
  journal = {IEEE Trans. Circuits Syst. Video Technol.},
  year = {2026},
  doi = {10.1109/tcsvt.2025.3642578},
}

Loading related content...

Figure