Continuous Resolution Scaling for Efficient Vision

This direction is no longer present in the current build. Its last version stays readable and citable here.

Version 1 of 123 papers23 conceptsfrozen 2026-10-04

In computer vision, the standard approach to processing images is surprisingly rigid. Whether a model is looking at a high-resolution close-up of a panda or a tiny, blurry insect, it typically processes both at the exact same fixed resolution—often $224 \times 224$ pixels. As noted by Mingjian Zhu and colleagues in their work on Dynamic Resolution Networks, this "one-size-fits-all" strategy is computationally wasteful.

A growing body of research is moving away from these fixed constraints toward Continuous Resolution Scaling. The goal is to adaptively adjust the computational budget—whether through pixels, tokens, or model width—to match the specific complexity of every unique input.

The Discrete Bottleneck

Most current "adaptive" models are not truly fluid; they choose from a small, predefined menu of options. For example, a Vision Transformer (ViT) might be trained to handle three discrete patch lengths. While this is better than a single fixed size, it creates a "granularity gap."

Researchers like Yichen Zhu argue that these discrete choices fail to capture the optimal efficiency-accuracy trade-off. In ReViT, the authors demonstrate that "easy" images can be processed with far fewer tokens than "hard" ones. However, if the model is restricted to a few fixed resolutions, it often over-allocates resources to images that fall between those levels. This discretization problem extends to object detection as well; Leilei Cao and Yao Xiao point out in their work on EMface that standard feature pyramids assign objects to discrete levels based on size, failing to account for the continuous nature of object scales in the real world.

Learning the Optimal Scale

To move toward continuity, models must learn to predict the required resolution on-the-fly. This is often achieved through a "router" or a "scale factor metric."

  1. Dynamic Tokenization: In the realm of Multimodal Large Language Models (MLLMs), Long Cui and the authors of ViCO use a lightweight router to identify semantically important regions, compressing less relevant patches. Similarly, Junhong Shen introduced CAT, which uses a nested architecture to allow for more granular control over token allocation based on image complexity.
  2. Super-Resolution Recovery: For tasks like person re-identification, Ke Han and Yan Huang developed the PRI model. Instead of applying a fixed super-resolution factor, the model predicts a dynamic "soft label" for the scale, balancing the need for detail against the risk of introducing noise.
  3. Search Region Regulation: In visual tracking, the "search window" size is critical. Jiawen Zhu and Xin Chen argue in SRRT that moving from discrete scale factors to continuous scale estimation allows the tracker to adjust its spatial window precisely relative to the target's dynamics.

Multi-Dimensional Adaptation

The most advanced iterations of this idea don't just scale resolution; they scale the model itself. Jinyang Guo and Wanli Ouyang proposed Multi-Dimensional Pruning (MDP), which simultaneously prunes network channels and reduces spatial/temporal resolutions.

This holistic approach is echoed in AdaSGN by Lei Shi and colleagues, which adaptively reduces both the number of joints processed in skeleton-based recognition and the model size. By treating these parameters as continuous or highly granular variables rather than fixed configurations, these frameworks achieve significant speedups (often over 50%) with negligible drops in accuracy.

Why Continuity Works

The intuition behind continuous scaling is resource alignment. In a discrete system, a model might have to choose between "too much" and "not enough" computation. A continuous system allows the model to find the "Goldilocks zone"—the absolute minimum resolution or model width required to solve a specific task.

As Junshan Hu demonstrates with TokenFLEX, training models to be flexible across a range of visual scales not only saves inference time but can also reduce training costs. By breaking the reliance on fixed grids and discrete steps, vision models are becoming more like biological systems: focusing intense resources on complex details while efficiently skimming the rest.

Papers in this version

History & citationVersion 1 · 2026-10-04

No longer present in the current build · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_continuous_resolution_scaling_for_effici_0ea7698e,
  title = {Continuous Resolution Scaling for Efficient Vision},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/continuous-resolution-scaling-for-efficient-vision/rdr_0ea7698e2f174821b073fbd01f4539cb},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_continuous_resolution_scaling_for_effici_0ea7698e_v1,
  title = {Continuous Resolution Scaling for Efficient Vision},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/continuous-resolution-scaling-for-efficient-vision/rdr_0ea7698e2f174821b073fbd01f4539cb/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}