StageVAR: Stage-Aware Acceleration for Visual Autoregressive Models

Senmao Li, Kai Wang, Salman H. Khan, Fahad Shahbaz Khan, Jian Yang, Yaxing Wang

Introduction

Visual Autoregressive (VAR) models have revolutionized image synthesis by predicting images across multiple scales, from low-resolution "thumbnails" to high-fidelity outputs. However, the computational cost scales quadratically with the number of tokens, making high-resolution generation slow. StageVAR is a training-free framework that accelerates these models by up to 3.4× by exploiting the different computational requirements of various generation stages.

Stage-Aware Generation Analysis

The researchers observed that image generation follows a predictable trajectory. By monitoring metrics like CLIP (semantic alignment) and DINO (structural consistency), they identified three distinct stages:

  1. Semantic Establishment: The model determines the core content (e.g., "a captain at sea").
  2. Structure Establishment: The spatial layout and object boundaries are defined.
  3. Fidelity Refinement: The model adds fine-grained textures and high-frequency details.
Quantitative and visual evidence for the three stages of generation: Semantic Establishment, Structure Establishment, and Fidelity Refinement.

As illustrated in the analysis, semantic and structural features stabilize well before the final scale. This suggests that the heavy Transformer computation in the final steps is largely redundant for maintaining the image's "meaning."

Key Optimizations

StageVAR introduces two primary techniques to reduce the workload during the Fidelity Refinement stage:

1. Semantic Irrelevance

The authors found that once the semantic content is locked in (typically by scale 16), the text prompt no longer needs to guide the generation. By setting the Classifier-Free Guidance (CFG) to zero in later scales, the model bypasses expensive text-conditioning logic.

The 'Earring Test' demonstrates that disabling CFG too early (e.g., Scale 16) results in the loss of prompt-specific details like earrings, whereas disabling it during the refinement stage (Scale 40+) preserves them perfectly.

2. Low-Rank Feature Compression

High-resolution features are mathematically "low-rank," containing significant redundancy. StageVAR uses Random Projection (RP) to compress these features and a Representative Token Restoration (RTR) strategy to reconstruct the output. This allows the model to achieve high-fidelity results while processing far fewer tokens.

Results and Performance

StageVAR was tested on the Infinity-2B and Infinity-8B models across benchmarks such as GenEval, DPG, and COCO.

  • Speed: Achieved a 3.4× speedup on Infinity-2B.
  • Quality: Maintained near-identical performance to the "vanilla" model, with only a 0.01 drop on GenEval.
  • Efficiency: Consistently outperformed other acceleration methods like SkipVAR by preserving high-frequency details that others often blur.

Conclusion

By recognizing that the final steps of image generation are primarily about texture rather than "thinking," StageVAR provides a highly efficient, training-free method to speed up state-of-the-art visual autoregressive models. This stage-aware philosophy offers a promising direction for optimizing other generative architectures.

Summary updated

BibTeX
@inproceedings{li2026stagevar,
  author = {Senmao Li and Kai Wang and Salman H. Khan and Fahad Shahbaz Khan and Jian Yang and Yaxing Wang},
  title = {{StageVAR: Stage-Aware Acceleration for Visual Autoregressive Models}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=C6kxImRizt},
}
Figure