Calibrating Attention-Based Token Pruning for Efficient Multi-Image Synthesis in UMMs
Lists
Only you see this
Abstract
Unified Multimodal Models (UMMs) enable powerful multi-image synthesis but incur substantial computation from long visual contexts. While token pruning offers a direct route to accelerating inference, existing methods are primarily adapted from understanding tasks, suffering from insufficient reference image exploration, confused conditional guidance, and pooling-induced ambiguity that degrades generation quality. To address these issues, we propose CATP, the first training-free context-compression method tailored to UMM generation, which \textbf{C}alibrates the discrepancies of general \textbf{A}ttention-based \textbf{T}oken \textbf{P}runing for better quality-efficiency trade-off. Concretely, building on independent layer-wise pruning during denoising, we introduce Guidance-Consistent Pruning (GCP) to synchronize pruning decisions across conditioning branches, and a Taylor-Calibrated Attention Metric (TCAM) that derives a theoretically grounded second-order correction for pooling-induced bias. In addition, a Fast Scoring Operator (FSO) fuses the calibration into tiled online Softmax to reduce pruning cost. Extensive experiments across representative UMMs and benchmarks demonstrate that CATP consistently outperforms state-of-the-art token-reduction methods, achieving up to 2.09$\times$ attention-core speedup.
BibTeX
@misc{wu2026calibrating,
author = {Zimeng Wu and Qijian Tian and Penghui Du and Changqian Yu and Jiaxin Chen and Yunhong Wang},
title = {{Calibrating Attention-Based Token Pruning for Efficient Multi-Image Synthesis in UMMs}},
year = {2026},
eprint = {2610.39467},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.39467},
}Loading related content...