In the world of Document OCR, high resolution is a double-edged sword. To catch a tiny decimal point or a superscript "2," a model needs a dense grid of visual tokens. However, feeding thousands of tokens into a Large Multimodal Model (LMM) decoder is computationally expensive, slowing down inference and quickly hitting context limits.
Existing solutions usually try to prune "redundant" tokens or merge similar ones. But as the authors of "Carry the Fine Print" argue, document images are unique: a tiny, visually "boring" symbol can be the most important part of a transcript. Their solution, Select and Compensate, offers a way to slash token counts by 50% or more while ensuring the "fine print" isn't left on the cutting room floor.
TL;DR
The "Select and Compensate" framework compresses visual tokens for OCR by choosing a sparse set of "anchor" tokens and enriching them with pooled information from discarded neighbors. This approach maintains near-uncompressed accuracy at 50% token retention, significantly reduces inference time, and prevents the "runaway" text generation common in other compression methods.
What Problem Does This Paper Address?
Most visual token compression methods assume that "salience" (how much a region stands out) or "similarity" (how much it looks like its neighbors) are good proxies for importance. In document OCR, this assumption fails.
As shown in Figure 1, a missing exponent or a dropped decimal point completely changes the meaning of a report. Furthermore, repeated entries in a table might look identical (high similarity), but their specific locations are crucial for structural integrity. When we compress these images, we often lose these "weak salience" details, leading to transcription errors or "hallucinations" where the model loses its place and generates endless, repetitive text.
Methods
The authors propose a two-stage framework that keeps the underlying OCR backbone frozen, making it a "plug-and-play" enhancement.
- Selection (The "Where"): Instead of a uniform grid, the model identifies informative "anchors."
- Adaptive Resolution: It first analyzes the image to shrink blank margins and background space, fitting more "content-heavy" areas into the token budget.
- Learned Ranking: It uses a scorer that combines global importance (page context) with local contrast. If a token looks different from its immediate eight neighbors, it’s flagged as potentially containing a high-contrast detail like a small symbol.
- Compensation (The "What"): This is the core innovation. Choosing where to keep a token shouldn't limit what evidence it carries.
- Discarded tokens (donors) are routed to the nearest retained anchors.
- The model pools the content and the relative spatial offset of these donors into a fixed-size residual (using "vector moments").
- This residual is added to the anchor's embedding, effectively allowing a single anchor token to "carry" the visual evidence of its discarded neighborhood.
Datasets
The framework was evaluated across five different model backbones (including Ovis, Logics, and Qwen2.5-VL) using:
- OmniDocBench: A comprehensive benchmark for page-level transcription (text, formulas, tables).
- MPDocBench: Used for document-level reconstruction across multiple pages.
- Training: The compressors were trained on the olmOCR-mix-1025 dataset, which contains a diverse mix of document types.
Key Insights and Ideas
- Decoupling Address from Evidence: The most powerful idea here is that a token's position in the decoder sequence (its "address") doesn't have to represent only the pixels at that exact spot. By treating anchors as "buckets" for local evidence, the model preserves the "fine print" without increasing the sequence length.
- The "Runaway" Effect: One surprising finding was that poor compression doesn't just cause typos; it causes the decoder to lose structural context. In tests, baseline compression methods caused over 300 pages (out of 1,651) to hit the maximum output length limit because the model got "stuck." Select and Compensate reduced this to just 15 pages.
- Local Contrast is a Proxy for Detail: While global salience might favor a big bold header, local contrast is much better at finding the tiny dot of a decimal point or the thin line of a fraction bar.
Key Results
The results show that you can have your cake and eat it too: high compression and high accuracy.
- Near-Lossless Performance: On the Ovis backbone, the uncompressed model scored 96.09 on OmniDocBench. With 50% fewer tokens, Select and Compensate reached 95.42. For comparison, other state-of-the-art pruning methods like RTPrune dropped to 73.13 at the same budget.
- Efficiency Gains: Because the model generates fewer "runaway" tokens and processes a shorter visual sequence, it finished batch processing 35–40% faster than other compression adaptations.
- Robustness at High Compression: Even at a extreme 25% retention rate, the method outperformed standard global selectors by 4.70 points, proving that the "Compensation" stage becomes increasingly vital as the budget tightens.
Limitations and Future Work
- Backbone Sensitivity: While the method works across various models, the gains vary. For instance, on the Qwen backbone, it excelled at text but trailed slightly on complex formulas compared to some baselines.
- Task Specialization: The current training focuses purely on OCR. The authors note that these compression weights do not preserve general VQA (Visual Question Answering) performance; a model compressed this way is an OCR specialist, not a generalist.
- Input Rule Generalization: The adaptive resolution rules were developed after inspecting specific benchmarks; their performance on entirely unseen document formats remains a target for future study.
Summary updated