Aspect Ratio Preserving Vision for Document Understanding

This direction is merged into another direction: aspect ratio distortion. Its last version stays readable and citable here.

Version 1 of 126 papers27 conceptsfrozen 2026-10-04

The standard pipeline for computer vision models involves a destructive first step: resizing every input image into a fixed square, typically 224x224 or 336x336 pixels. While this simplifies batch processing and positional embeddings, it introduces geometric distortion that "squashes" or "stretches" the visual content. For natural images of dogs or trees, this is often tolerable. However, for documents, mobile UIs, and infographics—where the meaning is encoded in the precise shapes of characters and the spatial relationships of a layout—this "square bottleneck" is a primary cause of model failure.

The Cost of Geometric Distortion

When a long, narrow text line or a tall mobile screenshot is forced into a square, the aspect ratio is violated. As Adrian Chan and colleagues observed in HATFormer, squashing vertical strokes in handwritten Arabic text obscures the fine details necessary for transcription. Similarly, Kenton Lee notes in the development of Pix2Struct that this warping degrades a model's ability to perform spatial reasoning, as the relative distances between visual elements are no longer consistent with the original document.

Beyond legibility, Lijie Wang demonstrated that even in aesthetic assessment, resizing destroys the compositional integrity of an image. An 18% reduction in error was achieved simply by allowing the model to "see" the original aspect ratio of panoramas and non-standard photos.

Dynamic Tiling and Flexible Patching

To solve this, researchers have moved toward architectures that adapt to the image rather than forcing the image to adapt to the architecture. The core intuition is to break the image into a variable number of patches or "tiles" that maintain their original proportions.

  1. Dynamic Partitioning: Models like AdaptVision by Wengang Zhou and Hao Feng use a module that scales the number of visual tokens based on the image's complexity and shape. Instead of a fixed grid, the model allocates more tokens to text-dense areas and preserves the native aspect ratio.
  2. Image Slicing: DeepSeek-VL2, led by Zhiyu Wu, employs a dynamic tiling strategy. If an image is very wide, it is sliced into multiple square tiles that are processed individually and then reassembled. This allows the underlying Vision Transformer (ViT) to work on undistorted sub-images while the language model understands their global context.
  3. Native Resolution Encoders: The most direct approach is seen in UniViTAR by Limeng Qiao, which is designed to natively process any resolution or aspect ratio without cropping or resizing, ensuring that fine-grained details are never lost to interpolation.

Managing the Token Explosion

If a model processes a high-resolution document at its native aspect ratio, it generates a massive number of visual tokens, which slows down inference. This "token explosion" is the primary trade-off for accuracy.

To counter this, Jiaxin Zhang introduced "visual slimming" in DocKylin. By adaptively pruning redundant tokens (like white space in a document) while keeping the aspect ratio intact, the model maintains high performance without the computational cost of a full high-resolution grid. Similarly, Chaohu Liu and the HRVDA team use instruction-based filtering to focus only on the visual tokens relevant to the user's query, effectively "ignoring" the parts of a large document that don't matter.

Why It Works: Spatial Consistency

The reason these methods outperform standard resizing is spatial consistency. In a document, the relationship between a header and a paragraph is a fixed geometric fact. When Gilles Baechler and the ScreenAI team apply flexible patching to mobile screens, the model learns to associate text with its specific UI button because the distance between them hasn't been warped by a resizing algorithm.

By preserving the aspect ratio, we ensure that the features the model learned during pretraining (like the curves of a letter or the straight edge of a table) look the same regardless of whether they appear in a square photo or a long vertical infographic. As Cheng Cui demonstrates with PaddleOCR-VL, this native understanding is what allows ultra-compact models to outperform much larger ones that are still struggling to see through the "fog" of geometric distortion.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_aspect_ratio_preserving_vision_for_docum_f275e042,
  title = {Aspect Ratio Preserving Vision for Document Understanding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/aspect-ratio-preserving-vision-for-document-understanding/rdr_f275e04264774f03ba33f85696d1f31b},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_aspect_ratio_preserving_vision_for_docum_f275e042_v1,
  title = {Aspect Ratio Preserving Vision for Document Understanding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/aspect-ratio-preserving-vision-for-document-understanding/rdr_f275e04264774f03ba33f85696d1f31b/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}