Abstract
Document dewarping is crucial for many applications. However, existing learning-based methods primarily rely on supervised regression with annotated data without leveraging the inherent geometric properties in physical documents to the dewarping process. Our key insight is that a well-dewarped document is characterized by transforming distorted feature lines into axis-aligned ones. This property aligns with the inherent axis-aligned nature of the discrete grid geometry in planar documents. In the training phase, we propose an axis-aligned geometric constraint to enhance document dewarping. In the inference phase, we propose an axis alignment preprocessing strategy to reduce the dewarping difficulty. In the evaluation phase, we introduce a new metric, Axis-Aligned Distortion (AAD), that not only incorporates geometric meaning and aligns with human visual perception but also demonstrates greater robustness. As a result, our method achieves SOTA results on multiple existing benchmarks and achieves 18.2%~34.5% improvements on the AAD metric.
Introduction
Most document dewarping methods treat the problem as a standard image-to-image regression task, often missing the fundamental geometry of the paper itself. When capturing images of documents—such as book pages or receipts—curves, folds, and perspective distortions are common. "Dewarping" is the process of digitally flattening these images to make them readable for humans and searchable for Optical Character Recognition (OCR) engines.
While deep learning has improved this field, existing models often struggle with complex deformations because they rely on brittle supervision signals. This paper introduces a "principle-driven" approach based on the axis-aligned property: the idea that a perfectly rectified document is essentially a grid where feature lines must align with the X and Y axes. By baking this geometric constraint into training, inference, and a new evaluation metric (AAD), the authors achieve state-of-the-art results, improving dewarping accuracy by up to 34.5%.
Methodology
The researchers built their solution on a lightweight convolutional neural network (CNN), termed Grid-Net, that predicts a 2D unwarping grid. Their innovation lies in three key areas:
- Axis-Aligned Geometric Constraint (Training): During training, the model maps the predicted grid into a "UV space" (a normalized coordinate system) and calculates a loss based on the variance of points along rows and columns. This explicitly penalizes the network if it produces "wavy" or tilted results, ensuring the document is treated as a discrete grid.
- Axis-Alignment Preprocessing (Inference): The system uses a "coarse-to-fine" strategy. It makes an initial prediction to calculate the document's principal orientation, then rotates and crops the original image to align it better before a second "fine" pass. This makes the model much more robust to camera tilt.
- Axis-Aligned Distortion (AAD) Metric: Existing metrics like MS-SSIM or Aligned Distortion (AD) often fail to align with human perception. AAD uses image gradients and optical flow to mathematically measure how well dewarped feature lines align with coordinate axes.
Key Results and Insights
The model was trained on synthetic datasets like Doc3D and UVDoc and evaluated on standard benchmarks including DocUNet and DIR300. The "Axis-Aligned" approach outperformed previous state-of-the-art methods across several dimensions:
- Geometric Accuracy: On the DIR300 benchmark, the method improved the AAD metric by 34.5% compared to the previous best model (UVDoc).
- OCR Performance: The ultimate test of dewarping is readability. Character Error Rate (CER) dropped by 14.8% on DocUNet and 23.7% on DIR300.
- Robustness: Qualitative results show the model handles extreme folds and crumpled paper more gracefully than models lacking axis-aligned constraints.
By shifting from "learning what a document looks like" to "enforcing what a document is geometrically," the model gains a mathematical "north star" independent of specific text or language. The paper also highlights that optimizing for the right metric is critical; their AAD metric aligns much more closely with human visual perception than previous standards.
Conclusion and Future Work
While the method is highly effective, the iterative inference (the coarse-to-fine pass) adds some processing time, though it remains suitable for most mobile applications. Future work may extend this axis-aligned principle to other image rectification tasks, such as straightening architectural photos or flattening textures on 3D objects. Currently, the model assumes a standard document grid; exploring how it handles non-linear layouts, such as circular text, remains an open area for research.
Summary updated