Introduction: The Challenge of Document Hierarchy
When humans view a complex form or a mobile application interface, we intuitively group elements. We recognize that a label belongs to a specific checkbox, and together they form a logical "Choice Group." However, digital representations like PDFs or scanned images often discard this structural logic, leaving only raw pixels or a flat stream of text characters.
Existing models often treat documents as simple sequences of words. This approach faces a "chicken-and-egg" dilemma: accurate document understanding often requires a correct reading order, yet determining that order requires an understanding of the document's hierarchical structure. Standard "top-to-bottom, left-to-right" heuristics frequently fail in multi-column or nested layouts.
The LayerDoc Framework
To address these issues, the authors propose LayerDoc, a framework that reconstructs document structure through recursive, bottom-up grouping. Instead of predicting the entire hierarchy simultaneously, LayerDoc operates layer-by-layer.
Multimodal Contextual Encoding
The core of LayerDoc is a multimodal encoder that evaluates potential "parent-child" relationships between document elements using three primary signals:
- Semantic Cues: Textual content is processed using SentenceBERT (SBERT) to capture the meaning of labels and values.
- Spatial Cues: The model utilizes normalized coordinates of bounding boxes to understand geometric proximity.
- Visual Cues: Features are extracted directly from the document image via a visual encoder to identify visual boundaries and styles.
Recursive Grouping and Constraint Inference
The process begins with elementary tokens (words or icons). LayerDoc generates candidate parent boxes and uses a Transformer-based architecture to predict the likelihood of a link and classify the parent element type (e.g., "Choice Field" or "Table Row").
Because many overlapping candidate boxes may be generated, the model employs a global optimization step—a variation of the Maximum Independent Set problem—to select the most likely set of non-overlapping parents. These new parent boxes then serve as the "children" for the subsequent layer, repeating until the document is unified under a single root node.
Key Insights
- Hierarchy Informs Reading Order: By traversing the extracted tree structure, LayerDoc generates a reading sequence that is significantly more accurate than traditional OCR heuristics.
- Semantic Importance: Integrating SBERT for semantic analysis provided a substantial performance boost (8-14%), confirming that understanding text content is vital for logical grouping.
- Computational Efficiency: The layer-wise approach is highly efficient, requiring approximately 10x fewer forward passes than models that attempt exhaustive pairwise comparisons across the entire document.
Experimental Results
LayerDoc was evaluated on the Hierarchical Forms, RICO (mobile app layouts), and FUNSD datasets. Key findings include:
- Accuracy: A 10-15% relative improvement in spatial region classification and group identification over baselines like DocStruct and Form2Seq.
- Reading Order: An 8-12% improvement in inferred token reading order and semantic word grouping.
- Robustness: Higher Mean Average Precision (mAP) in reconstructing full document trees, particularly in complex, nested scenarios.
Limitations and Future Directions
While LayerDoc represents a significant advancement, it is susceptible to recursive error propagation, where a mistake in a lower layer (e.g., grouping words into lines) can affect all subsequent higher-level groupings. Future work may explore more sophisticated global optimization or "backtracking" mechanisms to allow the model to revise lower-layer decisions based on higher-level context. Additionally, testing cross-dataset generalization—such as applying models trained on forms to scientific papers—remains an area for further exploration.
Summary updated