When humans read a complex document—whether it is a legal contract, a technical manual, or a textbook—we do not perceive it as a single, continuous ribbon of text. Instead, we navigate it as a nested hierarchy. We understand that a specific sub-clause is governed by a broader section heading, and that a bullet point inherits the conditional constraints of its introductory preamble.
However, when documents are prepared for modern natural language processing pipelines, they undergo an act of violent simplification: document flattening. Standard text extraction tools strip away layout metadata, HTML tags, and section boundaries, reducing a rich, multi-dimensional tree into a linear sequence of tokens.
Recent research across information retrieval and document intelligence demonstrates that this flattening process causes severe hierarchical context loss. When a document's logical skeleton is discarded, downstream models struggle to maintain semantic integrity, leading to retrieval failures, misinterpretations, and hallucinations.
How Flattening Destroys Semantic Context
To understand why flattening degrades model performance, consider how traditional Retrieval-Augmented Generation (RAG) frameworks operate. As noted by Wenyu Tao et al. in their work on TreeRAG, standard systems treat documents as flat sequences of text chunks. This structural blindness manifests in two primary failure modes:
1. The Contextual Disconnect
When a long document is broken into fixed-length text chunks, paragraphs are separated from their parent headings. As Seongtae Hong et al. observe in the context of industrial manuals, separating nested lists and content from their headers strips away the exact semantic context required for accurate retrieval. A chunk containing the text "Apply lubrication every 50 hours" becomes useless—or dangerously ambiguous—if the parent heading "Component X (Optional Add-on)" is left behind in a previous chunk.
2. The Conditional Collapse
In regulatory and legal domains, meaning is highly conditional. Alan Buabuchachart et al. demonstrate that standard NLP techniques treat nested list items as isolated units, stripping away the essential context provided by introductory preambles. Without inheriting the parent constraints, specific provisions can lose their conditional meaning entirely. For instance, if a preamble states "The following exceptions apply only to entities operating under License A:", flattening the subsequent bulleted list makes it impossible for a model to discern that the rules are conditional rather than universal.
This problem extends to web documents and user interfaces. The authors of WebFormer and Jocelyn Qiaochu Chen et al. point out that treating web pages as linear text sequences discards the hierarchical relationships defined by the HTML DOM tree. When the DOM is flattened, models lose the grouping signals that visually and logically link related items, such as products in a grid or fields within a form, a limitation highlighted by Yuxuan Lu et al. in web-agent training.
Reconstructing the Document Skeleton
To prevent this loss of context, a major branch of research focuses on reconstructing or preserving the document's hierarchical skeleton before any downstream processing occurs.
Flat Representation (Lossy):
[Chunk 1: Intro] -> [Chunk 2: Sub-clause A] -> [Chunk 3: Sub-clause B]
Hierarchical Representation (Preserved):
[Document Root]
/ \
[Section 1] [Section 2]
/ \
[Heading 1.1] [Heading 1.2]
| |
[Paragraph A] [Nested List]
For legacy formats like PDFs, which are optimized for visual rendering rather than structural data, this requires specialized parsing. Yue Zhang 0073 introduces a transition-based parser called PDF-to-Tree to organize disorganized text blocks back into a clean hierarchy of sections, subsections, and paragraphs. Similarly, Tong Zhu et al. develop the CED framework to extract the structural "skeleton" or table of contents from long documents, reversing the flattening that typically occurs during plain-text or OCR conversion.
This structural restoration is not a new desire; early knowledge management systems faced identical hurdles. For example, Eric Schoen et al. designed graphical editors in the 1980s to prevent the loss of context that occurred when navigating complex inheritance hierarchies via text-only interfaces. In the early 2000s, Hervé Déjean et al. focused on converting flat PDFs into structured XML, while Raymond Kosala et al. and Mohammed J Zaki et al. argued that treating tree-structured XML/HTML data as flat strings or "bags of words" discards the vital organizational context embedded within tags.
Why Tree-Aware Architectures Work
Once a document's hierarchy is preserved, retrieval and reasoning systems can exploit it using mechanisms that fundamentally outperform linear sequence processing.
Bidirectional and Multi-Granularity Retrieval
Instead of searching through a flat pool of isolated text chunks, advanced frameworks navigate the document tree dynamically. Xueyu Chen et al. introduce a two-stage retrieval framework (HTSIR) that builds a tree of summaries, allowing language models to inspect information at multiple levels of granularity—from broad section overviews down to specific text blocks.
When a query requires fine-grained details that depend on high-level context, the system can perform bidirectional traversal. As designed in TreeRAG, the model locates a relevant node and traverses upward to inherit the parent context, or downward to gather supporting details, ensuring the semantic integrity of the retrieved information.
Contextualized Query Routing
Flattening a document also ruins the query side of information retrieval. If a user searches for a specific topic located deep within a sub-heading, converting the path of headings into a flat string discards metadata about term importance.
To solve this, Andrew Yates et al. incorporate contextual vectors that explicitly account for heading hierarchies. This allows models to distinguish between a primary title and a minor sub-heading. Xinshi Lin et al. expand on this by using structural metadata to link concepts across different hierarchical headings, ensuring that the precise path of headings guides the retrieval engine to the correct paragraph.
Structural First-Class Citizens
In modern LLM applications, frameworks like PDFTriage, authored by Jon Saad-Falcon et al., treat document structure as a first-class citizen. Instead of feeding raw text to an LLM, they provide the model with a triage layer of metadata and callable retrieval functions aligned with the document's layout.
This architectural shift directly addresses the "lost in the middle" phenomenon. As Chenxi Lin et al. note, standard LLMs treating documents as linear sequences often ignore critical context when it is obscured by irrelevant information in long prompts. By filtering information through a structural lens, models can prune irrelevant branches of text entirely before generation.
Conclusion
The shift from linear document processing to hierarchy-aware architectures represents a fundamental realization: the structure of a document is an inseparable part of its meaning. Whether through structure-aware chunking methods like StyleDFS (Seongtae Hong et al.), modular educational assistants (the authors of EduMod-LLM), or multi-agent legal frameworks (Petros Raptopoulos et al.), preserving the document tree ensures that downstream models reason over complete, context-rich information rather than fragmented snippets of text.
Key Foundational Work
An Algorithm for One-page Summarization of a Long Text Based on Thematic Hierarchy Detection (2000) — contributed 1 concept elements to this direction.
IMPULSE: A Display Oriented Editor for STROBE (1983) — contributed 1 concept elements to this direction.