Traditional information retrieval models often treat a document as a flat "bag of words." In this paradigm, a keyword match occurring in a minor footnote carries the exact same mathematical weight as a match in the document's main title. However, human communication is inherently structured. Authors organize information using hierarchies—titles, sections, subheadings, tables, and metadata fields—to signal the relative importance and specificity of their ideas.
To exploit this organization, a prominent line of research focuses on structural and hierarchical term weighting. The core intuition is that where a term appears within a document or query hierarchy dictates its semantic density and relevance. By assigning non-uniform weights to terms based on their structural context, retrieval systems can move beyond simple keyword matching to capture the underlying architecture of information.
The Intuition of Structural Context
Why does structure matter? Consider a scientific paper containing a table. A word appearing in the table's caption or column header is highly descriptive of the entire dataset, whereas a word in a footnote might be an auxiliary detail. To exploit this asymmetry, Ying Liu et al. developed TableRank, demonstrating that treating metadata elements like captions as a distinct, high-density corpus yields far more precise table retrieval than searching flat text.
This principle extends naturally to web documents and semi-structured data. As Wen-tau Yih demonstrated, integrating structural metadata like HTML tags or document zones directly into learned term-weighting functions allows models to automatically discover that terms in titles or URLs are stronger indicators of core intent than body text. When documents are explicitly hierarchical, such as XML files, the challenge becomes aggregating this distributed evidence. Paul Ogilvie et al. addressed this by modeling how relevance probabilities propagate up from nested child nodes to parent containers, ensuring that the structural tree preserves informational value.
Exploiting Field Weights in Domain-Specific Search
The utility of structural weighting becomes particularly evident in specialized domains like expert search and enterprise retrieval. In an organization, finding a subject-matter expert requires looking beyond simple keyword proximity. Zhao Ru et al. demonstrated that weighting terms based on email fields (such as author, date, and title) significantly enhances known-item retrieval. In subsequent work, Zhao Ru et al. introduced "experience molecules," scaling term importance based on HTML headers and proximity to a candidate's name to better capture authoritative context.
Similarly, Qi Zhou 0001 et al. and Shenghua Bao et al. established that the structural location of a person's name—such as its placement within a technical section or under a specific semantic block header—is a far more reliable signal of expertise than simple text windows.
Beyond expert search, field weighting allows systems to prioritize expert-curated metadata over noisy unstructured text. For instance, in medical document retrieval, Abdullah Ahmed et al. utilized field-weighted retrieval (BM25F) to assign higher weights to controlled vocabularies like MeSH terms, ensuring that high-signal, expert-assigned metadata dominates the relevance score over ambiguous summary text.
Hierarchical Query Outlines and Specificity
Structure is not exclusive to documents; queries themselves can be hierarchical. In Complex Answer Retrieval (CAR), a user might submit a structured outline consisting of a broad page title and deeply nested subheadings to find highly specific passages.
Standard retrieval models struggle here because broad terms and granular terms are blended together. To solve this, Robert Litschko et al. proposed a hierarchical weighting strategy where terms located deeper in the query outline receive progressively higher weights. The intuition is that deeper subheadings reflect increased specificity and descriptive value for targeted content retrieval, preventing the system from returning overly general results.
To capture these relationships mathematically, Andrew Yates et al. mapped query terms to binary vectors indicating their exact position in a heading chain (e.g., title vs. subheading), allowing neural ranking models to learn distinct matching logic for different structural roles. This distinction is crucial because, as Sean MacAvaney et al. discovered, query facets can be split into "structural" (generic headings like History) and "topical" (specific headings like Westward Expansion) components. By modeling these components independently, systems avoid being misled by low-utility, generic structural terms.
Finally, this hierarchical mapping applies effectively to software engineering artifacts. Alessandro Bozzon et al. showed that in web application model repositories, keywords located in high-level architectural components (modules or entities) are far more indicative of core functionality than minor descriptive fields. Similarly, in software traceability recovery, Hui Gao et al. adjusted term importance based on whether a word appeared in a high-level requirement title versus a secondary alternative flow, bridging the semantic gap between abstract requirements and concrete source code.
Key Foundational Work
TREC 2005 Enterprise Track Experiments at BUPT (2005) — contributed 2 concept elements to this direction.
Using Language Models for Flat Text Queries in XML Retrieval (2003) — contributed 1 concept elements to this direction.