Document Architecture and Positional Features for Discourse Modeling

48 concepts40 papers1 proposal

When humans write, we do not simply generate a flat sequence of words. Instead, we follow highly predictable structural blueprints. A research paper almost always begins with an abstract and ends with a bibliography. A product review frequently places its summary verdict at the very beginning or the absolute end of a paragraph. An online forum user typically describes their problem context before concluding with a direct question or request for help.

Traditional natural language processing models often rely heavily on lexical content—the specific words chosen by an author. However, a parallel line of research demonstrates that a text segment's relative position within a document's architecture serves as an incredibly powerful, non-semantic proxy for its rhetorical function and discourse role. By engineering features that capture where a sentence, token, or citation lives within a document, paragraph, or section, models can infer structural intent with remarkable accuracy, even when the underlying text is noisy, informal, or completely unannotated.

The Intuition of Positional Consistency

Why do positional features work so well? The core reason is positional consistency: across a specific genre or document type, functional components consistently cluster at predictable relative intervals. Rather than relying on complex semantic understanding, a model can use a text segment's physical location as a strong prior probability for its communicative intent.

For instance, in document classification, Cornelia Caragea et al. demonstrated in Document Type Classification in Online Digital Libraries that tracking the spatial position of specific section headers (like an abstract or references section) acts as a primary skeleton for distinguishing research papers from theses, slides, or CVs. This structural consistency is so pronounced that even individual words can act as architectural anchors. As Alex Waibel et al. showed in Measuring the Structural Importance through Rhetorical Structure Index, words that appear with high positional consistency across different documents serve as highly effective structural markers for segmenting text, often outperforming rare, topic-specific keywords.

This principle applies directly to localized discourse elements:

Multi-Scale and Hierarchical Layout Features

Document architecture is inherently hierarchical. A word resides within a sentence, a sentence within a paragraph, a paragraph within a section, and a section within an entire document. Effective positional feature engineering captures these multi-level spatial signals simultaneously to map complex discourse relationships.

In text-level discourse parsing, Sujian Li et al. demonstrated in Text-level Discourse Dependency Parsing that multi-level positional features encoding a text unit's location across sentences, paragraphs, and the overall document layout provide critical constraints for predicting rhetorical links and identifying structural heads. Similarly, for essay scoring and discourse element identification, Wei Song et al. developed a hierarchical approach in Discourse Self-Attention for Discourse Element Identification in Argumentative Student Essays, showing that combining a sentence's localized paragraph position with its global document position significantly improves the identification of functional writing elements.

Even at the token level, spatial context alters meaning. Fahd Alotaibi et al. introduced structural context tagging in Mapping Arabic Wikipedia into the Named Entities Taxonomy, appending unique identifiers to tokens based on whether they appeared in titles, introductory sentences, or infoboxes. This allowed the classification model to weigh the semantic importance of a term differently depending on its architectural placement, capturing critical context that raw bag-of-words models overlook.

A Robust Shield Against Lexical Noise

One of the greatest advantages of positional features is their resilience. Lexical features are highly prone to noise: slang, spelling errors, domain shifts, or conversational fluff can easily confuse a purely word-based model. Positional features, by contrast, rely on the structural skeleton of the discourse, which remains remarkably stable across diverse contexts.

In community forums where text is highly unstructured, Alberto Barrón-Cedeño et al. found in Selecting Sentences versus Selecting Tree Constituents for Automatic Question Ranking that relative sentence position serves as an excellent heuristic to filter out greetings or sign-offs and isolate the core question intent. When tracking conversational dynamics on Wikipedia talk pages, Emily Jamison et al. showed in Adjacency Pair Recognition in Wikipedia Discussions using Lexical Pairs that incorporating a sentence's proximity to the beginning or end of a turn provides vital context for identifying reply-to relations without needing complex dialogue typologies.

This structural signal is so potent that it can even bypass text entirely or capture abstract document-level properties:

  • Acoustic Summarization: In Speech Summarization Without Lexical Features for Mandarin Broadcast News, Jian Zhang 0002 et al. showed that combining acoustic signals with purely structural cues—like segment position—could match the summarization performance of models utilizing full textual transcripts, proving that information density is deeply tied to structural layout.
  • Authorship Attribution: In Local Histograms of Character N-grams for Authorship Attribution, Hugo Jair Escalante et al. demonstrated that authors exhibit localized stylistic consistency. By comparing text segments at corresponding relative positions across documents, their approach achieved high accuracy even when training data was heavily imbalanced.
  • Media Bias Detection: In Detecting Media Bias in News Articles using Gaussian Bias Distributions, Wei-Fan Chen 0001 et al. demonstrated that the frequency and relative position of specific sentence-level attributes serve as stronger indicators of media bias than the exact sequential order of those sentences, using probabilistic models to capture these positional distributions.

Decoding Information Structure in Formal Literature

The utility of document-level positional features is perhaps most apparent in formal scientific and technical literature, where structural conventions are highly standardized.

When analyzing scientific abstracts, Anna Korhonen et al. observed in Identifying the Information Structure of Scientific Abstracts: An Investigation of Three Different Schemes that the normalized position of a sentence acts as a primary feature for identifying its rhetorical function (e.g., separating background from conclusions) due to the highly predictable narrative flow of condensed summaries. For full-text articles, Yufan Guo demonstrated in Improved Information Structure Analysis of Scientific Documents Through Discourse and Lexical Constraints that embedding structural discourse constraints—specifically a sentence's relative position within a section—allows models to accurately classify rhetorical roles even when labeled training data is scarce. This structural context also influences citation analysis; Simone Teufel et al. showed in Automatic classification of citation function that the physical location of a reference within the document's hierarchy provides a strong prior probability for the citation's rhetorical function.

Modern summarization and classification architectures exploit this property to handle long documents efficiently. For example, Wei Liu et al. in CIST@CL-SciSumm 2020, LongSumm 2020: Automatic Scientific Document Summarization utilized relative section indices and sentence offsets as core features to map semantic intent. Similarly, Chengzhi Zhang et al. in Enhancing Identification of Structure Function of Academic Articles Using Contextual Information used the normalized location of a text segment as a non-semantic feature to identify the functional role of chapters.

To bypass the strict memory limits of modern transformer models, Peng Cui et al. developed the Sliding Selector Network in Sliding Selector Network with Dynamic Memory for Extractive Summarization of Long Documents, proving that models can effectively learn the underlying discourse structure of long texts using window-level position embeddings alone, without requiring explicit structural metadata like section headers. By treating position as a foundational feature, these systems successfully navigate the macro-level architecture of human language.

Key Foundational Work

Towards Automatic Classification of Discourse Elements in Essays (2001) — contributed 1 concept elements to this direction.
Extracting Personal Names from Email: Applying Named Entity Recognition to Informal Text (2005) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 40 papers
  2. Version 1 · 2026-10-04 · 35 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_positional_feature_encoding_9cd1573a,
  title = {positional feature encoding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/positional-feature-encoding/rdr_9cd1573a8ebb4068a4e4840aa7f15161},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_positional_feature_encoding_9cd1573a_v2,
  title = {positional feature encoding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/positional-feature-encoding/rdr_9cd1573a8ebb4068a4e4840aa7f15161/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake