When humans read a complex document, solve a multi-hop puzzle, or follow a recipe, we rarely process information in isolated, parallel chunks. Instead, our understanding accumulates step-by-step: every piece of information we uncover changes how we interpret the next.
Yet, traditional natural language processing models frequently treat structured text tasks as independent classification or ranking problems. Whether identifying coreference chains, linking entities, or retrieving supporting evidence from a corpus, standard architectures often score candidates in isolation. This parallel approach suffers from a fundamental flaw: it ignores the deep, non-linear dependencies inherent in language.
To overcome this limitation, a growing body of research reformulates these problems as sequential decision-making processes. By transitioning from static, parallel classification to dynamic path-finding, models can maintain an evolving state of context, capture synergistic relationships between disparate facts, and optimize for global coherence.
The Pitfall of Isolated Decisions
To understand why a sequential approach is necessary, consider the task of coreference resolution or entity linking. If a model attempts to resolve multiple ambiguous mentions in a document simultaneously, it cannot leverage its own downstream choices to fix upstream mistakes.
Parallel Approach (Isolated & Error-Prone):
Mention 1 ("Apple") --> [Independent Ranker] --> Company? Fruit?
Mention 2 ("Mac") --> [Independent Ranker] --> Computer? Raincoat?
Sequential Approach (Context-Aware):
Mention 1 ("Apple") --> Resolved as Company
↓ (Context Updated)
Mention 2 ("Mac") --> Highly conditioned on "Company" --> Resolved as Computer
By framing these tasks as sequential decision processes, models can preserve the history of their choices. For instance, in Chinese zero pronoun resolution, Yin et al. demonstrated that moving away from isolated local classifications to a sequential framework allows the system to preserve the history of the coreference chain. Each candidate antecedent choice is explicitly informed by previous decisions, transforming the task into a global optimization problem.
This strategy proves equally powerful for entity linking and phrase grounding. Chen et al. introduced a hierarchical reinforcement learning framework that models mention detection and entity disambiguation sequentially. This allows the system to utilize the context of previously linked entities to guide subsequent decisions, effectively capturing intra-document dependencies that parallel extraction models miss.
Similarly, in the visual-linguistic domain, Dogan et al. formulated phrase grounding as a sequential process where the model tracks a history of grounded entities. This state tracking enables the system to resolve spatial and relational dependencies that are typically lost in independent matching schemes. When an agent must execute actions based on language, as explored by Suhr et al., resolving these sequential dependencies across an interaction history and an evolving environment state becomes the core requirement for success.
Multi-Hop Reasoning and Synergistic Retrieval
The limitations of isolated processing are even more pronounced in multi-hop question answering and fact verification. If a question requires connecting Fact A to Fact B to find an answer, Fact B might appear completely irrelevant to the initial query when evaluated in isolation.
Query: "Where was the director of the movie Inception born?"
Isolated Retrieval:
- Documents about "Inception" (High Relevance)
- Documents about "Christopher Nolan's birthplace" (Low Relevance to the word "Inception")
Sequential Retrieval:
Step 1: Find "Inception" --> Identify director: Christopher Nolan
Step 2: Update Query with "Christopher Nolan" --> Find birthplace (High Relevance)
Research in multi-hop dense retrieval confirms that order matters. Xiong et al. showed that treating multiple pieces of evidence as an unordered set during training leads to a substantial loss in retrieval accuracy. By enforcing a specific order on retrieved document sequences, the encoder explicitly learns the logical flow of multi-hop reasoning—discovering how the information in one document necessitates the retrieval of the next.
To capture these interactions dynamically, Sun et al. reformulated evidence selection as a path-finding process within a tree structure using Monte Carlo Tree Search (MCTS). By treating the selection of an information unit as conditional on previously selected units, the model captures synergistic dependencies—where two sentences collectively provide strong evidence, even if either one alone seems irrelevant.
Eliminating Redundancy Generatively
Sequential modeling also provides a natural defense against information redundancy. When traditional systems rank sentences or documents independently, they often return highly repetitive top results.
By contrast, treating evidence discovery as a sequence generation problem allows models to remain aware of what they have already selected. Chen et al. leveraged this by training a Transformer to generate a single sequence of evidence identifiers for fact verification. Because the model generates these tokens autoregressively, it naturally avoids redundancy and selects a more coherent, consistent set of evidence.
Similarly, Nishida et al. treated evidence extraction as a sequential summarization task using an RNN with a "glimpse" attention mechanism. This architecture ensures that the relevance of subsequent evidence is always conditioned on the facts already accumulated in the reasoning chain.
Mapping the Logical Flow of Arguments
Beyond retrieval and entity resolution, sequential dependencies form the bedrock of structural text analysis, such as argumentation mining. Arguments are rarely isolated claims; they are multi-step reasoning chains where premises support intermediate claims, eventually culminating in a primary conclusion.
As Stab et al. demonstrated, identifying these sequential dependencies and discourse structures is critical for evaluating the logical flow of persuasive essays and scientific articles. Without a sequential understanding of how these components connect, automated systems cannot effectively evaluate or optimize the structural integrity of text.
Why the Sequential Paradigm Works
The transition from parallel classification to sequential decision-making works because it fundamentally aligns the model's architecture with the structure of human language and logic. The paradigm succeeds due to three core mechanisms:
- Dynamic Contextualization: Instead of relying on static embeddings, the representation of the problem space updates after every decision. Past choices serve as a lens that clarifies future ambiguities.
- Synergy Discovery: It enables the identification of "hidden" relevance, allowing models to discover pieces of information that are only valuable when paired with previously extracted facts.
- Global Optimization: By framing tasks as paths or sequences, models can optimize for the overall coherence of the output—whether it is a coreference chain, an evidence tree, or an argumentative flow—rather than making a series of locally optimal but globally disjointed guesses.
Key Foundational Work
Deep Reinforcement Learning for Chinese Zero Pronoun Resolution (2018) — contributed 2 concept elements to this direction.
Argumentation Mining in Persuasive Essays and Scientific Articles from the Discourse Structure Perspective (2014) — contributed 1 concept elements to this direction.
Situated Mapping of Sequential Instructions to Actions with Single-step Reward Observation (2018) — contributed 1 concept elements to this direction.