Imagine reading a short news report and extracting three chronological facts: Event A happened before Event B, Event B happened before Event C, and Event C happened before Event A. To a human reader, this is an obvious physical impossibility—a temporal loop. Yet, standard natural language processing models fall into these traps constantly.
Why does this happen? Traditional systems treat temporal relation extraction as a series of isolated, local pairwise classification tasks. As noted by Sebastian Riedel 0001 et al. in Jointly Identifying Temporal Relations with Markov Logic, evaluating pairs of events in isolation lacks any mechanism to ensure that individual predictions align with a coherent chronological timeline. This local focus scales quadratically with the number of events, making it computationally expensive for long documents (Hayley Ross et al. in Exploring Contextualized Neural Language Models for Temporal Dependency Parsing), and frequently results in logical paradoxes.
Even modern deep learning architectures and large language models are highly susceptible to this flaw. For instance, Onkar Arun Pandit et al. demonstrate in Learning Rich Event Representations and Interactions for Temporal Relation Classification that advanced neural networks still produce globally incoherent timelines when operating purely on local pairs. Similarly, zero-shot evaluations of large language models by Chenhan Yuan et al. in Zero-shot Temporal Relation Extraction with ChatGPT reveal that while these models can identify rare local relations, they suffer from severe "temporal hallucinations" and logical contradictions, such as asserting a relationship in one query framing but failing to recognize its inverse in another.
To resolve this, a prominent research direction focuses on global temporal consistency constraints—algorithmic and architectural frameworks that force local predictions to adhere to the rigid, linear physics of time.
Bridging Local Evidence and Global Constraints via Optimization
To build a valid timeline, a system must enforce the mathematical properties of time—primarily transitivity ($A \prec B \land B \prec C \implies A \prec C$), symmetry, and mutual exclusivity. The dominant paradigm for achieving this is to overlay local classifiers with a global optimization framework, most notably Integer Linear Programming (ILP).
Instead of accepting the raw, independent outputs of local classifiers, an ILP framework treats them as soft preferences or weights. It then solves an optimization problem to maximize the overall confidence of the timeline subject to hard logical constraints.
Philip Bramsen et al. pioneered this approach in Inducing Temporal Graphs, using ILP to map text into Directed Acyclic Graphs (TDAGs) and ensuring that transitivity and connectivity are strictly maintained. This global correction layer fixes errors where local classifiers made contradictory choices. Similarly, Quang Do et al. in Joint Inference for Event Timeline Construction showed that enforcing transitivity and event-time alignment via ILP prevents cyclic orderings and significantly improves timeline accuracy over local baselines. This optimization is not limited to text documents; Tianyu Liu et al. demonstrate in Towards Time-Aware Knowledge Graph Completion that applying ILP to enforce temporal disjointness and ordering constraints over embedding-based plausibility scores improves relationship prediction precision by filtering out locally likely but globally impossible facts (such as employment occurring after an individual's death).
Beyond correcting errors, global constraints act as a powerful mechanism for incidental supervision and recall expansion. As Xiao Ling argues in Temporal Information Extraction, enforcing global transitivity allows a system to infer missing temporal links that are never explicitly stated in the text, drastically increasing recall. This property allows models to harvest supervision from partially annotated data; Ning Qiang et al. in Exploiting Partially Annotated Data in Temporal Relation Extraction used an ILP transitivity engine to automatically fill in missing labels during a bootstrapping process, bypassing the quadratic explosion of manual annotation.
Soft Logic and Multi-Dimensional Coherence
While hard constraints like ILP are highly effective, time in natural language is often probabilistic or multi-dimensional, involving not just order, but also duration and frequency. To handle this complexity, researchers have turned to probabilistic logic frameworks like Markov Logic Networks (MLNs) and Probabilistic Soft Logic (PSL).
In Jointly Identifying Temporal Relations with Markov Logic, the authors use logical formulas to link disparate local predictions into a coherent joint model, showing that document-level constraints provide the highest gains where local linguistic cues are weak. More recently, Yichao Zhou et al. in Clinical Temporal Relation Extraction with Probabilistic Soft Logic Regularization and Global Inference introduced a framework that turns transitivity rules into a differentiable loss function using PSL. This allows a neural network (like BERT) to be regularized by logical constraints directly during training, rather than relying solely on post-hoc inference pruning.
Furthermore, time is multi-dimensional. An event cannot have a duration longer than the time window in which it occurs. Bibo Cai et al. leverage this intuition in Mitigating Reporting Bias in Semi-supervised Temporal Commonsense Inference with Probabilistic Soft Logic. By using PSL to enforce logical dependencies across duration, frequency, and relative order, their model can triangulate missing commonsense information (e.g., realizing a short sub-event must fit inside a short main event), preventing the physically impossible combinations that independent multi-task models often produce.
Alternative Paradigms: Eliminating Pairs and Grounding in Physics
Instead of patching a broken pairwise classification system with post-hoc constraints, can we design architectures that are inherently consistent?
One prominent alternative is Direct Timeline Projection. In Temporal Information Extraction by Predicting Relative Time-lines, Artuur Leeuwenberg et al. propose a shift away from pairwise relations entirely. Their model directly predicts a linear relative timeline by assigning start points and durations to every event. Because every event is mapped to a continuous coordinate space, the resulting timeline is mathematically guaranteed to be free of loops or transitivity violations, while simultaneously reducing computational complexity from quadratic to linear.
A related structural approach involves decomposing intervals into endpoints. Chong Min Lee in Temporal Relation Identification with Endpoints breaks complex interval relations down into simpler pointwise comparisons (e.g., Start A vs End B). While independent endpoint classifiers can still produce impossible configurations (like a start time occurring after an end time), applying heuristic restoration or joint modeling ensures structural integrity.
Alternatively, neural architectures can be designed to maintain a global state internally. Yuanliang Meng et al. in Context-Aware Neural Model for Temporal Information Extraction introduce a Global Context Layer via external memory to resolve long-distance dependencies and logical inconsistencies during the initial extraction phase, eliminating the need for ad-hoc graph pruning.
Finally, temporal consistency can be reinforced by anchoring it to causality. Physical laws dictate that a cause must precede its effect. Qiang Ning et al. demonstrate in Joint Reasoning for Temporal and Causal Relations that optimizing temporal and causal relations simultaneously via ILP forces the timeline to respect causal physics. This insight is extended by Mahnaz Koupaee et al. in Causal Graph based Event Reasoning using Semantic Relation Experts, where constructing a global causal graph acts as a structural constraint that completely eliminates transitive incoherence across long narratives.