Extracting Military Event Temporal Relations via Relative Event Time Prediction and Virtual Adversarial Training

Jie Gong, Qiwang HuNAACL 2025 · DOI

Understanding the chronological sequence of military operations is a high-stakes task in intelligence analysis. At NAACL 2025, researchers introduced a specialized approach to Temporal Relation Extraction (TRE) that combines numerical time prediction with adversarial training. Their model, MFRV (Military Field Relative Virtual), demonstrates that domain-specific architectures still hold a significant edge over general-purpose large language models (LLMs) in complex reasoning tasks.

The Challenge of Military Temporal Relations

Temporal Relation Extraction involves determining the order of events—deciding if Event A happened before, after, or simultaneously with Event B. In the military domain, the stakes are uniquely high; an incorrect timeline in an intelligence report can lead to flawed strategic analysis or poor decision-making during crises.

Existing NLP models often struggle with military text due to its information density, long timeframes, and "fuzzy" temporal boundaries. Furthermore, general-domain datasets (like news) fail to capture the specific dynamics of military conflicts. Current models often compare event pairs directly without sufficient contextual "common sense" or the ability to generalize to the noisy language of military reporting.

Methodology: The MFRV Model

The authors propose MFRV, which utilizes a RoBERTa-base backbone enhanced by three key innovations designed to improve both precision and robustness.

The MFRV architecture integrates RoBERTa embeddings with relative time prediction and temporal commonsense knowledge.

1. Relative Event Time Prediction

Instead of treating "Before" or "After" as simple discrete labels, MFRV maps events to a continuous numerical timeline with values between -1 and 1. By learning to predict these relative timestamps as an auxiliary task, the model develops a more nuanced understanding of the distance and sequence between events. Ablation studies confirmed that this feature provides the strongest training signal for the model.

2. Virtual Adversarial Training (VAT) and MoCo

To improve generalization, the researchers apply small, calculated perturbations to the input data during training. This Virtual Adversarial Training forces the model to remain consistent even when phrasing is slightly altered.

The VAT and MoCo framework balances classification accuracy with adversarial robustness.

Training a model with multiple objectives—classifying relations, predicting relative time, and resisting noise—can lead to "gradient bias," where one task dominates the learning process. MFRV employs Multi-Objective Gradient Correction (MoCo) to dynamically balance these competing losses, ensuring stable and effective multi-task learning.

3. Temporal Commonsense Integration

The model integrates TEMPROB, a database of temporal logic (e.g., knowing that an "attack" usually precedes a "surrender"), providing a layer of external knowledge that helps resolve ambiguous event sequences.

Evaluation and Results

The study evaluates MFRV across three datasets, including a newly introduced specialized benchmark:

  • TRMF (Temporal Relations in the Military Field): A new dataset containing over 50,000 annotated event pairs from reports on the Russia-Ukraine war.
  • MATRES: A standard general-domain benchmark.
  • TCR: A refined dataset for cross-validation.

Key Performance Metrics

MFRV achieved state-of-the-art results, reaching an F1 score of 76.3 on the specialized TRMF dataset and 83.4 on the MATRES benchmark, surpassing previous leaders like SDLG.

A significant finding was the model's superiority over LLMs. When tested on the military-specific TRMF dataset, MFRV's F1 score of 76.3 far exceeded ChatGPT's score of 47.1. This highlights that for niche, high-precision fields, supervised models tuned with domain knowledge remain superior to zero-shot general-purpose AI.

Limitations and Future Directions

Despite its success, the model faces challenges with "EQUAL" and "VAGUE" relations, which are rare in training data and often context-dependent. Future work will focus on data augmentation for these classes. Additionally, the authors plan to move from local pair-wise logic to "global timeline construction" to ensure that document-wide event chains remain logically consistent (e.g., ensuring that if A < B and B < C, then A < C).

Summary updated

BibTeX
@inproceedings{gong2025extracting,
  author = {Jie Gong and Qiwang Hu},
  title = {{Extracting Military Event Temporal Relations via Relative Event Time Prediction and Virtual Adversarial Training}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  doi = {10.18653/v1/2025.findings-naacl.182},
}
Figure