R2A-TLS: Reflective Retrieval-Augmented Timeline Summarization with Causal-Semantic Integration

Chenlong Bao, Shijie Li, Minghao Hu, Ming Qiao, Bin Zhang, Jin-Tao Tang, Shasha Li, Ting WangEMNLP 2025

TL;DR

This paper introduces R2A-TLS, a novel framework for open-domain timeline summarization that tackles information overload and data sparsity by integrating retrieval, reflection, and deep retrieval. It uniquely addresses missing timeline events and details by analyzing causal chains and leveraging FrameNet semantics, achieving state-of-the-art results on the Open-TLS dataset.

What Problem Does This Paper Address?

Generating coherent and comprehensive timelines from vast, unstructured text streams (like news articles) is a significant challenge. Open-domain timeline summarization (TLS) specifically struggles with two main issues:

  1. Information Overload: The sheer volume of text makes it difficult to sift through relevant information and avoid noise. Existing retrieval strategies can be inefficient and accumulate irrelevant content.
  2. Data Sparsity: Even with retrieved information, crucial events or details might be missing, leading to fragmented narratives and incomplete timelines. This is exacerbated by the noisy nature of online news.

Current methods often tackle temporal modeling and event extraction separately, or rely on iterative questioning that can be computationally expensive and limited by initial retrieval quality. This paper aims to create a more robust and integrated approach to fill these gaps.

Methods

R2A-TLS employs a novel "retrieval-reflection-deep retrieval" closed-loop system. Here's a breakdown of its key components:

  • Initial Retrieval and Filtering:
    • The process starts with a search engine query for the target news.
  • The top 3 results are used to generate an initial timeline.
  • A dual-filtering mechanism is then applied to the remaining retrieved articles. This mechanism uses:
  • Topic Relevance: Checks if the article is directly related to the target news.
  • Information Gain: Assesses if the article provides new time nodes, key event progress, or adds details to existing timeline points.
  • Articles passing both filters are used to generate timeline fragments.
  • Reflection and Deep Retrieval:
    • Time Point Completion: The system analyzes the initial timeline for causal chain breaks. It uses a generic LLM to identify potential missing time points based on logical event relationships. These identified gaps are then reformulated into targeted queries for deep retrieval to fill these missing milestones.
  • Event Element Completion: This step leverages FrameNet semantics. An LLM recommends appropriate FrameNet frames for timeline events, identifying core event components (like participants, motivations). If core elements are missing, targeted questions are generated and rewritten to trigger deep retrieval for more detailed information.
  • Merge and Sort: Finally, the generated timeline fragments are merged and sorted chronologically to produce the final timeline.

Datasets

  • Open-TLS: This is the primary dataset used for training and evaluation. It's described as a typical dataset specifically crafted for open-domain TLS, containing 50 topics and 50 timelines. Each timeline consists of dates associated with sentences describing events.
  • Closed-domain Datasets (for fine-tuning): The paper mentions fine-tuning a domain-adapted LLM using a custom training set constructed from three annotated closed-domain timeline summarization datasets:
  • T17 (Binh Tran et al., 2013)
    • CRISIS (Tran et al., 2015)
    • ENTITIES (Gholipour Ghalandari and Ifrim, 2020)
  • Generalizability Datasets: To test generalizability, the model was evaluated on:
  • Event Storylines Corpus (ESC) (Caselli and Vossen, 2017)
    • HiEve (Glavaš et al., 2014)
  • Closed-domain Benchmarks (for comparison): Crisis and T17 were used for evaluating closed-domain TLS performance.

Key Insights and Ideas

  • Causal Chains are Crucial for Coherence: The paper strongly emphasizes that simply retrieving information isn't enough. Identifying and filling gaps in the causal chain of events is essential for creating a coherent timeline, especially when dealing with sparse data. This is why time point completion based on causal reasoning is a core component.
  • FrameNet Adds Semantic Depth: Beyond just temporal order, understanding the roles and components of events (via FrameNet) is vital for comprehensive summarization. This addresses the "missing details" problem effectively.
  • Iterative Reflection is Key: The "reflection" step, where the system actively looks for gaps and generates new queries, is what differentiates R2A-TLS. It's not just about retrieving more data, but about intelligently querying for specific missing pieces.
  • Dual-Filtering is More Effective: Separating topic relevance and information gain filtering into two distinct steps proved more effective than a single-step approach, leading to better performance.

Key Results

  • State-of-the-Art Performance: R2A-TLS achieved state-of-the-art (SOTA) results on the Open-TLS dataset, outperforming previous published approaches.
  • Quantitative Improvements:
    • A 5.1% absolute improvement in Date F1.
  • A 2.8% gain in Align F1 over the previous best-performing approach.
  • Ablation Studies Highlight Contributions:
    • Removing Time Point Completion (TPC) led to a substantial 6.6% absolute decline in Date F1.
  • Removing Event Element Completion (EEC) primarily impacted summary quality (ROUGE-N), with a 1.2% decrease.
  • The synergistic effect of both TPC and EEC was greater than the sum of their individual removals, proving their collaborative effectiveness.
  • Generalizability: The method showed good generalizability on domain-specific datasets (HiEve and ESC), achieving higher accuracy than a baseline fine-tuned directly on those datasets.

Limitations and Future Work

  • Computational Resources: The study acknowledges that significant computational resources are required, especially for LLM operations and search engine interactions.
  • Search Engine Dependency: The performance is influenced by the quality and capabilities of the search engine used. The paper notes that JINA, used for URL-to-text conversion, has systematic failures with dynamic content and complex layouts.
  • Prompt Engineering: The design of prompts significantly impacts model performance, requiring meticulous crafting.
  • Future Work: The authors suggest improving temporal reasoning abilities for LLMs, which is an area that has not yet received widespread attention. They also propose exploring the future trend of events from existing timelines.

Summary updated

BibTeX
@inproceedings{bao2025r,
  author = {Chenlong Bao and Shijie Li and Minghao Hu and Ming Qiao and Bin Zhang and Jin-Tao Tang and Shasha Li and Ting Wang},
  title = {{R2A-TLS: Reflective Retrieval-Augmented Timeline Summarization with Causal-Semantic Integration}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure