A Proposal: Interactively Learning to Summarise Timelines by Reinforcement Learning

Yuxuan Ye, Edwin SimpsonACL 2021

TL;DR

This paper proposes a novel approach to automatically generate concise, time-ordered summaries of events from news articles, a task known as Timeline Summarization (TLS). By leveraging Reinforcement Learning (RL) and interactive user feedback, their method aims to create personalized timelines that adapt to user preferences without requiring pre-existing reference summaries.

What Problem Does This Paper Address?

Tracking the evolution of significant events over time, like the COVID-19 pandemic, can be overwhelming due to the sheer volume of news coverage. Timeline Summarization (TLS) aims to distill this information into digestible, chronological summaries. However, existing TLS methods often rely on extractive techniques that can include irrelevant information, or abstractive methods that struggle with the lack of reference summaries for supervised learning. Furthermore, current systems lack the flexibility to adapt to new domains or cater to individual user interests. This paper addresses these limitations by proposing an interactive, RL-based framework for abstractive TLS that learns from user feedback.

Methods

The proposed method integrates several key components:

  • Event Detection and Clustering: Source documents are embedded into dense vectors, and then clustered using Affinity Propagation. This clustering groups sentences related to the same sub-event. Constraints are added to the clustering to handle documents published far apart.
  • Date Assignment: Each cluster is assigned a date based on the most frequently mentioned date within its documents.
  • Abstractive Summarization: A pre-trained abstractive Multi-Document Summarization (MDS) model (specifically PEGASUS) is used to generate summaries for each identified sub-event cluster.
  • Reinforcement Learning (RL) for Fine-tuning: This is the core innovation. The system uses an RL agent to fine-tune the abstractive summarizer. The RL agent learns a reward function based on user feedback. This feedback can include preferences for keywords, dates, or comparisons between different timeline versions.
  • Compound Reward Function: The RL agent is guided by a reward function that comprises four sub-rewards: topical coherence with the source documents, consistency with user feedback (keywords to include/exclude, pairwise preferences), and linguistic fluency.
  • Interactive Learning Loop: The user previews the generated timeline and provides feedback. This feedback is used to update the reward function, which then fine-tunes the summarizer through RL. This process repeats over several rounds, allowing the model to adapt and tailor timelines to user demands.

Datasets

The paper plans to evaluate their framework on two benchmark TLS datasets:

  • Timeline17: (Binh Tran et al., 2013; Tran et al., 2013)
  • Crisis: (Tran et al., 2015)

The paper also mentions using a fine-tuned version of PEGASUS on Multi-News (Fabbri et al., 2019) for the abstractive summarization task.

Key Insights and Ideas

  • Bridging the Gap in Abstractive TLS: The paper directly tackles the challenge of limited reference data for abstractive TLS by proposing an interactive learning approach, which is more practical than collecting extensive human-annotated timelines.
  • Personalization through Interaction: The core idea is that user feedback, even in the form of simple preferences, can be effectively used to guide an RL agent to generate timelines that are not only accurate but also relevant to the specific user's needs.
  • Modular Reward for Comprehensive Quality: The use of a compound reward function that considers topical coherence, user feedback, and linguistic fluency is crucial for generating high-quality timelines. This multi-faceted approach aims to avoid the pitfalls of optimizing for a single metric.
  • Leveraging Pre-trained Models: By starting with a powerful pre-trained abstractive summarizer like PEGASUS, the RL fine-tuning process can be more efficient and effective, focusing on adapting the model to the specific TLS task and user preferences.

Key Results

The paper is a proposal and outlines planned experiments. However, it does present preliminary results comparing PEGASUS-Multi_News with CLUST on an abstractive summarization task for event clusters.

  • PEGASUS-Multi_News outperforms CLUST: On Alignment ROUGE-1 and ROUGE-2 metrics, PEGASUS-Multi_News achieved scores of 0.089 and 0.019 respectively, while CLUST scored 0.082 and 0.02. This suggests that PEGASUS provides a strong basis for abstractive summarization of event clusters.

Limitations and Future Work

  • Simulated vs. Real Users: The paper acknowledges that initial user feedback will be generated through simulations. While this allows for rapid iteration, the authors plan to evaluate with real users once a working system is developed, as simulated feedback may not fully capture human nuances.
  • Designing Interaction Modes: A key challenge identified is designing suitable modes of interaction that effectively elicit useful feedback from users.
  • RL Variance: The authors recognize the high variance issue in RL when sampling gradients and plan to address it by running many episodes and potentially using an actor-critic approach.
  • Experimental Evaluation: The paper outlines a plan for evaluation using Concatenation ROUGE and Alignment ROUGE metrics, as well as user studies. The actual experimental results are pending.

Summary updated

BibTeX
@inproceedings{ye2021proposal,
  author = {Yuxuan Ye and Edwin Simpson},
  title = {{A Proposal: Interactively Learning to Summarise Timelines by Reinforcement Learning}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure