TL;DR
This paper introduces a novel interactive system for abstractive summarization of event news tweets. It allows users to explore information through a hierarchical bullet-style summary and concept expansion, offering a more flexible and detailed understanding of events than traditional static summaries. Usability studies show this approach is highly valuable and preferred by users for exploring event-related information.
What Problem Does This Paper Address?
Traditional multi-document summarization (MDS) often focuses on creating a single, static summary that captures the most important information. While effective for a quick overview, this approach can be limiting when users need to delve deeper into the nuances of an event or explore specific details. Existing interactive summarization methods primarily rely on extractive techniques, which are constrained by the original sentence structures. This paper addresses the need for a more flexible and detailed interactive summarization system, specifically for event news tweets, by leveraging abstractive summarization to enable richer information presentation and exploration.
Methods
The core of the system is an abstractive summarization approach built upon a consolidated knowledge representation called the Open Knowledge Representation (OKR).
- Open Knowledge Representation (OKR): This is a structured representation of information from multiple texts. It captures propositions and entities, collapsing coreferring concepts and maintaining links to their original mentions. This allows for a more granular understanding of the information.
- Interactive Summarization Modes:
- Bullet-Style Summary: Information is presented as a hierarchical list of bullet points. The most important facts are shown first, and users can "unfold" additional bullets to reveal more specific details. This is particularly useful for timelines of events.
- Concept Expansion: Users can hover over a concept (like a person or event) to see alternative mentions of that concept across the summary, providing a richer understanding of entities and their roles.
- Summary Generation:
- Proposition Grouping: OKR propositions are grouped based on argument links.
- Representative Sentence Generation: For each group, candidate sentences are generated by filling templates with various mentions. A representative sentence is chosen based on a combination of language model score, knowledge score, and length.
- Metadata Generation: Each sentence is associated with a knowledge score (reflecting mention frequency and informativeness), concept expansions, and a timestamp.
Datasets
The paper utilizes a dataset of human-annotated OKR structures released by Wities et al. (2017). Specifically, they used the 6 largest clusters of event tweets from this dataset, each containing approximately 100 tweets. This dataset served as a gold-standard for evaluating their system in isolation.
Key Insights and Ideas
- Abstractive Summarization for Interactivity: The paper argues that abstractive summarization is inherently more suitable for interactive settings than extractive methods. By generating new sentences, it can present information at a more atomic level and combine facts flexibly, which is crucial for detailed exploration.
- Hierarchical Information Presentation: The bullet-style summary with unfolding capabilities allows users to control the level of detail they consume, moving from a high-level overview to specific facts as needed. This is a significant improvement over static summaries that present all information at once.
- Concept Expansion for Deeper Understanding: The ability to see all mentions of a concept (e.g., different names for a suspect) provides valuable context and helps users connect disparate pieces of information, especially in fast-moving news events.
- Usability is Key: The study highlights that while the interactive system might be slightly more complex than static baselines, its perceived usefulness, satisfaction, and knowledge exploration capabilities far outweigh this.
Key Results
- Usability Study 1 (Preliminary): Six students found the prototype easy to use and showed willingness to use it frequently. The SUS scores ranged from 70 to 95, indicating high overall usability. Qualitative feedback helped improve the prototype.
- Usability Study 2 (Comparative): The interactive abstractive summary (IAS) system was compared against a raw tweet list and a static summary. The IAS system consistently received the highest ranks in Usefulness, Satisfaction, and Knowledge Exploration, demonstrating the substantial value of interactivity.
Limitations and Future Work
- Limitations:
- The current system assumes an external "black-box" tool for parsing original texts into OKR data.
- The summarization sentence generation uses a baseline method, which could be further enhanced.
- The ordering of summary sentences is based on a timeline for news tweets, which might not be optimal for all domains.
- Future Work:
- Porting the interactive abstractive method to other domains and corpora.
- Developing more sophisticated machine learning techniques for selecting the best representative sentences.
- Designing dedicated quality tests for interactive summaries.
- Integrating the system with a fully automated OKR parser for end-to-end functionality.
- Conducting more extensive user studies with the automated pipeline.
Summary updated