A Neural Model for Joint Event Detection and Summarization

Zhongqing Wang, Yue ZhangIJCAI 2017 · DOI

Extracting meaningful information from the chaotic stream of Twitter during breaking news events is a significant challenge in natural language processing. Traditional approaches rely on rigid "pipeline" architectures that process data in isolated stages. This paper, presented at IJCAI-17, introduces JEDS (Joint Event Detection and Summarization), a neural model that demonstrates how filtering noise, grouping tweets, and summarizing events can be learned together to improve overall performance.

The Problem: Pipeline Propagation Errors

Detecting and reporting events on Twitter typically involves three distinct sub-tasks:

  1. Filtering: Identifying relevant event-related tweets while discarding mundane chatter.
  2. Clustering: Grouping related tweets into specific event "buckets."
  3. Summarization: Selecting the most informative tweets to represent each cluster.

In a traditional pipeline, the output of the filter feeds the clusterer, which then feeds the summarizer. This structure suffers from two major flaws. First, errors in early stages—such as a false positive in filtering—propagate and degrade the performance of later stages. Second, these tasks are naturally interdependent; a tweet that serves as an excellent summary is, by definition, highly relevant to the event. Standard pipelines cannot exploit these bi-directional relationships.

The JEDS Architecture

The core innovation of JEDS is its ability to allow these three tasks to inform one another through shared representations and "neural stacking."

The JEDS architecture featuring neural stacking across detection, clustering, and summarization tasks.

Shared Tweet Representation

Every tweet is first processed by a Long Short-Term Memory (LSTM) network to create a dense vector ($H$). This vector captures the semantic meaning of the tweet and serves as the common foundation for all subsequent tasks.

Neural Stacking and Joint Training

Instead of passing raw data between modules, JEDS passes features. The hidden layer of the Mention Detection (filtering) task is fed as an input to the Clustering task. Similarly, clustering features are fed into the Summarization network. Because the entire stack is differentiable, the model can back-propagate errors from the summarization stage all the way back to the filtering stage. This allows the model to learn that if a tweet is "summarizable," it is likely highly relevant.

For clustering, the model employs a Siamese Network to calculate similarity between tweets, determining whether a new tweet should join an existing cluster or initiate a new one.

Experimental Setup and Datasets

To evaluate the model, the authors collected and annotated two new datasets using the Twitter streaming API:

  • Earthquake Domain: 47 events and ~12,000 tweets.
  • DDoS Attack Domain: 170 events and ~17,000 tweets.

Data was manually annotated based on NIST Topic Detection and Tracking (TDT) rules to ensure high-quality ground truth for both event detection and summarization quality.

Key Results

JEDS consistently outperformed both traditional discrete pipelines and independent neural pipelines across multiple metrics. The joint training approach proved particularly effective at reducing the error rate in clustering while maintaining high summarization quality.

Performance comparison of JEDS against various LSTM-based pipeline and joint learning baselines.
  • Clustering Accuracy: Measured by $C_{min}$ (where lower is better), JEDS achieved 45.8 on the Earthquake dataset, significantly outperforming the popular AS12 stream-clustering model (60.9) and Locality Sensitive Hashing (66.7).
  • Summarization Quality: Measured by ROUGE-1, JEDS scored 21.3, outperforming the state-of-the-art extractive summarizer LexRank (18.8) when used in a pipeline configuration.

Insights and Future Work

A critical finding of this research is the "Summarization Feedback" effect: training the model to recognize what an informative summary looks like actually improves its ability to filter out irrelevant noise in the initial detection phase. Furthermore, the study confirmed that neural representations (LSTMs) are superior to traditional "bag-of-words" features (TF-IDF) for short, slang-heavy social media text.

Despite these gains, the model remains purely extractive, meaning it selects existing tweets rather than generating original summary text (abstractive summarization). Future work may explore integrating abstractive components and moving toward "open-domain" detection that does not rely on pre-defined keywords.

Summary updated

BibTeX
@inproceedings{wang2017neural,
  author = {Zhongqing Wang and Yue Zhang},
  title = {{A Neural Model for Joint Event Detection and Summarization}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2017},
  doi = {10.24963/ijcai.2017/581},
}
Figure