Abstract
Identifying the salience (i.e. importance) of discourse units is an important task in language understanding. While events play important roles in text documents, little research exists on analyzing their saliency status. This paper empirically studies the Event Salience task and proposes two salience detection models based on content similarities and discourse relations. The first is a feature based salience model that incorporates similarities among discourse units. The second is a neural model that captures more complex relations between discourse units. Tested on our new large-scale event salience corpus, both methods significantly outperform the strong frequency baseline, while our neural model further improves the feature based one by a large margin. Our analyses demonstrate that our neural model captures interesting connections between salience and discourse unit relations (e.g., scripts and frame structures).
Introduction
In Natural Language Processing (NLP), significant progress has been made in identifying "what" happened (event extraction) and "who" was involved (entity extraction). However, determining the relative importance of these events—their salience—remains a challenge. For instance, in a legal news report, a "trial" is a central, salient event, whereas a passing mention of a distant "war" serves as background context. This research addresses the task of Event Salience Identification, aiming to automatically rank events within a document according to their narrative importance.
The Event Salience Corpus
To address the lack of large-scale, high-quality data for this task, the authors introduced the Event Salience Corpus. Derived from the Annotated New York Times Corpus, it leverages human-written abstracts as a ground truth: if an event mentioned in the full article also appears in the abstract, it is labeled as salient. This approach provides a massive, objective dataset for training and evaluation.
As shown in the dataset statistics, the corpus includes over 526,000 training documents, representing a significant scale-up from previous event-related datasets which typically numbered in the hundreds or low thousands.
Methodology
The paper proposes two primary modeling approaches to identify salience:
1. Feature-Based Model (LeToR)
This model utilizes a "Learning to Rank" (LeToR) framework to combine several intuitive features:
- Basic Discourse: Incorporates the frequency of the event and its location (e.g., events appearing earlier in an article are often more salient).
- Content Voting: Measures the average lexical similarity (via word embeddings) between an event and all other events and entities in the document. This captures how well an event aligns with the overall document theme.
2. Kernel-based Centrality Estimation (KCE)
The KCE model is a neural approach that moves beyond simple averaging. It employs Gaussian kernels to group interactions into "bins," allowing the model to learn that specific levels of similarity are more indicative of salience than others. By projecting events and entities into an embedding space, KCE learns to weight latent relations—such as narrative scripts or frames—that contribute to an event's centrality. This model is trained end-to-end, allowing embeddings to adapt specifically to the salience task.
Experimental Results
The models were evaluated on their ability to rank salient events at the top of a document's event list. The KCE model demonstrated superior performance across all primary metrics.
As indicated in the results, KCE achieved a Precision at 1 (P@01) of 0.5049, a substantial improvement over the 0.4542 achieved by the frequency baseline. Furthermore, the integration of entity information consistently boosted performance, confirming that events and their participants are inextricably linked in determining narrative importance.
Key Insights and Discussion
- Salience is Relational: Salient events are not just frequent; they form "tightly related groups" or script-like chains (e.g., arrest → charge → trial).
- Non-Linear Similarity: Analysis of the KCE model revealed that the highest weights were not always assigned to the most similar words. Mid-range similarities often helped the model distinguish the primary plot from background details.
- Cohesion and the Intrusion Test: To validate the model's capture of document cohesion, the authors performed an "intrusion test" by mixing events from different stories. KCE effectively identified "intruder" events because they lacked the tight relational web characteristic of the original story's salient events.
Conclusion
This work establishes a robust foundation for event salience identification by providing both a massive benchmark corpus and a high-performing neural model. By capturing complex discourse relations through kernel-based centrality, the KCE model significantly advances the state-of-the-art beyond traditional frequency-based methods. Future work may explore salience as a continuous spectrum rather than a binary classification and investigate the automatic discovery of narrative frames through salient event clusters.
Summary updated