Abstract
With the explosion of web information, measuring text correlation is essential for filtering redundant content in information flow services. Traditional methods typically represent text as high-dimensional vectors, which often fail to capture the core event-based structure of news. This paper proposes a passage-level event representation method that constructs an event connection graph from sentence-level extractions. By identifying key nodes as representatives, the method accurately calculates text correlation. Experimental results show that this unsupervised approach outperforms conventional similarity measures and remains robust in fine-grained correlation tasks.
Introduction
In the era of massive information flows, filtering redundant news from multiple sources is a major challenge. Traditional methods typically represent text as high-dimensional vectors (like TF-IDF or word embeddings) and calculate similarity using cosine distance. However, news articles are fundamentally about events—who did what, to whom, and where. Standard vector models often fail to capture this structural "eventness." While recent research has moved toward event extraction, most work remains at the sentence level. To accurately measure if two articles are correlated, we need a way to represent the entire passage's event structure rather than just looking at isolated sentences or keyword frequencies.
Methodology
The proposed method uses a multi-stage pipeline to transform raw text into a structured event representation:
- Key Sentence Selection: Using an enhanced TextRank algorithm, the model identifies the most important sentences in a document to filter out noise like background information or boilerplate text.
- Sentence-Level Event Extraction: For each key sentence, the model extracts events using the ZORE model, identifying a trigger word (usually a verb) and its associated elements (entities/nouns).
- Event Polygon Construction: Each sentence-level event is represented as a "polygon" where the trigger word is the central node connected to its elements.
- Passage-Level Graph Merging: The model merges these polygons into a Passage-Level Event Connection Graph. Connections are made if two events share the same entity or if their nodes have high semantic similarity (calculated via GloVe embeddings).
- Representative Selection: TextRank is applied to the final graph to rank nodes. The top-ranked trigger words, named entities, and nouns are selected as the "representatives" of the article.
- Correlation Calculation: To compare two articles, the model calculates the weighted average similarity between their respective sets of representatives.
Evaluation and Results
The researchers evaluated their approach on three datasets: Sogou News, ByteDance (Toutiao) News, and a manually annotated sports dataset designed to test fine-grained correlation.
- Performance vs. Baselines: On the Sogou dataset, the model achieved an F1 score of 0.83, significantly higher than TF-IDF (0.67) and other keyword-based measures.
- Robustness: While BERT-based models performed well on coarse categories, the graph-based approach was more robust on the manual dataset, where understanding specific "who-did-what" details was critical for determining correlation.
- Unsupervised Advantage: Unlike deep learning models that require massive labeled datasets, this graph-based approach is unsupervised, making it highly adaptable to different domains without retraining.
Conclusion
By prioritizing structural "eventness" over word frequency, the proposed model captures the "semantic backbone" of a story while ignoring noisy words that don't contribute to the core narrative. Merging sentence-level events allows the model to stitch together the story's timeline and participants across the entire article. Future work will explore integrating deep learning embeddings directly into the graph construction process and applying the method to multi-document summarization.
Summary updated