Abstract
This paper describes a fully implemented system for fusing related news stories into a single comprehensive description of an event. The basic components and the underlying algorithm are explained. The system uses a computationally feasible and robust notion of entailment for comparing information stemming from different documents. We discuss the issue of evaluating document fusion and provide some preliminary results.
Introduction
In an era of 24-hour news cycles, readers often consume multiple articles about the same event to ensure no detail is missed. While multi-document summarization typically provides a "highlight reel" by identifying common points, this research proposes a different approach: Document Fusion. Unlike summarization, which focuses on the intersection of information, Document Fusion aims for the union of all information. The goal is to create a single, comprehensive narrative that contains every unique detail from multiple sources while stripping away redundant content.
The Fusion Challenge
When a major event breaks, various news agencies provide core facts mixed with unique background information and different interviews. Standard text retrieval provides a list of these documents, but reading them all is laborious due to high redundancy. Conversely, standard summarization often discards the "long tail" of details—such as historical context or niche quotes—in favor of brevity. This system addresses the challenge of comprehensive event description: merging stories into a single document that is as concise as possible while retaining all information found in the original set.
The Fusion Pipeline
The implemented system operates through a five-step pipeline designed for robustness and narrative flow:
- Segmentation: Documents are broken into paragraphs rather than sentences. Paragraphs are more "context-complete," making them easier to compare and relocate without losing meaning.
- Computing Entailment: To avoid redundancy, the system determines if one paragraph entails another. A "shallow entailment score" is used, based on weighted word overlap. Words are weighted using Inverse Document Frequency (IDF), ensuring that rare, content-heavy terms carry more weight than common function words.
- Selecting a Base: To maintain narrative coherence, the system selects the longest document in the set to serve as the "narrative skeleton."
- Replacement: The system evaluates the base document and checks if any paragraph from another source is more informative (i.e., it entails the original but adds additional detail). If found, the paragraphs are swapped.
- Insertion: Paragraphs from other sources containing entirely new information (indicated by low similarity to the current draft) are identified and slotted into the text adjacent to the most related existing content.
Experimental Setup and Evaluation
The study utilized a test collection of 69 news stories categorized into 21 topics from sources like the L.A. Times and major wire services. To evaluate the informativity logic, a manually annotated dataset was created where humans judged the entailment relations between 12,083 pairs of text segments.
Key Results
The system demonstrated significant efficiency in condensing information. Fused documents were, on average, 55% the size of the total concatenated text of all sources, representing a substantial reduction in reading time without the loss of unique information.
Performance analysis showed that the system reached its optimal balance of precision and recall at an entailment (subsumption) threshold of approximately 0.2, as seen in the F-Score peaks. Furthermore, using an extrinsic evaluation via ad-hoc retrieval, the fused documents were found to be roughly 88.5% as "complete" as the full, redundant set of original documents.
Discussion and Limitations
While the system effectively shifts the focus from intersection to union, several challenges remain:
- Dangling Anaphora: Extracting paragraphs can cause pronouns (e.g., "he" or "the negotiations") to lose their referents. While the system uses TextTiling to group related segments, it lacks a full anaphora resolution component.
- Temporal Ordering: The system relies on content similarity for insertion rather than a chronological understanding, which can occasionally disrupt the natural flow of the event timeline.
- Shallow Parsing: Future iterations could improve entailment scores by moving beyond "bags of words" to analyze the relational structure of sentences (identifying agents and actions).
Conclusion
Document Fusion provides a robust proxy for logical entailment in news text using weighted term overlap. By starting with a "base document" strategy, the system preserves natural information flow while achieving significant compression, offering a promising path toward automated, comprehensive event reporting.
Summary updated