A New Method for Calculating Similarity between Sentences and Application on Automatic Abstracting

Wenqian Ji, Zhoujun Li, Wen-Han Chao, Xiaoming Chen

Abstract

Accurate sentence similarity measurement is critical for effective automatic text summarization. Traditional methods often rely on narrow features like keyword overlap, failing to capture semantic nuances or structural importance. This paper proposes a hybrid similarity metric combining TF-IDF, semantic analysis via lexical databases, sentential form, and length factors. This metric is integrated into an improved LexRank algorithm that incorporates position-based scoring and redundancy filtering. Experimental results on English (DUC 2004) and Chinese (HIT) datasets show substantial gains in ROUGE scores and high robustness against document noise.

Introduction

Automatic text summarization aims to condense large volumes of information into concise, representative summaries. At the heart of this process is the challenge of determining "sentence similarity." If a system cannot accurately assess how related two sentences are, it will struggle to identify the most representative content or, conversely, select redundant sentences that convey the same information in different words.

Existing methods are often one-dimensional. Vector space models (TF-IDF) miss semantic connections between synonyms, while purely semantic or syntactic models can be computationally prohibitive or overlook the surface-level importance of specific terms. This research addresses these gaps by introducing a holistic similarity model and a resilient graph-based summarization framework.

Methodology

The proposed solution follows a structured pipeline from raw text to evaluation, as illustrated in the general workflow below.

General workflow of the automatic summarization system

1. The Hybrid Similarity Metric

Instead of relying on a single feature, the authors combine four distinct components to calculate similarity:

  • Keyword Feature (TF-IDF): A standard vector space model using cosine similarity to capture shared vocabulary.
  • Semantic Feature: Utilizes lexical databases—HowNet for Chinese and WordNet for English—to calculate similarity between words even when they are not identical (e.g., "vibrant" and "energetic").
  • Sentential Form: A binary check to determine if sentences match in "form" (e.g., both are affirmative or both are negative), preventing false positives between sentences with opposite meanings.
  • Sentence Length: A penalty/reward factor based on the length ratio of the two sentences to ensure structural comparability.

2. Improved LexRank Summarization

The system builds upon LexRank, a graph-based algorithm where sentences are nodes and similarity scores are edges. The authors enhanced this model by integrating several scoring heuristics to better identify high-value content.

Detailed process for calculating sentence weights using the improved LexRank approach

As shown in the weighting process, the final sentence score is a composite of:

  • LexRank Weight: Centrality within the document graph.
  • PositionScore: Higher weights assigned to the first sentence of a paragraph, which statistically carries more informational weight.
  • InforScore: Identification of "indicator phrases" (e.g., "in this paper," "the purpose of...") that signal summary-worthy content.
  • Redundancy Resolution: An iterative selection process that filters out high-ranking sentences if they are too similar to those already chosen for the summary.

Evaluation and Results

The system was evaluated across two distinct benchmarks to demonstrate cross-linguistic capability: the Chinese Single-Document Corpus (HIT) and the English Multi-Document Corpus (DUC 2004).

Key Findings

  • Quantitative Gains: The improved method showed clear improvements using the ROUGE metric. For the English DUC 2004 dataset, the system increased ROUGE-1 scores by 9.5% and ROUGE-2 scores by 16.3% compared to basic TF-IDF methods.
  • Cross-Linguistic Performance: In the Chinese HIT corpus, the improved method consistently outperformed both pure keyword and pure semantic models at various compression rates (10% and 20%).
  • Noise Tolerance: The graph-based centrality approach proved highly robust. In stress tests where up to 16.7% of the document cluster consisted of irrelevant "noise" documents, the summary quality remained stable.

Conclusion

By treating sentence similarity as a composite of keyword, semantic, structural, and length features, this approach provides a more nuanced foundation for automatic summarization. While the system excels at identifying core content, future work will focus on sentence ordering for multi-document summaries and sentence compression/fusion to further refine the conciseness of the output.

Summary updated

Abstract

Sentence similarity computing plays an important role in machine question-answering systems, machine-translation systems, information retrieval and automatic abstracting systems. This article firstly sums up several methods for calculating similarity between sentences, and brings out a new method which takes all factors into consideration including critical words, semantic information, sentential form and sen-tence length. And on this basis, a automatic abstracting system based on LexRank algorithm is implemented. We made several improvements in both sentence weight computing and redundancy resolution. The system described in this article could deal with single or multi-document summarization both in English and Chinese. With evaluations on two corpuses, our system could produce better summaries to a certain degree. We also show that our system is quite insensitive to the noise in the data that may result from an imperfect topical clustering of documents. And in the end, existing problem and the developing trend of automatic summariza-tion technology are discussed.
BibTeX
@article{ji2009new,
  author = {Wenqian Ji and Zhoujun Li and Wen-Han Chao and Xiaoming Chen},
  title = {{A New Method for Calculating Similarity between Sentences and Application on Automatic Abstracting}},
  journal = {Intell. Inf. Manag.},
  year = {2009},
  doi = {10.4236/iim.2009.11007},
}
Figure