Identifying Topics by Position

Chin-Yew Lin, Eduard H. HovyANLP 1997 · DOI

TL;DR

This paper introduces an "Optimal Position Policy" (OPP) to identify topic-bearing sentences in texts by leveraging genre-specific discourse structures. By empirically determining which sentence positions are most likely to contain key topics, this method offers a robust and efficient way to improve applications like information retrieval and text summarization.

What Problem Does This Paper Address?

In our increasingly information-saturated world, efficiently identifying the main topics within documents is crucial. Traditional methods for topic identification often rely on complex parsing or semantic analysis, which can be brittle. While simpler methods like "Cue Phrase" and "Position" methods exist, they often lack accuracy or are not tailored to specific text genres. This paper tackles the challenge of systematically identifying topic-bearing sentences based on their position within a text, acknowledging that discourse structures vary significantly across genres and domains, and that a one-size-fits-all approach is insufficient.

Methods

The core of this paper is the development and evaluation of an Optimal Position Policy (OPP). The authors propose that texts within a genre exhibit predictable discourse structures, and that sentences carrying the most central topics tend to appear in specific locations.

The methodology involves:

  • Empirical Determination of Sentence Yield: They define "sentence yield" as the average number of unique topic keywords found within a sentence. This is calculated by matching keywords against sentences, with a specific "hit function" (H) that accounts for word overlap and uses a Fibonacci sequence to reward longer matches.
  • Training an Optimal Position Policy (OPP): Based on the empirically determined sentence yields, positions are ranked. The OPP is essentially a prioritized list of sentence positions (e.g., "title sentence," "second sentence of the first paragraph") that are most likely to contain topic information for a given genre.
  • Evaluation Metrics: The authors employ three key metrics to assess the effectiveness of the OPP:
  • Precision (Pm): Measures the proportion of selected windows (from the extracted text) that are also found in the human-generated abstract.
  • Recall (Rm): Measures the diversity of topic information covered in the extracted text compared to the abstract.
  • Coverage (Cm): Measures the proportion of sentences in the abstract that have at least one matching word or phrase with the extracted text.

Datasets

  • Training Data: Volume 1 of the Ziff-Davis newspaper text corpus, consisting of approximately 13,000 texts characterized as product announcements. Each text was accompanied by human-generated topic keywords and a short abstract.
  • Evaluation Data: A separate set of 2,907 texts from Volume 2 of the Ziff-Davis corpus, of the same nature and genre as Volume 1.

Key Insights and Ideas

  • Genre-Specific Discourse Matters: The paper strongly emphasizes that topic identification based on position must be tailored to the specific genre and domain of the text. A universal "position rule" is unlikely to be effective.
  • The Title is King (for this genre): For the product announcement genre studied, the title sentence was found to be exceptionally important, containing a significant portion of the topic keywords.
  • Early Sentences Dominate: Beyond the title, sentences in the early parts of paragraphs and texts generally carry more topic-relevant information than those at the end.
  • Last Sentence Hypothesis Debunked (for this genre): Contrary to some earlier hypotheses, the last sentence of a paragraph was not found to be particularly informative. Instead, the second-to-last sentence often held more weight, likely due to the prevalence of short paragraphs in the corpus.
  • Variable Window Sizes are Key: For matching text segments, using variable window sizes (rather than a fixed size) proved more effective in capturing topic relevance.

Key Results

  • The developed Optimal Position Policy (OPP) for the Ziff-Davis product announcement genre consistently identified high-yield sentences.
  • Evaluation I showed that an OPP derived from a test set was highly similar to the OPP derived from the training set, confirming the robustness of the position hypothesis for this genre.
  • Evaluation II demonstrated promising results:
  • Selecting the top 7 sentences (about 10% of an average text) according to the OPP achieved a precision of 0.38 and a recall of 0.35 (using window size 1).
  • Coverage scores were particularly high, with the top 10 OPP-selected sentences covering 95% of the abstract content when considering all window sizes. Even selecting just 7 sentences (about 10% of the text length) yielded a coverage score of 0.91.
  • The title sentence alone accounted for about 50% of the topic keywords.

Limitations and Future Work

  • Limited Genre Scope: The study focused on a specific genre (product announcements). The authors acknowledge that the derived OPP is genre-specific and would need to be re-derived for other genres.
  • Exact Matching: The matching process relied on exact word matches, without morphological normalization or anaphora resolution. This limitation likely reduced the scores and suggests that incorporating such techniques could further improve performance.
  • Goal-Oriented Search: The current evaluation assumes abstracts accurately represent the text's central topics. For goal-oriented topic search (where a user looks for specific, prespecified topics), a different method for deriving an OPP would be needed.
  • Augmentation Needed: The authors conclude that while the Position Method is powerful, it has limitations. To achieve even better results, it needs to be augmented with more precise techniques, such as those involving semantic analysis or other linguistic cues.

Summary updated

BibTeX
@inproceedings{lin1997identifying,
  author = {Chin-Yew Lin and Eduard H. Hovy},
  title = {{Identifying Topics by Position}},
  booktitle = {ANLP},
  year = {1997},
  doi = {10.3115/974557.974599},
}
Figure