Multiple Alignment of Citation Sentences with Conditional Random Fields and Posterior Decoding

Ariel Schwartz, Anna Divoli, Marti HearstEMNLP 2007

TL;DR

This paper tackles the challenge of aligning equivalent concepts across multiple scientific citation sentences, a crucial step for tasks like summarization and entity extraction. They introduce a novel approach that combines Conditional Random Fields (CRFs) for pairwise alignment with a posterior decoding algorithm for multiple alignment, achieving better results than existing methods.

What Problem Does This Paper Address?

Scientific literature is a vast and growing resource, and effectively extracting information from it is key for researchers. Citation sentences, which describe how a paper relates to prior work, are particularly valuable. However, these sentences often express the same underlying facts or concepts using different wording. To leverage this information for applications like automatic summarization or identifying equivalent entities, we need to align these semantically similar phrases across multiple citation sentences. This "multiple citance alignment" (MCA) problem is analogous to multiple sequence alignment in biology but is more subjective due to the fuzzy nature of semantic similarity. Existing methods struggle to handle the many-to-many nature of these alignments and incorporate diverse features effectively.

Methods

The authors propose a novel MCA algorithm that builds upon two key components:

  • Pairwise Alignment with Conditional Random Fields (CRFs): They adapt the CRF-based pairwise word alignment model from Blunsom & Cohn (2006). CRFs are discriminative models that can incorporate a rich set of features (orthographic, Markov, positional, and ontological) to learn the probability of aligning words between two sentences. Unlike traditional HMMs, CRFs can handle arbitrary dependencies between features.
  • Posterior Decoding for Multiple Alignment: To extend pairwise alignment to multiple sentences, they employ a posterior decoding algorithm, inspired by Schwartz & Pachter (2007). This approach aims to maximize an expected utility function, allowing for more flexible control over the alignment process than traditional Viterbi decoding. They introduce "gap" and "match" factors to tune the recall/precision trade-off, enabling finer control over the alignment.

Datasets

The authors created their own dataset for this task. They focused on the domain of molecular interactions and manually annotated citances to six target papers. This resulted in:

  • Training set: 40 citances from 4 target papers (10 each).
  • Development set: 51 citances from a fifth target paper.
  • Test set: 45 citances from a sixth target paper.

Pairwise word alignments were generated from these annotated citance groups, yielding 180, 1275, and 990 alignments for training, development, and testing, respectively. They note that for training the many-to-one CRF model, true-positive target words beyond the first were replaced with '*' to simulate a many-to-one scenario.

Key Insights and Ideas

  • Semantic Homology is Subjective: Unlike biological sequences where homology is well-defined, semantic homology in citances is more nuanced and can be misleading (e.g., "cell cycle" vs. "U3A cells"). This necessitates flexible feature incorporation.
  • Many-to-Many Alignment is Crucial: Citances can align words to multiple other words, a characteristic not well-handled by traditional one-to-one alignment methods.
  • Posterior Decoding Offers Control: The posterior decoding approach, with its tunable utility function (gap and match factors), provides a more granular control over the alignment trade-offs (recall vs. precision) compared to Viterbi decoding.
  • Domain-Specific Features Matter: While orthographic features are useful, incorporating domain-specific features (even if some ontological features didn't make the final cut) is important for capturing semantic similarity.

Key Results

The proposed CRF-based posterior decoding method demonstrated superior performance compared to a baseline using normalized-edit distance for alignment.

  • Recall/Precision: The posterior decoding method achieved significantly higher recall levels across various precision points compared to Viterbi decoding. For instance, they achieved 0.742 recall at 0.198 precision by adjusting the gap and match factors.
  • MCA Performance: The CRF posterior decoding method's MCA curve "clearly dominates" the normalized-edit-distance baseline curve in recall/precision plots.
  • Best AMA Score: The best Alignment Metric Accuracy (AMA) score of 0.886 was achieved by tuning the gap and match factors, indicating the effectiveness of the posterior decoding approach.
  • Error Analysis: A significant portion of errors were false negatives, often due to misalignments of subtypes (e.g., cdc, cdc6, cdc25A), opposites, or complex entities.

Limitations and Future Work

  • Dataset Size: The authors acknowledge that their experiments were limited by the size of the labeled data, suggesting that larger datasets would be beneficial.
  • Subjectivity of Semantics: The inherent subjectivity of semantic homology in citances remains a challenge.
  • Entity Definition: The definition of basic entities for alignment can vary, with noun-compounds and multi-word entities being natural choices, but automatic segmentation and recognition of these are difficult.
  • Future Work: The authors suggest investigating automatic segmentation and entity recognition techniques to better define the basic elements for alignment. They also note that it's unclear if their formulation is the most useful for all downstream applications that use citances.

Summary updated

BibTeX
@inproceedings{schwartz2007multiple,
  author = {Ariel Schwartz and Anna Divoli and Marti Hearst},
  title = {{Multiple Alignment of Citation Sentences with Conditional Random Fields and Posterior Decoding}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2007},
}
Figure