Abstract
Combining fine-grained opinion information to produce opinion summaries is important for sentiment analysis applications. Toward that end, we tackle the problem of source coreference resolution -- linking together source mentions that refer to the same entity. The partially supervised nature of the problem leads us to define and approach it as the novel problem of partially supervised clustering. We propose and evaluate a new algorithm for the task of source coreference resolution that outperforms competitive baselines.
Introduction: The Challenge of Source Coreference
In sentiment analysis, extracting "who" said "what" about "whom" is only the first step. To generate a coherent summary—such as determining the public consensus on a specific policy—we must group opinions by the same entity. This requires source coreference resolution: the task of determining which mentions (e.g., "Zacarias Moussaoui," "he," and "my") refer to the same real-world opinion-holder.
Standard coreference resolution typically requires training data where every noun phrase (NP) is labeled. However, in opinion mining, datasets like the MPQA corpus often only annotate NPs that act as sources of opinions. This leaves a significant amount of "objective" text unlabeled. These unlabeled mentions often act as vital "bridges" between opinion-bearing mentions. For instance, a name in an objective sentence might be the only link between a pronoun at the end of a document and a source mention at the beginning.
Methodology: The StRip Algorithm
The authors frame this task as partially supervised clustering. Unlike standard semi-supervised learning, this approach learns a clustering function from examples where only a subset of items are pre-assigned to clusters.
The core contribution is StRip (Structured RIPPER), an extension of the RIPPER rule-learning algorithm. While traditional coreference systems use a "classify then cluster" pipeline—training a classifier on pairs and then clustering as a post-process—StRip integrates clustering directly into the learning process.
When evaluating a rule (e.g., "If NPs share the same head word, they are coreferent"), StRip computes the transitive closure (the full chain of links) and evaluates the rule based on the quality of the resulting clusters. During training, StRip utilizes unlabeled NPs to help form these chains but ignores them when calculating accuracy or information gain. This allows the model to reward rules that successfully use unlabeled mentions to connect labeled sources.
Experimental Results
The performance of StRip was evaluated against several competitive baselines, including SVMs and standard RIPPER trained on both the MPQA and MUC (MUC6, MUC7) datasets. The results demonstrate that StRip's partially supervised approach significantly improves both clustering accuracy and link identification.
As shown in the results, StRip achieved a $B^3$ score of 83.2, outperforming the best baseline of 81.8. More notably, in terms of identifying positive coreference links (F1 score), StRip reached 67.1, a substantial improvement over the 60.9 achieved by the best baseline (SVM trained on MUC6). The superior performance of StRip on the "MPQA all" set compared to "MPQA src" validates the importance of leveraging unlabeled data through a partially supervised framework.
Conclusion
This research highlights that unlabeled "objective" data is not merely extra information but often the structural glue necessary for accurate coreference in opinion summarization. By forcing the rule learner to consider the global cluster structure rather than isolated pairs, StRip avoids local errors that disrupt coreference chains. Future work will extend this approach to target coreference resolution, grouping the subjects of opinions to provide a more complete summarization solution.
Summary updated