Abstract
We propose a triad-based neural network system that generates affinity scores between entity mentions for coreference resolution. The system simultaneously accepts three mentions as input, taking mutual dependency and logical constraints of all three mentions into account, and thus makes more accurate predictions than the traditional pairwise approach. Depending on system choices, the affinity scores can be further used in clustering or mention ranking. Our experiments show that a standard hierarchical clustering using the scores produces state-of-art results with gold mentions on the English portion of CoNLL 2012 Shared Task. The model does not rely on many handcrafted features and is easy to train and use. The triads can also be easily extended to polyads of higher orders. To our knowledge, this is the first neural network system to model mutual dependency of more than two members at mention level.
Introduction
Coreference resolution—the task of determining which mentions in a text (e.g., "Barack Obama," "he," and "the former president") refer to the same real-world entity—is a foundational challenge in natural language processing. Traditionally, researchers have relied on dyadic models (comparing two mentions at a time) or mention ranking models.
However, these approaches are limited by their inability to account for the inherent transitivity of coreference during the learning phase. While transitivity (if A=B and B=C, then A=C) is often enforced during post-processing clustering, dyadic models do not use this logic while extracting features. We propose a Triad Neural Network that models these relationships at a low level by processing three mentions simultaneously, leading to more accurate affinity scores.
Methodology
The core of our system is a neural network that accepts a triplet of mentions $(x_i, x_j, x_k)$ as input.
- Feature Extraction: Each mention is represented by its word tokens and Part-of-Speech (POS) tags, including a surrounding context window of eight words.
- Encoding and Attention: Mentions are processed through shared bidirectional LSTMs. A mutual attention mechanism allows each mention to "attend" to the other two in the triad, creating context-aware representations.
- Shared Context: The network generates three pairwise representations ($h_{ij}, h_{jk}, h_{ki}$) and combines them into a single "shared context" vector ($h_{ijk}$) via elementwise summation.
- Joint Decision: Three decoders work together to output a 3D binary vector. Because they share the $h_{ijk}$ context, the decision about whether A and B corefer is directly influenced by the relationship between B and C.
- Clustering: Affinity scores are averaged across multiple triads and then processed using standard hierarchical clustering (average linkage) to form the final entity groups.
Key Insights: The Bridge Effect
A primary advantage of the triad approach is the "bridge" effect. A third mention can provide the necessary evidence to link two otherwise ambiguous mentions. For example, if the relationship between a name and a distant pronoun is unclear, a more explicit intermediate mention in the triad can clarify the connection.
As illustrated above, the triad model significantly increases the affinity score (from 0.326 to 0.580) for ambiguous pairs by leveraging the linguistic evidence provided by the third mention in the triplet.
Experimental Results
We evaluated the model on the English portion of the CoNLL 2012 Shared Task using gold mentions. The Triad model significantly outperformed the dyadic baseline and set a new state-of-the-art for this task.
The triad system achieved an Average F1 of 77.93, outperforming the top participant in the original shared task (Chang, 77.22) and other established models like Fernandes (69.35). The model was particularly strong in the CEAF$\phi_4$ metric (73.78 F1), which measures cluster alignment accuracy.
Conclusion
By modeling mutual dependencies at the mention level, the Triad Neural Network captures transitivity more effectively than traditional pairwise systems. While the $O(N^3)$ computational complexity is a consideration for long documents, the performance gains on gold mentions demonstrate the power of higher-order modeling. Future work will focus on integrating this approach with end-to-end mention detection systems.
Summary updated