Introduction
Coreference resolution—the task of identifying all expressions that refer to the same real-world entity—is a foundational challenge in natural language processing. Most modern systems rely on automatic syntactic parsers to identify candidate "mentions" (e.g., noun phrases or pronouns). However, this dependency introduces a significant bottleneck: if a parser fails to correctly identify a phrase's boundaries, the coreference system cannot resolve it. In the OntoNotes dataset, approximately 3% of mentions are "invisible" to standard parsers because they lack a corresponding constituent in the parse tree, creating a theoretical performance ceiling.
The Error Tolerable System (ETS) is designed to break through this ceiling by incorporating fault tolerance into both the mention detection and entity clustering stages of the pipeline.
Mention Detection via Packed Forests
To address parsing errors, ETS moves away from relying on a single "best" parse tree. Instead, it utilizes a modified Charniak and Johnson parser to generate a packed forest. A packed forest is a compact representation of a massive number of potential parse trees for a single sentence.
By searching this forest for constituents with specific tags (such as NP, NNP, or PRP), the system can recover mentions that a standard parser might have misaligned or ignored. This approach reduced the rate of "unrecognized" mentions from 3% to just 0.5%, effectively raising the theoretical recall limit of the system to 99.5%.
Entity Resolution via Bell Tree Beam Search
Once mentions are identified, they must be grouped into entities. Traditional "greedy" approaches link mentions to their most likely antecedents locally, which can lead to global inconsistencies. ETS treats coreference as a global optimization problem.
The system uses a Maximum Entropy classifier to score mention pairs based on lexical, grammatical, and semantic features (such as WordNet distance and semantic roles). To find the optimal clustering, it employs a Bell Tree to represent the space of all possible partitions of the mentions. Because the number of possible partitions (the Bell Number) grows exponentially, a beam search is used to prune the search space, keeping only the most promising global candidates.
Evaluation and Results
ETS was developed and evaluated for the CoNLL-2011 Shared Task using the multi-genre OntoNotes corpus. The researchers compared the baseline system against versions incorporating the packed forest (sys_forest), the Bell Tree beam search (sys_btree), and the combined full system (sys_forest_btree).
As shown in the results, the full system achieved the highest performance, with a 59.71% Mention F-score and a MUC score of 44.97%. Notably, the system using packed forests performed nearly as well as a system using "gold" (human-annotated) parse trees, demonstrating that the "error tolerable" design successfully mitigates the impact of automatic parsing errors.
Conclusion
The ETS framework demonstrates that coreference resolution systems can be made robust to upstream errors. By utilizing packed forests for mention detection and global beam search for entity clustering, the system effectively bypasses the traditional 3% parsing bottleneck. While the authors noted that the increased number of candidates from the forest introduces some noise, the overall gains in recall and global optimization provide a significant advantage over standard greedy, tree-based systems.
Summary updated