TL;DR
This paper introduces a novel Hierarchical and Nested Named Entity Recognition (HNNER) model that can simultaneously identify entities with complex hierarchical relationships and nested structures. By employing a transition-based parser and a specialized loss function, the model achieves state-of-the-art results on medical datasets, outperforming previous methods that flatten or project hierarchical information.
What Problem Does This Paper Address?
Named Entity Recognition (NER) is a fundamental task in Natural Language Processing (NLP) that involves identifying and classifying mentions of entities (like people, organizations, or locations) in text. While traditional NER focuses on "flat" mentions (where entities don't overlap), many real-world scenarios, particularly in specialized domains like medicine, involve "nested" entities (e.g., "lung cancer" as a disease, and "lung" as an anatomical structure within it) and "hierarchical" entities (e.g., a specific symptom being a subtype of a broader disease category).
Existing approaches often struggle to effectively model these complex relationships. Some methods flatten hierarchies by treating each level as an independent class, losing valuable structural information. Others handle nested entities but don't explicitly leverage the hierarchical nature of the ontology, meaning lower-level classes don't help disambiguate higher-level ones. This is particularly problematic in domains like medicine, where understanding the precise relationship between symptoms, diseases, and anatomical locations is crucial for accurate information extraction. The paper aims to bridge this gap by developing a model that can capture both hierarchical and nested entity structures simultaneously.
Methods
The core of the proposed HNNER model is a transition-based parser, inspired by work on dependency parsing. This approach processes input text by making a sequence of decisions (actions) to build the desired output structure. The model uses a combination of a word stack (to temporarily hold words being considered for an entity) and a mention stack (to keep track of active entity mentions and their hierarchical relationships).
Key actions include:
- OUT: Moves a word from the buffer to the output.
- SHIFT: Moves a word from the buffer to the word stack.
- TRANSITION(a): Marks the start of a mention with label 'a' and pushes it onto the mention stack.
- REDUCE(a): Marks the end of a mention, popping actions from the mention stack until the corresponding 'TRANSITION(a)' is found, and then outputs the identified mention.
A novel aspect is the ability to handle arbitrary numbers of hierarchical and nested mentions. The model enforces rules that ensure transitions to lower hierarchical levels are only made if the parent level is already established. It also prioritizes reducing longer mentions first when multiple mentions start at the same point.
Furthermore, the paper introduces modifier classes to capture nuances like the absence or uncertainty of a condition, which can alter the meaning of an entity. These modifiers are treated as top-level classes that can be applied at any hierarchical level.
The model is trained using a transition-based neural architecture that incorporates LSTMs to encode the states of the word stack, mention stack, and output. A specialized loss function is proposed that encourages correct predictions of higher-level mentions, giving them more weight.
Datasets
The authors evaluate their model on two medical datasets:
- GENIA: A well-known dataset for biomedical NER, consisting of abstracts from MEDLINE. The paper uses it for both flat and hierarchical entity recognition experiments.
- MED18: A larger and more complex proprietary medical corpus containing annotated clinical reports in Portuguese. This dataset is highlighted for its deeper hierarchical dependencies and a greater number of entity categories.
Key Insights and Ideas
- Unified Hierarchical and Nested Modeling: The most significant insight is the development of a single transition-based framework that can simultaneously handle both hierarchical and nested entity structures, which was a limitation of prior work.
- Leveraging Hierarchy for Disambiguation: By explicitly modeling the hierarchy, lower-level entity classes can help disambiguate higher-level ones, leading to more accurate and nuanced recognition.
- Modifier Classes for Richer Semantics: The introduction of modifier classes allows for capturing important contextual information like absence or uncertainty, which is critical in domains like medicine.
- Transition-Based Flexibility: The transition-based approach provides a flexible mechanism to model complex, potentially overlapping, and multi-level entity structures without predefining the depth of the hierarchy.
Key Results
The HNNER model achieves state-of-the-art results on both flat and nested NER tasks across the GENIA and MED18 datasets.
- On flat NER tasks (Table 1), HNNER slightly outperforms previous methods.
- For nested NER (Table 3), HNNER demonstrates strong performance, outperforming other nested NER models.
- Crucially, when evaluated on hierarchical entity recognition (Table 4), the explicit hierarchical modeling of HNNER outperforms its non-hierarchical counterpart (HNNER+SUB), especially as the hierarchy deepens. This gap widens on more complex datasets like MED18, underscoring the benefit of their approach.
Limitations and Future Work
- Overlapping Mentions: The current model primarily handles non-overlapping mentions. The authors acknowledge that extending it to handle more general overlapping mentions with non-projective overlaps is a direction for future work.
- Schedule Sampling: To mitigate potential errors during testing, the authors suggest exploring schedule sampling techniques.
- Dataset Specificity: While evaluated on medical datasets, the generalizability to other domains with complex hierarchical and nested entities could be further explored.
Summary updated