Introduction
Complex Natural Language Processing (NLP) tasks often require high-level cognitive judgments that are inherently compositional. For instance, Targeted Sentiment Analysis involves two distinct steps: identifying the specific entity in a text (segmentation) and then determining the sentiment directed toward that entity (classification). Traditionally, these tasks are approached using monolithic models that attempt to predict a combined tag (e.g., B-Positive) in a single step.
However, monolithic approaches face two significant hurdles: the inherent difficulty of capturing distinct underlying structures simultaneously and the scarcity of high-quality, fully annotated datasets. This paper proposes a modular neural architecture that decomposes sequence labeling into simpler, interconnected sub-tasks, drawing inspiration from the "two-streams hypothesis" in cognitive neuroscience—treating segmentation as the "where" (location) and sentiment as the "what" (properties).
Methodology: Decomposing Sequence Labeling
The authors propose a framework based on the LSTM-CRF architecture, where the final label $y$ is decomposed into partial labels: $y^{seg}$ for segmentation and $y^{typ}$ for sentiment type. This modularity allows the model to provide explicit constraints or "hints" to the final decision module, simplifying the optimization landscape.
Architectural Variants
To integrate these modules, the paper evaluates three distinct information-sharing strategies:
- Twofold Modular (LSTM-CRF-T): A multi-task learning setup where sub-task modules and the final decision module share an embedding layer but maintain private LSTM layers.
- Twofold Modular Infusion (LSTM-CRF-TI): Hidden representations from the sub-task modules are concatenated and fed directly into the final decision module.
- Guided Gating Infusion (LSTM-CRF-TI(g)): The most advanced variant, which utilizes a sigmoid gating mechanism controlled by the final decision module. This allows the model to dynamically filter and "inhale" only the most relevant information from the segmentation and type modules.
Learning with Partial Labels
A primary advantage of this modular approach is its ability to learn from incomplete data. The loss function is defined as a linear combination of the negative log-likelihood of the final task and its sub-tasks. When a training instance is only partially annotated (e.g., it has entity spans but no sentiment labels), the model simply sets the loss for the missing tasks to zero. This allows the system to leverage large, easily obtainable datasets for simpler sub-tasks to improve performance on more complex target tasks.
Experimental Results
The models were rigorously tested across diverse benchmarks, including Targeted Sentiment (Mitchell et al. tweets), Aspect-Based Sentiment Analysis (SemEval 2016), and Named Entity Recognition (CoNLL).
Performance and Data Efficiency
The LSTM-CRF-TI(g) model achieved state-of-the-art results, particularly on the Targeted Sentiment task. In English tweet datasets, it reached an F1 score of 52.15, a substantial improvement over the previous benchmark of approximately 43.
Furthermore, the modular architecture demonstrated exceptional efficiency in low-resource scenarios. Experiments showed that a modular model trained on only 20% of fully labeled data—supplemented by partial labels—could nearly match the performance of a monolithic model trained on 100% of the full dataset.
Convergence and Gating
Beyond accuracy, modular models exhibited faster convergence, requiring significantly fewer training epochs than standard LSTM-CRFs. The success of the "Guided Gating" mechanism proved that the final decision module benefits significantly from the ability to selectively filter information from sub-task components.
Conclusion and Future Work
By decomposing complex sequence labeling into modular sub-tasks, this framework addresses both the learning difficulty of monolithic models and the challenge of data scarcity. While the current work focuses on a two-way split (segmentation vs. type), future research may explore more complex decompositions, such as joint entity and relation extraction, or applying these methods to low-resource languages where only basic linguistic annotations are available.
Summary updated