Sentiment Tagging with Partial Labels using Modular Architectures

Xiao Zhang, Dan GoldwasserACL 2019 · DOI

Introduction

Complex Natural Language Processing (NLP) tasks often require high-level cognitive judgments that are inherently compositional. For instance, Targeted Sentiment Analysis involves two distinct steps: identifying the specific entity in a text (segmentation) and then determining the sentiment directed toward that entity (classification). Traditionally, these tasks are approached using monolithic models that attempt to predict a combined tag (e.g., B-Positive) in a single step.

However, monolithic approaches face two significant hurdles: the inherent difficulty of capturing distinct underlying structures simultaneously and the scarcity of high-quality, fully annotated datasets. This paper proposes a modular neural architecture that decomposes sequence labeling into simpler, interconnected sub-tasks, drawing inspiration from the "two-streams hypothesis" in cognitive neuroscience—treating segmentation as the "where" (location) and sentiment as the "what" (properties).

Methodology: Decomposing Sequence Labeling

The authors propose a framework based on the LSTM-CRF architecture, where the final label $y$ is decomposed into partial labels: $y^{seg}$ for segmentation and $y^{typ}$ for sentiment type. This modularity allows the model to provide explicit constraints or "hints" to the final decision module, simplifying the optimization landscape.

Architectural Variants

To integrate these modules, the paper evaluates three distinct information-sharing strategies:

  1. Twofold Modular (LSTM-CRF-T): A multi-task learning setup where sub-task modules and the final decision module share an embedding layer but maintain private LSTM layers.
  2. Twofold Modular Infusion (LSTM-CRF-TI): Hidden representations from the sub-task modules are concatenated and fed directly into the final decision module.
  3. Guided Gating Infusion (LSTM-CRF-TI(g)): The most advanced variant, which utilizes a sigmoid gating mechanism controlled by the final decision module. This allows the model to dynamically filter and "inhale" only the most relevant information from the segmentation and type modules.
Comparison of modular architectures: (a) Twofold Modular (T), (b) Twofold Modular Infusion (TI), and (c) Guided Gating Infusion (TI(g)).

Learning with Partial Labels

A primary advantage of this modular approach is its ability to learn from incomplete data. The loss function is defined as a linear combination of the negative log-likelihood of the final task and its sub-tasks. When a training instance is only partially annotated (e.g., it has entity spans but no sentiment labels), the model simply sets the loss for the missing tasks to zero. This allows the system to leverage large, easily obtainable datasets for simpler sub-tasks to improve performance on more complex target tasks.

Experimental Results

The models were rigorously tested across diverse benchmarks, including Targeted Sentiment (Mitchell et al. tweets), Aspect-Based Sentiment Analysis (SemEval 2016), and Named Entity Recognition (CoNLL).

Performance and Data Efficiency

The LSTM-CRF-TI(g) model achieved state-of-the-art results, particularly on the Targeted Sentiment task. In English tweet datasets, it reached an F1 score of 52.15, a substantial improvement over the previous benchmark of approximately 43.

Furthermore, the modular architecture demonstrated exceptional efficiency in low-resource scenarios. Experiments showed that a modular model trained on only 20% of fully labeled data—supplemented by partial labels—could nearly match the performance of a monolithic model trained on 100% of the full dataset.

Performance of modularized vs. non-modularized models on Spanish and English datasets across different training data percentages.

Convergence and Gating

Beyond accuracy, modular models exhibited faster convergence, requiring significantly fewer training epochs than standard LSTM-CRFs. The success of the "Guided Gating" mechanism proved that the final decision module benefits significantly from the ability to selectively filter information from sub-task components.

Conclusion and Future Work

By decomposing complex sequence labeling into modular sub-tasks, this framework addresses both the learning difficulty of monolithic models and the challenge of data scarcity. While the current work focuses on a two-way split (segmentation vs. type), future research may explore more complex decompositions, such as joint entity and relation extraction, or applying these methods to low-resource languages where only basic linguistic annotations are available.

Summary updated

BibTeX
@inproceedings{zhang2019sentiment,
  author = {Xiao Zhang and Dan Goldwasser},
  title = {{Sentiment Tagging with Partial Labels using Modular Architectures}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1055},
}
Figure