Discriminative Models Can Still Outperform Generative Models in Aspect Based Sentiment Analysis

Dhruv Mullick, Alona Fyshe, Bilal GhanemarXiv 2022 · DOI · arXiv

Abstract

While generative models currently dominate many NLP benchmarks, this study demonstrates that discriminative models maintain a significant edge in complex Aspect-Based Sentiment Analysis (ABSA) tasks, particularly in cross-lingual and cross-domain settings. We introduce a "Masked" generative approach that prevents model hallucinations by constraining the decoder to the input text, resulting in performance boosts of up to 20%.

Introduction

Aspect-Based Sentiment Analysis (ABSA) involves identifying specific "aspects" in a review (e.g., "the service") and their associated sentiment (e.g., "positive"). Recently, the field has shifted from discriminative models, which label spans of text, to generative models that "write out" the aspect and sentiment. However, generative models often struggle with "grounding" in low-resource or cross-lingual settings, leading to hallucinations where the model might output translations rather than the source text.

Methodology

We compare three primary architectures:

  1. SPAN-MBERT (Discriminative): A state-of-the-art model using a multilingual BERT encoder to extract and classify text spans. It is inherently constrained to the input sentence.
  2. mBART Non-Masked (Generative): An encoder-decoder model that generates aspect-sentiment pairs as strings.
  3. mBART Masked (Proposed Hybrid): A novel contribution that applies logit masking during decoding to force the model to choose only tokens present in the original input, combining generative semantic understanding with discriminative constraints.

Experimental Setup

The study utilized standard ABSA benchmarks:

  • SemEval 2016 (Rest16): Restaurant reviews in English (En), Spanish (Es), and Russian (Ru).
  • SemEval 2014 (Lap14): Laptop reviews in English.
  • MAMS: A challenging English restaurant dataset where sentences contain at least two aspects with differing polarities.

Results and Discussion

The "Hard Task" Advantage

Discriminative models showed particular resilience on the MAMS dataset. As seen in Figure 1, SPAN-MBERT outperformed generative models on MAMS_En. This suggests that discriminative models benefit from only needing to learn decision boundaries, whereas generative models must master the complex joint probability of entire sequences.

Performance across different domains and languages. Note the SPAN-MBERT lead in MAMS.

Cross-Lingual Performance and Masking

Generative models are often "too smart," translating aspects back into the training language during cross-lingual tests (e.g., outputting "price" instead of "precio"). Our proposed mBART Masked model mitigates this. Figure 4 illustrates the dramatic improvements in cross-lingual settings, where masking provides significant gains—up to 16% in Spanish tests.

Cross-lingual performance comparison showing the effectiveness of the Masked approach.

In extreme "cross-lingual + cross-domain" settings (e.g., training on English Restaurants and testing on Spanish Laptops), the Masked approach yielded gains of up to 20%. This confirms that while generative models are the current industry standard for in-domain English tasks, discriminative models and hybrid "masked" generative approaches offer superior robustness in complex scenarios.

Conclusion

This research highlights that the shift toward generative models in ABSA is not without trade-offs. Discriminative models remain highly effective for complex multi-aspect tasks, and logit masking provides a powerful tool to ground generative models in cross-lingual contexts. Future work will explore how these models handle conflicting polarities and the scalability of masking techniques to larger LLMs.

Summary updated

BibTeX
@misc{mullick2022discriminative,
  author = {Dhruv Mullick and Alona Fyshe and Bilal Ghanem},
  title = {{Discriminative Models Can Still Outperform Generative Models in Aspect Based Sentiment Analysis}},
  year = {2022},
  eprint = {2206.02892},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2206.02892},
}
Figure