Bridge-Based Active Domain Adaptation for Aspect Term Extraction

Zhuang Chen, Tieyun QianACL 2021

TL;DR

This paper introduces "bridge-based" active domain adaptation for aspect term extraction (ATE). By constructing syntactic and semantic bridges, the method effectively transfers knowledge across domains without relying on manual annotations or expensive pre-trained models, achieving state-of-the-art results.

What Problem Does This Paper Address?

Aspect Term Extraction (ATE) is crucial for understanding sentiment in text, but it suffers from a significant challenge: the high cost of manual annotation. This makes it difficult to train models for new domains where labeled data is scarce. While domain adaptation techniques aim to transfer knowledge from labeled source domains to unlabeled target domains, existing methods struggle because aspect terms are often domain-specific and cannot be directly transferred. Previous approaches like "passive domain adaptation" rely on associating aspect terms with "pivot words" (e.g., opinion words or context terms), which often require manual effort to identify pivots or expensive computational resources. This paper addresses the need for a more efficient and effective unsupervised domain adaptation method for ATE that can transfer aspect terms even when they are not directly present in the target domain.

Methods

The core innovation of this paper is the introduction of "bridges" to facilitate knowledge transfer across domains. These bridges are designed to be domain-invariant and capture transferable information.

  • Syntactic Bridge: This bridge leverages the observation that while linguistic expressions can be flexible, syntactic structures (like Part-of-Speech tags and dependency relations) are more general across domains. Instead of using dependency relations to link pivot words to aspect terms, this method treats the syntactic roles themselves as "pivots." It encodes these roles into a vector representation for each word, effectively creating a syntactic fingerprint.
  • Semantic Bridge: This bridge builds upon the syntactic bridge by retrieving "transferable prototypes" from unlabeled target domain data. The goal is to find words in the target domain that play similar syntactic roles and have semantic similarity to source domain words. A novel syntax-enhanced similarity metric is proposed to find these prototypes, which are then aggregated to form the semantic bridge for a word. This allows for direct correlation between aspect terms across domains.
  • Bridge-based Sequence Tagging: The paper proposes an end-to-end tagger that fuses the original word embeddings with the constructed syntactic and semantic bridges using a gating mechanism. This enhanced embedding is then fed into a CNN feature extractor, followed by a token classifier for aspect term prediction. Domain adversarial training is also employed to further enhance domain invariance.

Datasets

The experiments were conducted on three conventional English datasets:

  • Restaurant (R): From SemEval 2014 and 2015.
  • Laptop (L): From SemEval 2014 and 2015.
  • Device (D): Collected by Hu and Liu (2004).

These datasets were used to create six directed transfer pairs (e.g., R→L, L→R, etc.) for evaluation. The paper does not introduce new benchmarks but uses existing ones for a comprehensive evaluation.

Key Insights and Ideas

  • Syntactic Roles as Pivots: Treating syntactic roles (POS tags, dependency relations) as intrinsic features for transfer, rather than just links, is a novel way to overcome domain specificity.
  • Direct Aspect-to-Aspect Transfer: The semantic bridge enables a more direct transfer of knowledge by linking aspect terms across domains through shared prototypes, bypassing the need for explicit opinion words or complex rule-based systems.
  • Leveraging Unlabeled Data Actively: The method actively seeks out transferable prototypes from unlabeled target data, making the domain adaptation process more efficient and less reliant on manual annotations.
  • Effectiveness of Bridges: Both syntactic and semantic bridges significantly improve performance over a baseline tagger without bridges, demonstrating their value in capturing domain-invariant information.
  • Low Computational Cost: The proposed method achieves state-of-the-art results with significantly lower computational cost and fewer trainable parameters compared to many BERT-based methods.

Key Results

  • The proposed SemBridge model achieves a new state-of-the-art performance, outperforming previous methods by a significant margin. For instance, it surpasses TIMN (Type-I) by 7.02% and BERT-Cross (Type-II) by 5.21% in average F1-scores.
  • Even the BaseTagger (without bridges) outperforms existing baselines, attributed to its CNN feature extractor and domain adversarial training.
  • SynBridge and SemBridge provide absolute gains of 1.80% and 2.68% respectively over BaseTagger.
  • The method demonstrates robustness to parsing noise and requires a relatively small percentage of unlabeled data (around 20-40%) to achieve satisfactory performance.
  • The computational cost is notably low, with SynBridge/SemBridge having only 1.4M parameters and a runtime of 12s per epoch, significantly less than methods like BERT-Cross (109M parameters, 84s runtime).

Limitations and Future Work

  • OTE Task Interference: When performing both Aspect Term Extraction (ATE) and Opinion Term Extraction (OTE) simultaneously, there's a slight performance decrease in AVG-AS compared to ATE-only. The authors suggest this might be due to interference between the two objectives without a sophisticated multi-task learning framework.
  • Dependency on Parsing: While robust to some noise, the method's performance is still affected by parsing errors. Future work could explore more robust parsing techniques or methods that are less sensitive to parsing quality.
  • Combination Strategy: While combining SynBridge and SemBridge (ComBridge) offers a slight improvement, the combination strategy is simple. More sophisticated fusion methods could potentially yield greater benefits.
  • Hyperparameter Sensitivity: The paper explores the impact of hyperparameters like λ, τ, and K, suggesting that careful tuning is necessary for optimal performance. Further research could investigate automated hyperparameter optimization.

Summary updated

BibTeX
@inproceedings{chen2021bridge,
  author = {Zhuang Chen and Tieyun Qian},
  title = {{Bridge-Based Active Domain Adaptation for Aspect Term Extraction}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure