Hyperbolic Disentangled Representation for Fine-Grained Aspect Extraction

Chang-Yu Tai, Ming-Yao Li, Lun-Wei KuAAAI 2022 · DOI

Abstract

Fine-grained aspect extraction is a critical task in opinion mining, yet existing weakly supervised methods often fail to capture the inherent hierarchical nature of language and the multi-faceted semantics of seed words. We present the Hyperbolic Disentangled Aspect Extractor (HDAE), a novel framework that utilizes hyperbolic geometry and disentangled representations. HDAE demonstrates substantial performance gains—up to 24.1% in F1 score—across multiple benchmarks while utilizing significantly fewer parameters than large-scale language models like BERT.

Introduction

Aspect extraction aims to identify specific topics within user reviews, such as "service" or "price." While supervised approaches are effective, they are limited by the need for extensive labeled datasets. Weakly supervised methods mitigate this by using "seed words" to guide the model. However, these methods typically operate in Euclidean space, which struggles to represent hierarchical "is-a" relationships without significant distortion. Furthermore, they often represent each seed word as a single point, ignoring the diverse latent semantics a word might possess in different contexts.

Figure 1: (a) Hierarchical relationships in color terminology. (b) Multiple latent meanings associated with the seed word 'picture'.

Methodology

The HDAE framework addresses these challenges through two primary innovations: hyperbolic embeddings and an aspect disentanglement module.

Hyperbolic Geometry in the Poincaré Ball

Unlike Euclidean space, hyperbolic space features a geometry where volume grows exponentially with distance from the origin. This property makes it ideal for embedding hierarchical structures like trees. HDAE employs a Hyperbolic Aspect Classifier that measures the relationship between review segments and aspects using hyperbolic distance, effectively capturing hyponymic relationships (e.g., recognizing "ultramarine" as a type of "blue" and "color").

Aspect Disentanglement Module

To account for semantic variety, HDAE generates multiple disentangled latent semantic vectors for each seed word. This process is governed by three types of Aspect-Aware Regularization:

  1. Seed Word Dependence: Maintains proximity between related seed words within an aspect.
  2. Latent Semantic Independence: Ensures that different latent meanings of a word remain distinct to minimize redundancy.
  3. Aspect Scope Confinement: Constrains all latent meanings of a word to the conceptual boundaries of its assigned aspect.
Figure 4: The Aspect Disentanglement Module architecture and its three regularization strategies.

During inference, the model utilizes Gumbel Softmax to select the most appropriate latent meaning for a given review segment, allowing for context-aware aspect assignment.

Experimental Results

HDAE was evaluated on the Amazon Product Reviews (OPOSUM) and SemEval-2016 Restaurant Reviews datasets. The model consistently outperformed previous weakly supervised baselines, achieving an average F1 improvement of 18.2% on Amazon reviews and 24.1% on restaurant reviews. Notably, HDAE proved highly effective in multilingual settings, outperforming fully supervised models in certain non-English datasets.

Beyond accuracy, HDAE is remarkably efficient. By leveraging the representational power of hyperbolic space, it achieves state-of-the-art results with only 2.5 million parameters—a 97.8% reduction compared to the 109.5 million parameters of BERT-based models.

Conclusion

The Hyperbolic Disentangled Aspect Extractor provides a robust solution for fine-grained aspect extraction by addressing the hierarchical and semantic complexities of natural language. Future work will explore non-uniform dependencies between seed words and extend the framework to the complete Aspect-Based Sentiment Analysis (ABSA) pipeline, incorporating sentiment prediction.

Summary updated

BibTeX
@inproceedings{tai2022hyperbolic,
  author = {Chang-Yu Tai and Ming-Yao Li and Lun-Wei Ku},
  title = {{Hyperbolic Disentangled Representation for Fine-Grained Aspect Extraction}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i10.21387},
}
Figure