Introduction
Monitoring public sentiment regarding vaccinations is a critical task for public health officials, particularly as new concerns and misinformation emerge in real-time on social media. However, traditional text-mining approaches often struggle to categorize these opinions effectively. This paper introduces Disentangled Opinion Clustering (DOC), a novel framework designed to separate the specific topic of a post (the aspect) from the author's sentiment toward that topic (the stance). By isolating these two components, the DOC model achieves superior performance in clustering vaccination-related discourse, even when encountering previously unseen topics.
The Challenge: Stance Interference in Opinion Mining
Public opinion on vaccines is highly dynamic. New issues—such as mRNA technology, fertility concerns, or booster mandates—frequently surface. Supervised models trained on fixed labels are often unable to detect these emerging topics.
Furthermore, standard text-clustering methods suffer from "stance interference." For instance, consider two tweets: "mRNA vaccines are poison" and "The Pfizer vaccine is safe." Both discuss the aspect of safety, yet their stances are diametrically opposed. Traditional models often place these into different clusters because the linguistic markers of stance (e.g., "poison" vs. "safe") outweigh the markers of the shared topic. This interference creates fragmented clusters, making it difficult for researchers to analyze the full breadth of the debate surrounding a single issue.
Methodology: Disentangled Opinion Clustering (DOC)
The DOC model addresses stance interference through a two-stage training process centered on a Siamese Autoencoder architecture.
1. Unsupervised Pre-training
The model begins with a DeBERTa-based encoder-decoder structure. It is pre-trained on a massive unlabelled corpus of 4.37 million vaccination-related tweets. Using a masked token prediction task, the model learns the general linguistic patterns and vocabulary specific to the vaccination domain.
2. Supervised Disentanglement and the Swapping Mechanism
The core innovation of DOC is the separation of a sentence's representation into two distinct latent vectors: $z_s$ (representing stance) and $z_a$ (representing aspect). To ensure these vectors are truly independent, the researchers implemented a Swapping Autoencoder strategy.
As illustrated in the figure above, the model takes pairs of tweets that share the same aspect but differ in stance. It then swaps the aspect embeddings ($z_a$) between the two tweets. The decoder is then challenged to reconstruct the original text using these swapped representations. If the reconstruction is successful, it demonstrates that the aspect vector $z_a$ captures the topic information independently of the stance. Additionally, the model utilizes Sentence-BERT (SBERT) as an inductive bias to help the aspect vector focus on the specific text spans that define the topic.
Data and Experimental Setup
The model was evaluated using two primary public Twitter datasets:
- CMF (Covid-Moral-Foundation): Tweets categorized by moral foundations (e.g., Care/Harm, Liberty/Oppression).
- VAD (Vaccination Attitude Detection): Tweets covering eight specific aspect categories, such as Side Effects, Immunity Level, and Economic Effects.
Results and Performance
The DOC model demonstrated significant improvements over state-of-the-art baselines, including SBERT and specialized clustering models like SCCL.
- Clustering Accuracy: On the VAD dataset, DOC achieved a Clustering Accuracy (CA) of 66.7%, outperforming SBERT-k-means (60.5%).
- Zero-Shot Generalization: One of the model's greatest strengths is its ability to generalize. When trained on the CMF dataset and tested on the VAD dataset (which contains different topics), DOC maintained a high performance of 56.2% CA. This suggests the model learns the nature of how aspects are expressed rather than just memorizing specific keywords.
- Latent Space Visualization: t-SNE projections revealed that DOC’s aspect vectors ($z_a$) form tight, distinct clusters. In contrast, standard embeddings often result in a "hairball" where different topics are inextricably mixed due to stance interference.
Conclusion and Future Work
The DOC model provides a robust solution for disentangling stance from aspect in social media analysis. By utilizing a swapping autoencoder, the framework allows for cleaner, more actionable topic clustering that is resilient to the "noise" of polarized sentiments.
While the model currently requires a pre-defined number of clusters ($k$) and a large pre-training corpus, it represents a significant step forward in automated opinion mining. Future research will explore methods for automatically determining optimal cluster counts and applying this disentanglement framework to other highly polarized domains, such as political discourse and product reviews.
Summary updated