Unsupervised Opinion Summarisation in the Wasserstein Space

Jiayu Song, Iman Munire Bilal, Adam Tsakalidis, Rob Procter, Maria Liakata

Abstract

Opinion summarization aims to condense multiple documents into a single representative text. However, social media data—such as tweets and Reddit threads—presents significant challenges due to its noisy, ungrammatical, and stylistically diverse nature. We present WassOS, an unsupervised model that treats documents as probability distributions within the Wasserstein space. By explicitly disentangling semantic meaning from syntactic style and utilizing the Wasserstein barycenter to identify the mathematical "center of mass" of opinions, WassOS produces summaries that preserve core meaning more effectively than previous state-of-the-art methods. Evaluations across Twitter, Reddit, and Amazon datasets demonstrate superior performance in both automated ROUGE metrics and human-rated meaning preservation.

Introduction

The proliferation of user-generated content on social media has created a need for robust opinion summarization tools. While existing models perform well on structured reviews (e.g., Amazon), they often falter when applied to the "noise" of platforms like Twitter or Reddit. In these domains, posts are frequently ungrammatical and laden with slang.

Traditional models typically represent documents in Euclidean space and use simple averaging to generate a summary representation. This approach often conflates the "what" (semantics) with the "how" (syntax), leading to summaries that are a "confusing soup" of conflicting writing styles rather than a clear synthesis of opinions. Furthermore, standard distance metrics like KL-divergence can suffer from vanishing gradients when comparing non-overlapping distributions. WassOS addresses these issues by leveraging the geometric properties of the Wasserstein space.

Methodology: The WassOS Framework

WassOS introduces three primary technical innovations to handle noisy text clusters:

1. Disentangled Latent Space

Using a Variational Autoencoder (VAE), the model splits a document's internal representation into two distinct latent spaces: one for semantics and one for syntax. This separation ensures that the core opinion is not diluted by the idiosyncratic writing styles of individual users.

2. Wasserstein Barycenter

Instead of calculating a simple vector average, WassOS treats each document as a distribution. The summary is computed as the Wasserstein barycenter—the optimal point that minimizes the Wasserstein distance to all individual document distributions in the cluster. This metric is more robust to noise and provides a smoother geometry for optimization compared to Euclidean or KL-divergence-based methods.

Figure 1: The WassOS architecture, illustrating the disentanglement of semantic and syntactic spaces and the calculation of the summary via the Wasserstein barycenter.

3. BERT-Guided Decoder

To translate the mathematical summary back into natural language, the model employs a GRU decoder enhanced with a Transformer layer. This layer is initialized with parameters from the middle layers of BERT, which are specifically chosen for their ability to capture strong syntactic features, thereby improving the grammaticality of the generated text.

Experimental Setup and Datasets

The model was evaluated on three diverse datasets to test its generalizability:

  • Twitter: 2,214 clusters of tweets concerning COVID-19 and political events.
  • Reddit: A new dataset of 40 threads from r/COVID19_support, featuring expert-written gold-standard summaries.
  • Amazon: A standard benchmark consisting of 60 product review clusters.

Results and Discussion

WassOS demonstrated significant improvements over previous state-of-the-art models like Copycat and COOP. On the Twitter dataset, the model showed a relative improvement of 5% to 11% in ROUGE scores. On Reddit, it achieved the highest ROUGE-1 and ROUGE-2 scores among compared models.

Figure 3: ROUGE score comparison across Twitter, Amazon, and Reddit datasets. WassOS (T-center) consistently outperforms baselines on noisy social media data.

In human evaluations, WassOS was particularly noted for its Meaning Preservation. Across all three domains, human judges found that WassOS was significantly better at capturing the actual gist of the source documents. Ablation studies further confirmed that both the disentanglement of latent spaces and the Transformer-based decoder were critical to the model's success; removing either component resulted in a marked decline in summary quality.

Conclusion

WassOS represents a step forward in unsupervised summarization by moving beyond simple Euclidean averaging. By treating opinions as distributions in Wasserstein space, the model successfully navigates the noise of social media text. While the model currently faces some challenges with sentence redundancy—likely due to the beam search algorithm and latent feeding mechanisms—it provides a robust framework for capturing global properties in diverse document clusters. Future work will focus on scaling the model to larger clusters and addressing scenarios involving diametrically opposed viewpoints within a single cluster.

Summary updated

Abstract

Opinion summarisation synthesises opinions expressed in a group of documents discussing the same topic to produce a single summary. Recent work has looked at opinion summarisation of clusters of social media posts. Such posts are noisy and have unpredictable structure, posing additional challenges for the construction of the summary distribution and the preservation of meaning compared to online reviews, which has been so far the focus of opinion summarisation. To address these challenges we present \textit{WassOS}, an unsupervised abstractive summarization model which makes use of the Wasserstein distance. A Variational Autoencoder is used to get the distribution of documents/posts, and the distributions are disentangled into separate semantic and syntactic spaces. The summary distribution is obtained using the Wasserstein barycenter of the semantic and syntactic distributions. A latent variable sampled from the summary distribution is fed into a GRU decoder with a transformer layer to produce the final summary. Our experiments on multiple datasets including Twitter clusters, Reddit threads, and reviews show that WassOS almost always outperforms the state-of-the-art on ROUGE metrics and consistently produces the best summaries with respect to meaning preservation according to human evaluations.
BibTeX
@inproceedings{song2022unsupervised,
  author = {Jiayu Song and Iman Munire Bilal and Adam Tsakalidis and Rob Procter and Maria Liakata},
  title = {{Unsupervised Opinion Summarisation in the Wasserstein Space}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  doi = {10.18653/v1/2022.emnlp-main.589},
}
Figure