Guiding Sentiment Analysis with Hierarchical Text Clustering: Analyzing the German X/Twitter Discourse on Face Masks in the 2020 COVID-19 Pandemic

Silvan Wehrli, Chisom Ezekannagha, Georges Hattab, Tamara Boender, Bert Arnrich, Christopher IrrgangWASSA 2024 · DOI

Introduction

Public health emergencies like the COVID-19 pandemic trigger an "infodemic"—an overwhelming surge of information and discourse on social media. For health authorities, monitoring this sentiment is crucial for understanding public concerns and identifying misinformation, but the sheer volume of data makes manual monitoring impossible. Traditional "social listening" techniques often rely on "flat" topic modeling, which groups posts into a fixed number of broad categories. The limitation of this approach is that broad categories like "Politics" or "Health Measures" often hide the granular narratives that actually drive public opinion. This paper presents a framework that combines hierarchical text clustering with deep-learning-based sentiment analysis to navigate these complex social media landscapes.

Methodology

The researchers developed a three-stage pipeline to process over 350,000 German tweets from 2020.

  1. Sentiment Classification: The team utilized GBERT, a German-specific BERT model. To handle the informal and often "messy" language of social media, they employed Task-Adaptive Pretraining (TAPT)—further training the model on the mask-related dataset—followed by supervised fine-tuning on a manually annotated subset of 2,200 posts.
  2. Hierarchical Clustering: Instead of standard flat clustering, they used the Sub-Cluster Component Algorithm (SCC). This builds a tree-like structure based on document embeddings, allowing researchers to zoom from 20 broad topics down to over 1,000 highly specific sub-clusters.
  3. Treemap Visualization: To make the hierarchy interpretable, they used treemaps where the size of a box represents the volume of posts and the color represents the average sentiment score (from deep purple for negative to dark green for positive).

A significant finding of the study was that social media activity was driven more by policy changes than by infection rates.

Timeline of Twitter posts and 7-day incidence rate in Germany (2020), highlighting key policy events.

As shown in Figure 1, the largest single spike in discourse occurred in April 2020, coinciding with the announcement of mandatory mask-wearing across Germany. This suggests that social media serves as a primary arena for the public negotiation of new social norms following government mandates. Interestingly, while infection rates (the 7-day incidence rate) rose sharply in late 2020, the volume of mask-related posts did not see a proportional increase, indicating that the topic had become somewhat "normalized" or overshadowed by other lockdown measures.

Hierarchical Sentiment Analysis

The core advantage of the hierarchical approach is its ability to reveal nuances hidden within broad categories. While the general sentiment regarding masks was predominantly negative or neutral, specific sub-topics told different stories.

Hierarchical treemap visualization showing Level 1 (broad topics) and Level 3 (fine-grained sub-topics of Politics and Demonstrations).

Figure 4 illustrates how the framework allows researchers to "zoom in" on Topic 3 (Politics and demonstrations). While the parent topic is generally negative, the hierarchy reveals specific drivers of this sentiment. For instance, sub-topics related to anti-mask demonstrations in Berlin and Leipzig were major sources of sustained negativity. Conversely, the hierarchy revealed that Topic 10 (DIY masks) actually held a slightly positive sentiment—a detail that would have been swallowed by the general negativity of the larger dataset in a flat model.

Key Results and Model Performance

The fine-tuned GBERT_TAPT+SFT model was highly effective, achieving a weighted F1-score of 80.96%, significantly outperforming the "out-of-the-box" GBERT (54.05%). The SCC algorithm successfully identified a flexible hierarchy of 20 top-level topics, 104 mid-level topics, and 1,051 fine-grained clusters.

Despite these successes, the authors noted limitations. The word "Maske" in German is a homonym that can refer to beauty products or costumes, introducing some noise. Additionally, like most NLP models, GBERT struggled with sarcasm and irony, which are prevalent in political Twitter discourse.

Conclusion

By integrating hierarchical clustering with sentiment analysis, this framework provides a scalable and nuanced way for public health officials to perform "infoveillance." It moves beyond broad generalizations to identify the specific narratives shaping public opinion during a crisis, offering a powerful tool for navigating the modern infodemic.

Summary updated

BibTeX
@inproceedings{wehrli2024guiding,
  author = {Silvan Wehrli and Chisom Ezekannagha and Georges Hattab and Tamara Boender and Bert Arnrich and Christopher Irrgang},
  title = {{Guiding Sentiment Analysis with Hierarchical Text Clustering: Analyzing the German X/Twitter Discourse on Face Masks in the 2020 COVID-19 Pandemic}},
  booktitle = {WASSA},
  year = {2024},
  doi = {10.18653/v1/2024.wassa-1.13},
}
Figure