SoulMate: Short-Text Author Linking Through Multi-Aspect Temporal-Textual Embedding

Saeed Najafi Pour, Saeid Hosseini, Wen Hua, Mohammad Reza Kangavari, Xiaofang Zhou

Linking authors across the noisy, informal landscape of social media is a daunting task for natural language processing. Between slang, abbreviations, and the sheer brevity of tweets, traditional text-mining often falls short. SoulMate, a new framework presented in this paper, proposes a solution by embedding not just what users say, but when they say it.

TL;DR

SoulMate is a novel framework for linking authors of short-text content by combining textual embeddings with multi-aspect temporal context (hours, days, seasons). By learning how word relationships shift over time and dynamically extracting "hidden concepts" through unsupervised clustering, it identifies highly correlated user communities more accurately than traditional models.

What Problem Does This Paper Address?

The authors identify four major hurdles in linking authors of short texts like tweets:

  1. Mismatched Content: Informal language (e.g., "arvo" for afternoon) and typos make direct word matching unreliable.
  2. Temporal Skewness: Word proximity patterns change over time. For instance, "Cold" and "Drink" are much more likely to appear together in summer than in winter.
  3. Conceptual Gaps: Two authors might use different words but talk about the same underlying concept (e.g., "overconsumption").
  4. Knowledge Base Bias: Relying on external databases like Wikipedia can deviate from the "real nature" and specific slang of a microblog corpus.

Methods

The SoulMate framework operates in two main phases:

Offline Phase: Building the Foundation

  • Temporal Slabs: The system uses Hierarchical Agglomerative Clustering (HAC) to group similar time periods into "slabs." For example, it might group Monday through Friday as a "weekday" slab because user behavior is consistent across those days (see Figure 3).
  • TCBOW (Temporal Continuous Bag of Words): This is an extension of the standard CBOW model (Figure 6). It learns word vectors specifically for different temporal slabs, capturing how word meanings and associations evolve across hours, days, and seasons.
  • Concept Discovery: Instead of using a fixed external knowledge base, SoulMate clusters tweet vectors using methods like DBScan or K-Medoids to find latent concepts directly from the data.
The CBOW architecture serves as the baseline for the proposed temporal-textual embedding.

Online Phase: Linking Authors

  • Contextual Similarity: It generates "context vectors" for authors that include both content (the words used) and concepts (the clusters they belong to).
  • SW-MST (Stack-Wise Maximum Spanning Tree): To find communities, the authors developed a graph-cutting algorithm that extracts the most tightly connected subgraphs of authors from a similarity graph.
The overall SoulMate framework, showing the transition from offline embedding to online subgraph mining.

Datasets

The researchers evaluated SoulMate using a massive real-world dataset:

  • Twitter Dataset: 8 million English tweets collected in Australia over a full year.
  • Filtered Subset: Approximately 1 million geo-tagged tweets from 4,000 users were used for the core experiments to ensure high-quality temporal and spatial data.

Key Insights and Ideas

  • Time is a Feature, Not a Nuisance: Word co-occurrence is highly dependent on temporal facets. Figure 1 illustrates how "Cold+Drink" peaks in summer, while "Drive+Work" follows strict hourly patterns. Ignoring this leads to incorrect similarity scores.
  • Self-Sourced Concepts: By using unsupervised clustering on the tweet corpus itself, the model avoids the bias of external knowledge bases and better captures the specific vernacular of the user group.
  • Level vs. Depth Similarity: The model doesn't just look at one time facet; it looks at how they nest (e.g., an hour within a specific day slab), providing a "multi-aspect" view of the data.
Heatmaps showing how word pair probabilities shift across different hours and seasons.

Key Results

  • Superior Embeddings: The TCBOW model outperformed standard CBOW, Skip-gram, and GloVe in word analogy tests, proving more resistant to the noise typical of microblogs.
  • Higher Precision: The combined "SoulMateJoint" model achieved a precision of 0.67 for high textual and conceptual similarity, significantly outperforming baselines like TF-IDF (0.21) and Exact Matching (0.39).
  • Semantic Recovery: Even when authors used completely different words (low textual similarity), the concept-based module could still detect correlations with 30% precision, whereas traditional models failed entirely (near 0%).

Limitations and Future Work

The authors acknowledge that the significance of a tweet (e.g., its popularity or "weight") isn't currently factored into the concept extraction. They suggest that future iterations could grant higher importance to more influential or widely shared content. Additionally, the number of discoverable concepts is currently limited by the size of the provided dataset; larger corpora would likely yield even more granular conceptual linking.

Summary updated

Abstract

Linking authors of short-text contents has important usages in many applications, including Named Entity Recognition (NER) and human community detection. However, certain challenges lie ahead. Firstly, the input short-text contents are noisy, ambiguous, and do not follow the grammatical rules. Secondly, traditional text mining methods fail to effectively extract concepts through words and phrases. Thirdly, the textual contents are temporally skewed, which can affect the semantic understanding by multiple time facets. Finally, using the complementary knowledge-bases makes the results biased to the content of the external database and deviates the understanding and interpretation away from the real nature of the given short text corpus. To overcome these challenges, we devise a neural network-based temporal-textual framework that generates the tightly connected author subgraphs from microblog short-text contents. Our approach, on the one hand, computes the relevance score (edge weight) between the authors through considering a portmanteau of contents and concepts, and on the other hand, employs a stack-wise graph cutting algorithm to extract the communities of the related authors. Experimental results show that compared to other knowledge-centered competitors, our multi-aspect vector space model can achieve a higher performance in linking short-text authors. Additionally, given the author linking task, the more comprehensive the dataset is, the higher the significance of the extracted concepts will be.
BibTeX
@article{pour2022soulmate,
  author = {Saeed Najafi Pour and Saeid Hosseini and Wen Hua and Mohammad Reza Kangavari and Xiaofang Zhou},
  title = {{SoulMate: Short-Text Author Linking Through Multi-Aspect Temporal-Textual Embedding}},
  journal = {IEEE Trans. Knowl. Data Eng.},
  year = {2022},
  doi = {10.1109/tkde.2020.2982148},
}
Figure