Linking authors across the noisy, informal landscape of social media is a daunting task for natural language processing. Between slang, abbreviations, and the sheer brevity of tweets, traditional text-mining often falls short. SoulMate, a new framework presented in this paper, proposes a solution by embedding not just what users say, but when they say it.
TL;DR
SoulMate is a novel framework for linking authors of short-text content by combining textual embeddings with multi-aspect temporal context (hours, days, seasons). By learning how word relationships shift over time and dynamically extracting "hidden concepts" through unsupervised clustering, it identifies highly correlated user communities more accurately than traditional models.
What Problem Does This Paper Address?
The authors identify four major hurdles in linking authors of short texts like tweets:
- Mismatched Content: Informal language (e.g., "arvo" for afternoon) and typos make direct word matching unreliable.
- Temporal Skewness: Word proximity patterns change over time. For instance, "Cold" and "Drink" are much more likely to appear together in summer than in winter.
- Conceptual Gaps: Two authors might use different words but talk about the same underlying concept (e.g., "overconsumption").
- Knowledge Base Bias: Relying on external databases like Wikipedia can deviate from the "real nature" and specific slang of a microblog corpus.
Methods
The SoulMate framework operates in two main phases:
Offline Phase: Building the Foundation
- Temporal Slabs: The system uses Hierarchical Agglomerative Clustering (HAC) to group similar time periods into "slabs." For example, it might group Monday through Friday as a "weekday" slab because user behavior is consistent across those days (see Figure 3).
- TCBOW (Temporal Continuous Bag of Words): This is an extension of the standard CBOW model (Figure 6). It learns word vectors specifically for different temporal slabs, capturing how word meanings and associations evolve across hours, days, and seasons.
- Concept Discovery: Instead of using a fixed external knowledge base, SoulMate clusters tweet vectors using methods like DBScan or K-Medoids to find latent concepts directly from the data.
Online Phase: Linking Authors
- Contextual Similarity: It generates "context vectors" for authors that include both content (the words used) and concepts (the clusters they belong to).
- SW-MST (Stack-Wise Maximum Spanning Tree): To find communities, the authors developed a graph-cutting algorithm that extracts the most tightly connected subgraphs of authors from a similarity graph.
Datasets
The researchers evaluated SoulMate using a massive real-world dataset:
- Twitter Dataset: 8 million English tweets collected in Australia over a full year.
- Filtered Subset: Approximately 1 million geo-tagged tweets from 4,000 users were used for the core experiments to ensure high-quality temporal and spatial data.
Key Insights and Ideas
- Time is a Feature, Not a Nuisance: Word co-occurrence is highly dependent on temporal facets. Figure 1 illustrates how "Cold+Drink" peaks in summer, while "Drive+Work" follows strict hourly patterns. Ignoring this leads to incorrect similarity scores.
- Self-Sourced Concepts: By using unsupervised clustering on the tweet corpus itself, the model avoids the bias of external knowledge bases and better captures the specific vernacular of the user group.
- Level vs. Depth Similarity: The model doesn't just look at one time facet; it looks at how they nest (e.g., an hour within a specific day slab), providing a "multi-aspect" view of the data.
Key Results
- Superior Embeddings: The TCBOW model outperformed standard CBOW, Skip-gram, and GloVe in word analogy tests, proving more resistant to the noise typical of microblogs.
- Higher Precision: The combined "SoulMateJoint" model achieved a precision of 0.67 for high textual and conceptual similarity, significantly outperforming baselines like TF-IDF (0.21) and Exact Matching (0.39).
- Semantic Recovery: Even when authors used completely different words (low textual similarity), the concept-based module could still detect correlations with 30% precision, whereas traditional models failed entirely (near 0%).
Limitations and Future Work
The authors acknowledge that the significance of a tweet (e.g., its popularity or "weight") isn't currently factored into the concept extraction. They suggest that future iterations could grant higher importance to more influential or widely shared content. Additionally, the number of discoverable concepts is currently limited by the size of the provided dataset; larger corpora would likely yield even more granular conceptual linking.
Summary updated