You are your Metadata: Identification and Obfuscation of Social Media Users using Metadata Information

Beatrice Perez, Mirco Musolesi, Gianluca Stringhini

TL;DR

This paper reveals that metadata on social media platforms like Twitter is surprisingly unique and can be used to identify users with high accuracy, even when content is ignored. Attempts to obfuscate this metadata proved largely ineffective, highlighting a significant privacy risk for users.

What Problem Does This Paper Address?

Online social networks (OSNs) generate vast amounts of data, and while much research has focused on identifying users from the content of their posts (text, images), the privacy implications of metadata have been largely overlooked. Metadata, such as account creation time, number of followers, and tweet timestamps, is often considered less sensitive. However, this paper questions that assumption, investigating whether this seemingly innocuous information can uniquely identify users and if it can be effectively obscured to protect privacy.

Methods

The researchers framed user identification as a classification problem. They employed three state-of-the-art machine learning algorithms: Multinomial Logistic Regression (MLR), Random Forest (RF), and K-Nearest Neighbors (KNN). They systematically explored combinations of various metadata fields (features) to build "behavioral signatures" for users. To assess obfuscation effectiveness, they applied data randomization and anonymization techniques, specifically rounding data values, and measured the impact on identification accuracy. They also analyzed the computational cost (execution time) of these classification methods.

Datasets

The study utilized a dataset of approximately 151 million tweets from over 11 million users collected via the Twitter Streaming Public API between October 2015 and January 2016. For their analysis, they filtered this down to users with at least 200 tweets, resulting in a final dataset of 5.4 million users. This dataset is not a newly introduced benchmark but rather a collection of publicly available Twitter data.

Key Insights and Ideas

  • Metadata is a powerful identifier: The paper strongly suggests that metadata, often dismissed as non-sensitive, is highly unique and can effectively distinguish users.
  • Content isn't everything: This research shifts the focus from analyzing post content to understanding the privacy risks embedded in the "behind-the-scenes" data.
  • Obfuscation is surprisingly difficult: Simple data perturbation methods, like rounding, are not enough to prevent identification, even when a significant portion of the data is altered.
  • KNN shines for identification: Among the tested classifiers, KNN demonstrated a strong balance of accuracy and computational efficiency for user identification.
  • Scalability is a challenge: As the number of users (classes) increases, the computational cost of multi-class classification becomes a significant bottleneck, particularly for MLR.

Key Results

  • High Identification Accuracy: Using a supervised learning algorithm (specifically KNN), the researchers achieved approximately 96.7% accuracy in identifying a user within a group of 10,000.
  • Top 10 Candidates Boost Accuracy: When considering the top 10 most likely candidates, the accuracy increased to an impressive 99.22%.
  • Ineffective Obfuscation: Even after perturbing 60% of the training data, it was still possible to classify users with an accuracy higher than 95%.
  • Account Creation Time is Highly Discriminative: The "Account Creation Time" (ACT) feature, which includes date and time components, proved to be a very strong predictor, yielding near-perfect accuracy (99.98%) when used alone with KNN.
  • Execution Time Bottleneck: MLR was significantly slower than KNN and RF, especially as the number of users increased, highlighting the challenge of scaling multi-class classification.

Limitations and Future Work

  • Twitter as a Case Study: While the methods are presented as generalizable, the study's findings are based solely on Twitter data. Future work could explore other platforms.
  • Obfuscation Techniques: The paper tested only basic randomization (rounding). More sophisticated obfuscation techniques might offer better privacy guarantees, though the paper suggests a trade-off between privacy and data utility.
  • Scalability Solution: The authors implemented a "divide and conquer" algorithm to address the scalability issue of multi-class classification, which they demonstrated to be effective without sacrificing accuracy. This algorithmic contribution could be further explored.
  • Ethical Considerations: While the study acknowledges ethical concerns and obtained IRB approval, the implications of such powerful identification capabilities on user privacy warrant ongoing discussion and responsible data handling practices.

Summary updated

Abstract

Metadata are associated to most of the information we produce in our daily interactions and communication in the digital world. Yet, surprisingly, metadata are often still categorized as non-sensitive. Indeed, in the past, researchers and practitioners have mainly focused on the problem of the identification of a user from the content of a message. In this paper, we use Twitter as a case study to quantify the uniqueness of the association between metadata and user identity and to understand the effectiveness of potential obfuscation strategies. More specifically, we analyze atomic fields in the metadata and systematically combine them in an effort to classify new tweets as belonging to an account using different machine learning algorithms of increasing complexity. We demonstrate that, through the application of a supervised learning algorithm, we are able to identify any user in a group of 10,000 with approximately 96.7% accuracy. Moreover, if we broaden the scope of our search and consider the 10 most likely candidates we increase the accuracy of the model to 99.22%. We also found that data obfuscation is hard and ineffective for this type of data: even after perturbing 60% of the training data, it is still possible to classify users with an accuracy higher than 95%. These results have strong implications in terms of the design of metadata obfuscation strategies, for example for data set release, not only for Twitter, but, more generally, for most social media platforms.
BibTeX
@inproceedings{perez2018you,
  author = {Beatrice Perez and Mirco Musolesi and Gianluca Stringhini},
  title = {{You are your Metadata: Identification and Obfuscation of Social Media Users using Metadata Information}},
  booktitle = {ICWSM},
  year = {2018},
  url = {https://openreview.net/forum?id=90P9-1v-9Y},
}
Figure