Detection of Steganographic Techniques on Twitter

Alex Wilson, Phil Blunsom, Andrew D. KerEMNLP 2015 · OpenReview

Abstract

We propose a method to detect hidden data in English text. We target a system previously thought secure, which hides messages in tweets. The method brings ideas from image steganalysis into the linguistic domain, including the training of a feature-rich model for detection. To identify Twitter users guilty of steganography, we aggregate evidence; a first, in any domain. We test our system on a set of 1M steganographic tweets, and show it to be effective.

Introduction

Steganography is the practice of concealing a secret message (the payload) within an innocuous "cover" medium, such as an image or a text file. While image-based steganography has been extensively researched, linguistic steganography—hiding data within natural language—presents unique challenges. Text has a significantly lower information capacity than images, and subtle changes in wording can easily disrupt the semantic or syntactic flow, making detection by human judges difficult.

This research focuses on CoverTweet, a state-of-the-art linguistic steganography system that utilizes the Paraphrase Database (PPDB) to modify tweets. Although CoverTweet was previously deemed secure against human evaluation, this paper investigates whether a machine-learning approach, specifically one that aggregates evidence over time, can reliably detect its use.

Methodology

The detection problem is framed as a binary classification task: distinguishing between an "innocent" user and an "active" steganographer. The authors employ a feature-rich model combined with a linear ensemble classifier.

Feature Engineering

A total of 812 features were developed to capture statistical anomalies:

  • Linguistic Baselines: Word counts, character distributions, and stop-word frequencies.
  • N-gram Likelihoods: Using a 5-gram language model to calculate the log-likelihood of word sequences.
  • Structural Patterns: A 10-gram model of word lengths was used to detect unusual patterns resulting from synonym substitution.
  • Informed Warden Features: Adhering to Kerckhoffs’ principle, the detector assumes knowledge of the steganographic algorithm. Features were extracted based on the PPDB rules used by CoverTweet, such as the proportion of words in a tweet that appear in the database.

Pooled Steganalysis

The core innovation of this work is the application of "pooled steganalysis" to the linguistic domain. Rather than analyzing a single tweet in isolation—which often yields insufficient evidence—the "warden" (detector) aggregates features across a batch of tweets from the same user. By calculating the mean, variance, maximum, and minimum of features across these batches, the system can identify subtle but persistent statistical drifts that characterize steganographic activity.

Results and Discussion

The effectiveness of the proposed system was evaluated across several datasets, including Automatic CoverTweet (A-CT), Manual CoverTweet (M-CT), and the baseline T-Lex system.

The Impact of Batch Size

The primary finding is that detection accuracy is highly dependent on the number of observations. As shown in Figure 1, the error rate for all systems decreases significantly as the batch size increases.

Error rate vs. Batch size for various steganographic systems.

For the Automatic CoverTweet (A-CT) system, the error rate drops sharply, reaching near-zero levels as the batch size approaches 1,000. Even the Manual CoverTweet (M-CT) system, where a human operator selects the most natural-looking tweets, shows a clear downward trend in error rate as more data is pooled, although it remains more resilient than the fully automated version.

Individual vs. Pooled Detection

The limitations of single-tweet analysis are starkly illustrated by the Area Under the Curve (AUC) metrics. For a single tweet from the Manual CoverTweet dataset, the AUC was a mere 0.509, indicating performance no better than random guessing. However, the power of pooling becomes evident when examining larger batches.

ROC curves comparing individual tweet detection vs. pooled detection (batch of 100).

As seen in Figure 2, the ROC curve for a batch of 100 tweets from the A-CT system shows near-perfect detection (AUC 0.9631). This demonstrates that while individual stego-tweets may appear perfectly natural to humans, the cumulative statistical distribution of words and phrases across multiple messages becomes increasingly anomalous.

Conclusion

This research demonstrates that linguistic steganography on Twitter, even when designed to fool human judges, is vulnerable to feature-rich statistical analysis. The introduction of pooled steganalysis provides a powerful tool for "wardens" to identify active steganographers by aggregating evidence over time. Future work will focus on developing more universal detectors that can maintain high performance even when the specific steganographic system is unknown.

Summary updated

Abstract

We propose a method to detect hidden data in English text. We target a system previously thought secure, which hides messages in tweets. The method brings ideas from image steganalysis into the linguistic domain, including the training of a feature-rich model for detection. To identify Twitter users guilty of steganography, we aggregate evidence; a first, in any do- main. We test our system on a set of 1M steganographic tweets, and show it to be effective.
BibTeX
@inproceedings{wilson2015detection,
  author = {Alex Wilson and Phil Blunsom and Andrew D. Ker},
  title = {{Detection of Steganographic Techniques on Twitter}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2015},
  url = {https://openreview.net/forum?id=H1EUwfMuWS},
}

Loading related content...

Figure