Semantic-Preserving Linguistic Steganography by Pivot Translation and Semantic-Aware Bins Coding

Tianyu Yang, Hanzhou Wu, Biao Yi, Guorui Feng, Xinpeng Zhang

This paper presents a sophisticated approach to linguistic steganography—the art of hiding secret data within natural language text. By combining machine translation "pivoting" with a novel semantic-aware coding scheme, the authors bridge the gap between high-capacity data hiding and the need to maintain the original meaning of the text.

TL;DR

The researchers propose a method that hides secret information by translating a sentence into a "pivot" language (e.g., German) and then back into the original language (English). By using a new "Semantic-Aware Bins" (SaBins) coding strategy during the final translation step, they achieve high data payloads while ensuring the resulting text remains semantically consistent with the original.

What Problem Does This Paper Address?

Linguistic steganography (LS) typically falls into two camps:

  1. Modification-based (MLS): These methods tweak existing text (like swapping "happy" for "joyful"). They preserve meaning well but can only hide a tiny amount of data.
  2. Generation-based (GLS): These use language models to generate text from scratch. While they can hide a lot of data, the resulting text often lacks "semantic consistency"—it might be fluent but completely irrelevant to any real-world context, making it suspicious.

The challenge is creating a "best of both worlds" system: one that can hide a significant amount of data (high payload) without changing the core message of the cover text or making the writing look like "word salad" to automated detection tools (steganalysis).

Methods

The authors introduce a framework called En2Ge2En (English-to-German-to-English) combined with SaBins coding.

  • Pivot Translation: Instead of just generating text, the system takes a "cover" sentence in English and translates it into German (the pivot). This German version acts as a semantic anchor.
  • The Second Leg (Ge2En): During the translation back to English, the model must choose which words to output. This is where the data is hidden.
  • Semantic-Aware Bins (SaBins): Standard methods (like "Bins" coding) randomly assign words to binary groups to represent bits. This often forces the model to pick a word that doesn't fit the context just to satisfy the bit requirement. SaBins improves this by using WordNet to group synonyms together in the same "bins." This ensures that when the model is forced to choose a word to represent a specific bit string, it has a pool of semantically appropriate options to choose from.
  • Self-Supervised Training: To prevent the model from sounding "robotic," the authors trained the second translation model (Ge2En) specifically on the outputs of the first model (En2Ge), ensuring the two halves of the system are statistically aligned.

Datasets

  • WMT 2016 English-German: A massive dataset of ~4.5 million sentence pairs used to train the Transformer-based translation models.
  • WordNet: A lexical database used to identify synonym relationships for the SaBins coding scheme.
  • Evaluation Set: 10,000 natural English texts were used to test the system's performance against various metrics.

Key Insights and Ideas

  • Translation as Paraphrasing: The core "aha moment" is treating translation as a high-entropy paraphrasing tool. Because there are many ways to translate a single sentence, that "extra" space in language can be harvested to hide data.
  • Meaning-Preserving Constraints: By grouping synonyms in the coding bins, the authors effectively "guide" the steganographic process. It’s no longer a random choice between unrelated words; it’s a choice between different ways to say the same thing.
  • The Payload-Quality Trade-off: The authors use a "step value" ($s$) to control how often they embed bits. This allows a user to dial the system from "maximum security" (low payload, high quality) to "maximum capacity" (high payload, lower quality).

Key Results

  • Superior Semantic Preservation: Using BERTScore (which measures semantic similarity in vector space), the proposed method significantly outperformed traditional generation-based methods. Even at higher bitrates, the stego-text remained remarkably close in meaning to the original cover text.
  • High Payload: The method achieved much higher "Bits Per Word" (BPW) than traditional modification-based methods, reaching up to 3.0 BPW in some configurations.
  • Steganalysis Resistance: In tests against state-of-the-art deep learning "stegalyzers," the SaBins method was harder to detect than the baseline Bins method. For instance, at a payload of 0.33 BPW, detection accuracy dropped from ~83.5% (baseline) to ~81.1% (proposed).

Limitations and Future Work

  • Synonym Coverage: The current system relies on WordNet. If a word isn't in the synonym database, the "semantic-aware" benefit is lost. The authors suggest more advanced clustering of word embeddings as a future improvement.
  • The "Model Gap": Even without hiding data, machine-translated text has a different statistical signature than human-written text. Expert stegalyzers can still detect the presence of a model, even if they can't prove data is hidden.
  • Complexity: Training two large Transformer models is computationally expensive, which might limit the system's use on low-power mobile devices.

The authors conclude that by making steganography "semantic-aware," we can move closer to covert communications that are not only high-capacity but also indistinguishable from natural human discourse.

Summary updated

Abstract

Linguistic steganography (LS) aims to embed secret information into a highly encoded text for covert communication. It can be roughly divided to two main categories, i.e., modification based LS (MLS) and generation based LS (GLS). MLS embeds secret data by slightly modifying a given text without impairing the meaning of the text, whereas GLS uses a well trained language model to directly generate a text carrying secret data. A common disadvantage for MLS methods is that the embedding payload is very small, whose return is well preserving the semantic quality of the text. In contrast, GLS enables the data hider to embed a large payload, which has to pay the high price of uncontrollable semantics. In this article, we propose a novel LS method to modify a given text by pivoting it between two different languages and embed secret data using a semantic-aware information encoding strategy. Our purpose is to alter the expression of the given text, enabling a large payload to be embedded while keeping the semantic information unchanged. Experiments have shown that the proposed work not only achieves a large embedding payload, but also shows superior performance in maintaining the semantic consistency and resisting linguistic steganalysis.
BibTeX
@article{yang2024semantic,
  author = {Tianyu Yang and Hanzhou Wu and Biao Yi and Guorui Feng and Xinpeng Zhang},
  title = {{Semantic-Preserving Linguistic Steganography by Pivot Translation and Semantic-Aware Bins Coding}},
  journal = {IEEE Trans. Dependable Secur. Comput.},
  year = {2024},
  url = {https://openreview.net/forum?id=G1B31xdzmZ},
}

Loading related content...

Figure