TL;DR
STEALTH is a breakthrough in linguistic steganography that allows hidden messages to be reliably decoded even when the sender and receiver use different hardware, quantization levels, or entirely different AI model families. By replacing fragile "bit-exact" requirements with a robust "exponential race" sampler and rateless spinal codes, it achieves provable security without the deployment headaches of previous methods.
What Problem Does This Paper Address?
Linguistic steganography hides secret data inside text generated by Large Language Models (LLMs). To a casual observer (or a "warden"), the text looks like a normal, benign AI-generated comment or chat message.
Historically, these schemes have relied on a "bit-exact" assumption: the receiver must be able to recompute the sender's token probability distributions exactly, down to the last bit. However, modern deep learning hardware is notoriously non-deterministic. Floating-point math isn't associative, meaning that changing a GPU architecture (e.g., from an NVIDIA Ampere to a Hopper), switching from 16-bit to 4-bit quantization, or even just scoring text in parallel instead of token-by-token can subtly shift those probabilities.
As the authors demonstrate, these tiny shifts are catastrophic for traditional schemes like Arithmetic Coding or Discop, causing decoding to fail 100% of the time.
Methods
The authors introduce STEALTH, which stands for Steganographic Text Encoding Agnostic to Low-level Tensor Hardware. It uses three key components to achieve robustness:
- The Exponential Race Sampler: Instead of standard sampling, the sender uses a keyed pseudorandom "exponential race" (a variant of the Gumbel-max trick). Tokens "race" based on their probability and keyed noise. This is robust because a small probability mismatch only flips the winning token if the race was extremely close.
- Rateless Spinal Codes: The message is split into chunks and hashed into a "spine." Each token in the generated text carries evidence for a specific message chunk. Because the code is "rateless," the sender can simply keep generating text until the receiver has enough evidence to decode.
- Beam Search Decoding: The receiver doesn't expect a perfect match. They use a beam search with a "soft" likelihood function. If a token was flipped due to a hardware mismatch, the beam search can often find the correct path by looking at the evidence provided by surrounding tokens.
- Byte-Keyed Noise: To work across different model families (which have different vocabularies and tokenizers), STEALTH can key its noise based on the UTF-8 bytes of a token rather than its vocabulary index.
Datasets
The researchers evaluated STEALTH using:
- Models: Three major model families: Gemma-2 (9B/2B), Llama-3.1/3.2 (8B/3B), and Qwen2.5 (7B/1.5B).
- Prompts: 200 news articles from the CNN/DailyMail dataset were used to provide realistic contexts for generated comments.
- Hardware: Experiments were run across different NVIDIA architectures: Ampere (RTX A6000) and Hopper (H100).
Key Insights and Ideas
- The "Bit-Exact" Myth: The paper proves that recomputing exact probabilities is nearly impossible in heterogeneous environments. Even "hardening" schemes that use 64-bit floats are slow and easily detectable by a warden who knows the model.
- Localizing Errors: In traditional schemes, one wrong bit at the start of a message corrupts everything that follows. In STEALTH, a mismatch at one token is just a local "flip" that doesn't necessarily impact the rest of the sequence.
- Cross-Family Communication: One of the most surprising results is that a sender using Llama-3 can communicate with a receiver using Qwen2.5. Even though their tokenizers split the text differently, the byte-keyed noise and spinal code allow the receiver to "re-sync" and extract the message.
Key Results
- 100% Robustness: While bit-exact baselines failed completely under hardware or quantization changes, STEALTH successfully decoded 100% of messages across every tested configuration, including 4-bit NF4 quantization and cross-GPU transfers.
- High Payload: STEALTH delivers 0.19–0.26 bits per token for 32-bit messages. This is an order of magnitude more efficient than previous robust (but receiver-free) schemes like Zamir's.
- Undetectability: The authors tested STEALTH against state-of-the-art steganalyzers (including fine-tuned RoBERTa models). The detectors performed no better than random chance (AUC ~0.5) at distinguishing stegotext from honest model output.
Limitations and Future Work
- Efficiency Trade-off: While robust, STEALTH requires more tokens to convey a message than a theoretical (but fragile) bit-exact scheme that reaches the entropy limit.
- Stopping Rule Uncertainty: A sender might not know exactly how much text their receiver needs if they don't know the receiver's specific hardware setup. The authors suggest senders can add a "safety margin" of extra tokens.
- Future Directions: The authors suggest exploring more advanced beam search techniques or optimizing the spinal code parameters to further increase the payload-to-robustness ratio.
Summary updated