Provably secure image steganography hides a message by letting the message bits choose which image tokens a generative model samples. A token here is one entry from the discrete visual vocabulary of a vector-quantized (VQ) tokenizer, a network that turns each image patch into a codebook index. Because the stego image is an ordinary sample from the model, steganalysis detectors do no better than chance. The weakness is the channel. After JPEG compression the receiver re-tokenizes a slightly altered image, some tokens come back wrong, and the message is corrupted. The strongest current fix, CMSTEG, runs 10,000 gradient-descent steps per image at the receiver. It also sends error-correction hints in a second, text-based steganographic channel, so the system breaks if that caption is stripped.
We propose that the receiver repair corrupted tokens with a masked image model instead: a bidirectional transformer trained, in the style of BEiT, to predict hidden tokens from the visible ones around them. The receiver flags tokens whose identity the channel has made uncertain, masks them, and decodes each one by combining two sources of evidence. The first is what the degraded pixels say. The second is what the surrounding tokens and a caption, which the receiver generates from the received image itself, say the token should be. The sender is untouched, so provable security is inherited unchanged. There is no side channel, and decoding takes a few forward passes instead of thousands of optimization steps.
Core Claim
Distribution-preserving embedding makes stego tokens distributed exactly as the generator's samples. A model trained on that same token distribution is therefore a correct Bayesian prior for the sender's true tokens. This lets the receiver decode by maximum a posteriori estimation rather than by inverting the channel through optimization. For each flagged position $i$ the receiver picks
$$\hat q_i = \arg\max_{q} \Big[\log p_{\text{ch}}(y_i \mid q) + \log p_{\text{mask}}(q \mid \hat q_{\setminus i}, c)\Big],$$
where $q$ ranges over codebook entries, $y_i$ is the tokenizer encoder's continuous output on the received patch, $p_{\text{ch}}$ is a calibrated channel likelihood (a temperature-scaled softmax over codebook distances, optionally refined by a small learned head), $\hat q_{\setminus i}$ are the current estimates at all other positions, and $c$ is the caption. The receiver repeats this for a few rounds, unmasking the most confident positions first. Any errors that remain are absorbed by a standard error-correcting code on the message.
The claim is falsifiable. At JPEG quality 75 on 384×384 images, this receiver should match CMSTEG's reported token recovery rate (above 98%) and effective capacity (above 3,200 bits). It should do so without the text side channel and with at least two orders of magnitude less receiver compute.
There is a real tension that the work must measure rather than assume. In provably secure embedding, a position carries roughly as many bits as its predictive entropy. The positions that carry the most message are therefore the ones any prior predicts worst. The prior helps most where JPEG flips low-entropy tokens, such as smooth regions where a near-tie between codebook entries is broken the wrong way. It helps little where it flips high-entropy texture tokens. How errors and entropy co-occur decides whether the idea works.
Research Program
The pipeline reuses the sender of an ImageNet-trained autoregressive VQ generator of the type CMSTEG uses. We will run a smaller open model if the 3-billion-parameter one is unavailable, and report CMSTEG numbers only where the settings match.
On top of the generator's frozen tokenizer we train a bidirectional masked token model on token maps of the generator's own samples, not of real images. This keeps the prior matched to the stego distribution. Masking follows the MIM practice of high ratios. We also include a structured component that masks the spatially clustered positions JPEG actually corrupts, measured by pushing samples through the real codec. This borrows the idea, from SemMIM's text-guided masking and AMLRIS's alignment-aware masking, that masking should target informative regions. Here the target is regions damaged by the channel rather than regions relevant to the text.
The model cross-attends to a caption generated at the receiver from the received image. This is how we use the cross-modal gap. Text is the compact, discrete description that survives JPEG almost perfectly even when patch-level tokens do not. CMSTEG transmits text as a hint. We instead recompute the semantics from the image at the receiver, so nothing extra is sent. Learning Robust Visual-Semantic Embeddings and A Visual Attention Grounding Neural Model for Multimodal Machine Translation find that global text vectors are weak, coarse signals. That is why the caption enters the model through token-level cross-attention and not as a pooled vector.
Three analyses accompany the system. The first maps, per position, the generator's entropy, the probability that JPEG flips the token, and the prior's top-1 accuracy, which directly tests the tension above. The second calibrates $p_{\text{ch}}$ across JPEG qualities and ImageNet-C corruptions. The third audits security by confirming that steganalysis error stays near 0.5 with SRNet. It should, since stego images are generated exactly as before.
Baselines
The naive reference point is hard re-tokenization: the receiver encodes the received image, takes the nearest codebook entry at each position, and decodes. CMSTEG reports that the effective capacity of prior provably secure methods falls to nearly zero under JPEG quality 75. This is the floor we must lift. It costs one encoder pass.
The strongest existing method is CMSTEG with discrete token optimization and cross-modal text error correction. The file reports token recovery above 98% and effective capacity above 3,200 bits under JPEG quality 75 at 384×384, with steganalysis error about 0.5 (CMSTEG). We will run it on the same generator, images, and channel, and report wall-clock time on the same GPU. Beating it means equal or higher effective capacity with no text channel and under one percent of its receiver time. Matching its capacity at a small fraction of the compute already counts as a win.
A second strong competitor is message-level error correction alone. We wrap the message in a BCH or LDPC code and pick the code rate to maximize net capacity under the same channel, with an equal tuning budget over rates. If its net capacity matches ours, the masked prior is unnecessary.
A third is a robust tokenizer encoder, fine-tuned so that JPEG-degraded patches map to the clean token. It is trained with the same data and GPU hours as our masked model. In the spirit of RoSteALS's differentiable noise training, it moves robustness into the encoder rather than into a prior.
Two ablations isolate the key ingredient. One replaces the bidirectional prior with the generator's own causal conditional, which uses only past context, to test whether bidirectional masking matters. The other removes the caption to test whether the cross-modal conditioning adds anything beyond image context. For ImageNet, the class label may already carry the caption's information.
Steganography schemes from other families, CRoSS and MDDM, are reported as context on cost only. They hide different payloads, use iterative diffusion inversion at the receiver, and do not offer the same provable-security guarantee. CRoSS reports 21.02 dB recovery PSNR under Gaussian noise with σ=30. MDDM reports close to 100% extraction under JPEG quality 95. Neither is directly comparable on capacity at quality 75.
Why not just…?
Why not just add a strong error-correcting code and skip the learned prior? This is a real threat and may win at mild compression. A code pays a fixed parity overhead whatever the content. The prior instead exploits the fact that most flipped tokens are predictable from context. The two combine rather than compete, so the honest question is whether the prior raises net capacity at the best code rate. The experiment sweeps code rates for both systems. If prior plus code does not beat code alone by a clear margin at quality 75, the claim fails.
Why not just fine-tune the tokenizer encoder to be JPEG-robust? This may remove much of the error at its source, and it is cheap at test time. It does have to generalize across corruptions it was not trained on. The file notes that methods trained with fixed noise simulators degrade on unseen corruptions, and CRoSS reports baselines dropping to about 13 dB under heavy noise (CRoSS). The prior, in contrast, is independent of the channel except through $p_{\text{ch}}$. We test both alone and stacked, on JPEG and on held-out ImageNet-C corruptions. If the robust encoder alone matches the stack, the proposal shrinks to a recommendation for that encoder.
Why not just reuse CMSTEG's optimization, or the generator's own causal probabilities, as the prior? The optimization works, but it costs 10,000 steps per image and needs the side channel. The causal conditional is free, but at position $i$ it ignores everything after $i$, including the tokens below and to the right that constrain a patch most. The causal-prior ablation measures exactly this gap. If it closes the gap, the masked model is not needed.
Proposed Experiments
All experiments use 384×384 generations from the ImageNet-trained generator. The channel is real libjpeg compression at qualities 95, 85, 75, and 50, plus the ImageNet-C corruption suite used by RoSteALS, resizing, and composite degradations. The metrics are token recovery rate, net effective capacity after error correction, receiver wall-clock time and FLOPs, and SRNet steganalysis error.
The first decisive check is the head-to-head at JPEG quality 75 against CMSTEG's reported figures of above 98% recovery and above 3,200 bits. We remove CMSTEG's text channel for both systems, then restore it for CMSTEG only, which shows what the side channel is worth.
The second is the entropy–error–prior map. It predicts where the method can and cannot help. If corrupted tokens sit mostly at high-entropy positions, the prior's ceiling is low, and we report that as a negative finding.
The third is the caption ablation, run both on ImageNet and on a captioned generator trained on COCO. On ImageNet, class labels may make text redundant. On COCO, multi-object scenes should make receiver-side captions informative. A gain that appears only on COCO would locate where cross-modal conditioning matters.
The last is a stress test with severe composite corruption. It shows whether the closed-form posterior degrades gracefully when $p_{\text{ch}}$ is miscalibrated, and whether recalibrating only the temperature on a handful of received images restores it.
References
- Provably Secure Image Robust Steganography via Cross-modal Error Correction
- BEiT: BERT Pre-Training of Image Transformers
- Semantics-enhanced Cross-modal Masked Image Modeling for Vision-Language Pre-training
- AMLRIS: Alignment-aware Masked Learning for Referring Image Segmentation
- Learning Robust Visual-Semantic Embeddings
- A Visual Attention Grounding Neural Model for Multimodal Machine Translation
- RoSteALS: Robust Steganography using Autoencoder Latent Space
- CRoSS: Diffusion Model Makes Controllable, Robust and Secure Image Steganography
- MDDM: Practical Message-Driven Generative Image Steganography Based on Diffusion Models