Sampling Inside the Box: Unfolded Langevin Decoding for Quantization-Consistent JPEG Restoration

A JPEG file does not record the image. It records, for every 8×8 block and every frequency, which quantization bin the true coefficient fell into. Any image whose coefficients land in those same bins is an equally valid decoding of the file. The set of such images is a box, an axis-aligned region in the frequency domain. Generative decoders now produce sharp, realistic decodings from heavily compressed files, but they do it by sampling first and correcting afterwards. The strongest of them, High-Perceptual Quality JPEG Decoding via Posterior Sampling, clips its output back into the box as a final step, and its authors report that this clipping damages the textures the generator just produced. We propose a decoder whose samples stay inside the box at every step of their construction. It is built as an unrolled Langevin sampler: a fixed number of noisy gradient steps turned into network layers, with an exact box projection after each step. A single noise-temperature knob moves the decoder, at inference time, from the blurry but low-error average image to sharp posterior samples.

The central observation is simple. The true image always lies in its box, so the true posterior (the distribution of images consistent with the file) sits entirely inside the box. Its mean, the minimum-mean-squared-error estimate, lies inside the box too, because the box is convex. Being consistent therefore costs an ideal decoder nothing. When post-hoc clipping hurts perceptual quality, it is a sign that the sampler put probability mass where the posterior has none. We treat that damage as a measurable symptom of a misspecified sampler, not as an unavoidable "perception-consistency tradeoff." We then ask whether a sampler that is structurally forbidden from leaving the box learns a better posterior than one that is corrected at the end.

Core Claim

An unrolled projected-Langevin network trained adversarially is exactly quantization-consistent by construction. Conditioned on a temperature $\tau$, it traces the perception-distortion curve with one set of weights. The claim is that it matches or beats post-hoc-projected generative decoding on realism, beats it on fidelity to the original image's structure, and runs in a small, fixed number of stages. The claim is falsifiable in a specific way. If a one-shot generator trained with the projection inside its training forward pass reaches the same point, then iterating inside the box is not what matters. In that case the contribution shrinks to controllability and compute, and we will say so.

Relation to Prior Work

Four lines of prior work bear on this proposal, and each plays a different role.

Consistency without the tradeoff. Two dual-domain networks enforce quantization consistency, but only as deterministic regressors trained on squared error. D3 uses a box-constrained loss built from the JPEG quantization table and a one-step sparse-coding module. DMCNN clips predicted DCT coefficients into their bins with a "rectify unit." Both therefore produce the conditional-mean blur that The Perception-Distortion Tradeoff proves is unavoidable for low-distortion estimators. Their projection machinery is what we reuse; their objective is what we replace.

Generative decoders that motivate the proposal. Two stochastic decoders are the direct motivation. Man et al. train an RRDB conditional GAN with first- and second-moment penalties and project afterwards; this is the source of the reported conflict. Explorable Decoding of Compressed Images predicts the in-bin offset directly, so its outputs are consistent by design. It lets a user explore alternatives through a latent control, but offers no principled path between the average image and realistic samples.

Unrolled samplers that supply the method. Deep unfolding of MCMC kernels turns 8 to 64 Langevin steps into a WGAN-trained network with the forward operator in every stage. It assumes linear measurements with Gaussian noise and has no mechanism for box constraints. DGUNet supplies the stage design, with learned gradient steps, multi-scale prior modules and stage-specific weights, but it is deterministic and unconstrained. FlowLPS shows that alternating Langevin exploration with proximal pulls toward the measurements balances realism and fidelity. It is training-free, takes about 264 seconds per image, and was tested only on smooth degradations.

Constraints on the design. Two papers limit what we can do. JPEG Inspired Deep Learning replaces hard rounding with a soft quantizer, which cannot guarantee that outputs stay in their bins; we therefore keep hard projection and rely on it having a well-defined gradient almost everywhere. Human Preference Score shows that FID rewards realistic local texture even when global structure is wrong. So FID alone cannot adjudicate our claim, and we evaluate structure separately.

Research Program

The measurement model is $y = Q(Ax)$, where $A$ is the orthonormal 8×8 block DCT, $Q$ is rounding to the bin grid given by the file's quantization table $\Delta$, and $y$ are the stored bin centers. The consistent set is $\mathcal{C}(y)=\{x : y-\Delta/2 \le Ax \le y+\Delta/2\}$. Because $A$ is orthonormal, projecting onto this set is exact and cheap: transform, clip each coefficient to its bin, and transform back.

Each of the $K$ stages (we will sweep $K$ from 4 to 16) performs one projected Langevin step:

$$x_{k+1} = P_{\mathcal{C}(y)}\!\left(x_k + \eta_k\, s_{\theta_k}(x_k, y, \tau) + \sqrt{2\eta_k \tau}\,\xi_k\right).$$

In this update, $P_{\mathcal{C}(y)}$ is the box projection. $s_{\theta_k}$ is a learned stage-specific prior network, a DGUNet-style multi-scale module that also receives the quantization table. $\eta_k$ is a learned step size, $\xi_k$ is standard Gaussian noise, and $\tau\in[0,1]$ is the temperature. The first stage starts from the plain JPEG decode, which is already inside the box.

Two practical details matter. First, 8-bit pixel rounding and range clipping can push a few coefficients out of their bins. We will therefore alternate the box projection with pixel-range projection and report the exact re-encoding rate: the fraction of outputs that re-compress to the identical bitstream. Second, the gradient of the clip is zero for saturated coefficients. We will compare plain backpropagation with a straight-through variant.

Training samples $\tau$ uniformly for every batch. The loss has three parts: a WGAN adversarial term weighted by $\tau$; a squared-error term on the mean of several samples, which anchors the average to the minimum-error estimate as in Man et al.; and a variance-matching term. At $\tau=0$ the network is trained purely toward the minimum-error estimate. This is what makes one set of weights cover the whole curve.

The theory gives a built-in calibration test. If the decoder is an exact posterior sampler at $\tau=1$, its expected squared error equals twice the minimum mean squared error. The reason is that the sample and the true image are then independent draws from the same posterior, so both deviate from the posterior mean with the same variance. Sample PSNR should therefore sit about 3 dB below the $\tau=0$ PSNR. This mirrors the factor-of-two bound in Rethinking Lossy Compression. A gap much larger than 3 dB means the samples are off-posterior; a gap much smaller means they have collapsed toward the mean.

We will also measure the projection-induced change, the size of the correction when an unconstrained sampler's output is clipped, as a diagnostic of posterior misspecification across all methods.

Baselines

Plain JPEG decode. The naive reference point is the standard JPEG decode. It is consistent and deterministic, and every method must beat it on every metric.

Distortion anchor. The distortion anchor is DMCNN, which reports 29.73 dB PSNR on LIVE1 at quality factor 10. Our $\tau=0$ output should come within a few tenths of a dB of that figure, retrained on our data. If it does not, the unrolled structure costs fidelity and that must be reported.

Strongest competitor. The strongest existing method is Man et al.'s projected posterior sampler, Ours-P. It reports FID ~4.78 on ImageNet-ctest10k at QF 10, against ~13.71 for the previous best method. It is the strongest competitor because it already targets exactly our operating point: consistent, stochastic and realistic. We will retrain it on our training set with a matched parameter count and a matched GPU-hour budget, using the same moment penalties and the same hyperparameter search budget. Beating it means lower FID at equal or better sample PSNR and structural error, with exact consistency.

Key ablation 1: projection-aware one-shot generator. This ablation isolates what the proposal adds. It is the same RRDB generator, but with the box projection inside its training forward pass, so the adversarial loss sees the clipped output.

Key ablation 2: bounded-offset parameterization. This is the Bahat-Michaeli approach, where a tanh-scaled offset keeps each coefficient inside its bin, applied to the same generator.

Key ablation 3: our network without in-loop projection. This is our unrolled network with projection only at the end.

These three ablations separate three possible sources of any gain: iterating, projecting in the loop, and merely training while seeing the constraint.

Training-free guidance. FlowLPS-style Langevin-proximal sampling with the box projection as its proximal step will be run on a 500-image subset only, because of its cost. It is reported as a quality ceiling under unlimited compute. No reported result was found on Lacuna for any training-free guidance method on JPEG restoration.

Why not just…?

Why not just clip the output of a strong generator, as Man et al. do? Their own paper reports that clipping sharp samples degrades their perceptual quality. That is the gap this work targets. The comparison between our decoder and the retrained Ours-P settles whether in-loop projection removes the loss. The projection-induced-change diagnostic shows where along the frequency spectrum the damage occurs.

Why not just train the one-shot generator with the projection in its forward pass? This is the most serious threat. Such a generator can learn to put its texture into coefficients that clipping leaves alone, and it is cheaper than unrolling. If it matches our decoder on FID, PSNR and structural error, we will concede that iteration is not needed for quality. The remaining claims would then be inference-time control of the tradeoff with one network, compute-adjustable depth, and calibration at the 3 dB target. Ablation 1 is designed to decide this directly.

Why not just bound the output by construction, as Explorable Decoding does? Bounded offsets guarantee consistency, but they squash each coefficient through a saturating function independently. This gives no way to move between the minimum-error estimate and posterior samples. It may also bias samples away from bin edges, where true coefficients often lie. Ablation 2 tests this. The per-bin histogram of sample coefficients, compared with the true in-bin histogram estimated from uncompressed data, would reveal the bias.

Why not just use a pretrained diffusion model with box-projected guidance? This probably gives the best quality per image. FlowLPS shows how effective such samplers are, and also that they cost about 264 seconds per image and can drift in identity. We report it as a ceiling. The question for us is how much of that quality a 4-to-16-stage network recovers at millisecond-scale cost. The file suggests millisecond scale is achievable, based on D3's reported ~12 ms and the ~700× speedup reported for unrolled MCMC.

Proposed Experiments

Training and test data. Training follows Man et al. on ImageNet, with quality factors 5 to 30 sampled per batch. Tests use ImageNet-ctest10k and LIVE1 at QF 5, 10 and 20, and FFHQ for faces. No per-benchmark numbers for FFHQ were found on Lacuna, so all FFHQ baselines are retrained.

Metrics. For each method we report sample PSNR and minimum-error PSNR, FID, LPIPS, BiRQA (BiRQA, chosen for its resistance to metric gaming), the exact re-encoding rate, the calibration gap from 3 dB, and runtime. Because FID is blind to structural errors, we add two structural checks. The first is error on edge maps. The second, on FFHQ, is face-identity cosine similarity to the original. We also run a paired forced-choice human study, 200 images per quality factor, comparing our decoder with Ours-P. This is the final test of the perception claim.

Decisive checks. The first is whether our decoder beats projection-aware one-shot training (ablation 1) on FID at matched sample PSNR. The second is whether the whole perception-distortion curve traced by our single network, swept through $\tau$, dominates the curve obtained by retraining Ours-P at several adversarial weights. The third is whether the calibration gap stays near 3 dB across quality factors.

Controlled posterior test. As a sanity check where the posterior can be approximated, we JPEG-compress MNIST at very low quality. We compare samples against an expensive long-run projected Langevin reference chain and measure how well each method's sample distribution matches it inside the box. Two outcomes would retract the central claim: a positive result on MNIST that vanishes on ImageNet, or a projection-aware one-shot generator that matches our decoder everywhere.