A diffusion model is a generator that starts from random noise and removes it step by step. Studies of video diffusion keep finding the same pattern. The noisy early steps decide the coarse layout and motion of a scene, and the quiet late steps fill in texture. Holding structural guidance past the early phase makes the output worse. Gigapixel images, such as whole-slide pathology scans of 100,000 by 100,000 pixels, cannot fit in GPU memory. Diffusion on them has to run tile by tile, with some mechanism that keeps the tiles globally consistent. Current mechanisms apply that consistency pressure at every step, in the same way for every tile. We propose to keep global coupling only until each tile's coarse layout has "committed," meaning its predicted low-frequency content has stopped changing, and then release that tile to generate fine detail on its own.
The point is that a gigapixel image is not spatially uniform. A tile of homogeneous stroma probably settles its layout almost immediately. A tile straddling a tumor boundary may keep shifting for much longer. A single fixed cutoff, which is what the video literature uses, would then either over-constrain the first kind of tile or under-constrain the second. Scheduling each tile by its own measured commit time should give three things at once: seamless global structure, unconstrained cellular detail, and less work spent on the expensive global coupling.
Core Claim
In patch-based diffusion over ultra-high-resolution and gigapixel images, the step at which each tile's coarse layout stabilizes varies across the image and can be measured during sampling at negligible cost. Releasing global guidance per tile at that step should dominate two alternatives on the trade-off between consistency and fidelity: uniform guidance at any strength, and a single global cutoff at any fraction of the trajectory. Adaptive release should also reduce the total cost of the coupling.
Concretely, let $\hat x_0^{(i)}(t)$ be the model's predicted clean image for tile $i$ at step $t$, and let $L$ be a fixed low-pass filter. We track the relative change of that tile's coarse content between consecutive steps:
$$ \delta_i(t) = \frac{\lVert L\hat x_0^{(i)}(t) - L\hat x_0^{(i)}(t+1)\rVert}{\lVert L\hat x_0^{(i)}(t)\rVert}. $$
Tile $i$ is released once $\delta_i$ stays below a threshold $\epsilon$ for $k$ consecutive steps. After release, the tile keeps only its local conditioning and receives no further global guidance.
Relation to Prior Work
The motivation comes from video diffusion, where several groups independently locate structure in the early window. Motion and appearance separate across the denoising timeline, and constraining motion at all steps hurts quality, as shown by Liu et al.. Motion features are most effective as guidance early, according to Xiao et al.. Restricting camera conditioning to early steps improves video quality in Skorokhodov et al. and AC3D. Early-only correspondence guidance prevents texture bleeding in Gu et al.. Attention edits confined to the first 20–40% of steps fix structural glitches and sequence multiple events in Bu et al. and TS-Attn.
All of these methods choose one cutoff by hand for the whole frame, and all were evaluated at modest resolution. We take their principle and replace the hand-picked global cutoff with a measured, per-region one. That per-region adaptivity has no role at video-frame scale but plausibly matters across a slide.
The method we reuse and compete against is FreeAdapt. For 4K/8K restoration, it synchronizes tiles with a downsampled global reference in two ways: it replaces each tile's Fourier phase with the reference's phase (phase encodes layout), and it injects global context into self-attention. It applies this guidance statically across the trajectory. We keep its two guidance operators unchanged and change only when, and for which tiles, they act.
The other key constraint comes from Diffusion-Based Generation of Histopathological Whole Slide Images at a Gigapixel Scale. That paper generates 65,536² slides with cascaded resolution-conditioned stages, uses grid-shifted tiling to remove seams, and reports that a rigid downsampling constraint suppresses new cellular morphology unless relaxed by a parameter $r$. This result is exactly the tension we target: tight global constraints fight fine detail. The paper resolves it by loosening the constraint uniformly, whereas we resolve it by switching the constraint off per tile at the right time. The two knobs are not exclusive, and we test them against each other.
Two further works are adjacent but do not do what we propose. WSI-Diffusion generates slide embeddings hierarchically, with a slide-level blueprint model followed by a separate patch-level model. MExD runs diffusion only over the one-dimensional label vector. Both keep the coarse-to-fine hierarchy in separate modules and never schedule spatial guidance within a single denoising trajectory.
Research Program
The first stage is diagnostic and can falsify the whole idea. We run a pretrained patch diffusion sampler on whole-slide tiles and on UHD restoration inputs, and record $\delta_i(t)$ for every tile. From these traces we estimate the distribution of commit times.
Two outcomes would each kill or reshape the proposal. The video-derived premise may not transfer to histology: if coarse content in pathology tiles does not stabilize before late steps, early-only scheduling has no foundation. Alternatively, commit times may be nearly identical across tiles, with the spread small relative to the trajectory length. In that case per-tile adaptivity collapses to a single fixed cutoff, and the honest contribution is simply "transfer the video cutoff." We would report it as such. We also check whether commit time correlates with tissue content, such as tumor-boundary versus background tiles, using the tissue annotations available in Camelyon16.
The second stage builds the scheduler as a training-free wrapper. Unreleased tiles receive FreeAdapt's phase guidance and global-attention guidance. They also exchange boundary attention with unreleased neighbors, the cross-tile message passing that grid-shift approximates by moving coordinates. Released tiles drop all three. Because coupling costs scale with the number of unreleased tiles, we report wall-clock time and peak memory alongside quality.
The threshold $\epsilon$ and the window $k$ are tuned once on held-out images. They are then frozen for all benchmarks, so that we are not tuning a cutoff by hand under another name.
Baselines
The naive reference point is independent tiled sampling with no global coupling. It sets the floor for seams, color drift and hallucination.
The strongest existing method on restoration is FreeAdapt with its published static guidance, run with the same backbones it reports (standard LDM, StableSR, DiffBIR). For this method the file contains only relative figures, not absolute scores. FreeAdapt reports PSNR gains "typically exceeding 2 dB" over its backbones and a DISTS improvement of roughly 26.3% on dehazing (FreeAdapt). We will therefore rerun it ourselves on UHD-LL, UHD-Haze, UHD-Blur and 4K-Rain13k, using the same sampler, step count and GPU as our method.
On gigapixel histology, the strongest competitor is the cascaded sampler with grid-shift and relaxed constraint $r$ from the gigapixel WSI generation paper. Its only reported quantity in the file is cost, about 40 minutes per 65,536² slide on one GPU. We found no reported quality number on Lacuna for it on TCGA-BRCA, so we will reproduce it.
Fairness means identical denoiser weights, step counts and hardware. Every baseline knob also gets the same tuning budget on the same held-out split: FreeAdapt's guidance strength, the relaxation $r$, and any cutoff fraction.
Two ablations isolate our key ingredient. The first is a single global cutoff, swept over the 20–40% range the video literature uses and then tuned to its best value. The second is a randomized per-tile schedule whose release times are drawn from our measured commit-time distribution but assigned to the wrong tiles. It matches our compute exactly and tests whether measuring each tile matters, or whether merely spreading release times does.
Beating the baselines means lying strictly above the trade-off curve that the uniform-strength and global-cutoff sweeps trace out. On that curve, one axis is consistency, measured as a seam error on overlapping tile borders and low-frequency agreement with the global reference. The other axis is fidelity or detail, measured as PSNR/DISTS on restoration and as pathology-feature Fréchet distance plus high-frequency power spectra on histology. We also require no loss in wall-clock time.
Why not just…?
Why not just use a fixed early cutoff, say 30%, as the video papers do? This is the most serious threat, and the first-stage diagnostic exists to settle it. If commit times are tightly clustered, a well-tuned fixed cutoff will match us and we will say so. Per-tile scheduling earns its keep only if the tuned fixed-cutoff ablation leaves boundary tiles with seams, or homogeneous tiles with suppressed detail, and our method fixes both at once.
Why not just turn down FreeAdapt's guidance strength? A weaker constant guidance also trades consistency for detail. The video evidence (Skorokhodov et al., Gu et al.) suggests the damage comes from guidance being present at late steps, not from how strong it is. Weakening guidance everywhere also weakens it in the early window where layout is decided. The strength sweep in our baselines tests this directly. If some constant strength reaches our point on the trade-off curve, timing is not the operative variable.
Why not just rely on grid-shift and the relaxation $r$? These already remove seams and restore morphology in the gigapixel generation paper. However, $r$ loosens the constraint equally for every tile and every step, and the resulting slide still takes about 40 minutes. If tuned grid-shift plus $r$ matches our quality, our remaining claim is cost reduction, which we measure directly. We also test the combination, since per-tile release can sit on top of grid-shift.
Proposed Experiments
On the four UHD restoration benchmarks, we report PSNR and DISTS for each backbone under naive tiling, static FreeAdapt, best global cutoff, randomized release, and per-tile release, together with seam error and runtime.
On TCGA-BRCA whole slides, we compare against the grid-shift cascade in two settings: generation, and super-resolution from 150 µm/px guides down to 0.3 µm/px. The file warns that ImageNet-based FID misjudges histology, so we measure Fréchet distance in a pathology feature space instead. We add seam error and the agreement of tissue-type proportions with the coarse guide.
As a downstream check of biological plausibility, an attention-MIL classifier trained on real Camelyon16 slides is applied to synthetic slides conditioned on tumor-bearing guides. The test asks whether focal tumor regions survive as detectable signal or get smoothed away by over-long global guidance.
The decisive result is the pair of comparisons against the global-cutoff and randomized-release ablations at matched compute. If per-tile measured release does not beat both, the idea reduces to known practice.
References
- Separate Motion from Appearance: Customizing Motion via Customizing Text-to-Video Diffusion Models
- Video Diffusion Models are Training-free Motion Interpreter and Controller
- AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers
- VideoSwap: Customized Video Subject Swapping with Interactive Semantic Point Correspondence
- BroadWay: Boost Your Text-to-Video Generation Model in a Training-free Way
- TS-Attn: Temporal-wise Separable Attention for Multi-Event Video Generation
- FreeAdapt: Unleashing Diffusion Priors for Ultra-High-Definition Image Restoration
- Diffusion-Based Generation of Histopathological Whole Slide Images at a Gigapixel Scale
- From Representation Space to Prognostic Insights: Whole Slide Image Generation with Hierarchical Diffusion Model for Survival Prediction
- MExD: An Expert-Infused Diffusion Model for Whole-Slide Image Classification