Scene-Locked Tiling for Flicker-Free Gigapixel Video Processing

A model that restores, recolors, or restains a gigapixel image cannot see the whole image at once. It has to cut the image into thousands of tiles, process each tile separately, and stitch the results back together. The best current methods hide the stitching seams with randomness: random tile offsets, noise drawn fresh for each tile, and blend regions that adapt to the content. On a single image this works well. On a video, or on a stack of neighboring tissue slices, the same randomness is redrawn at every frame. Seams that were invisible in any one frame then crawl, and textures boil from frame to frame. We argue that much of the flicker in tiled gigapixel pipelines is self-inflicted by the tiling procedure rather than caused by the underlying model. It can be removed almost for free by attaching every random choice to a location in the scene instead of to a frame index.

The proposal combines two cheap ingredients. First, the tile grid, the blend weights, and the diffusion noise (the random starting point a diffusion model, a generator that turns noise into an image by repeated denoising, refines into its output) are defined once on a canonical scene canvas. They are carried into each frame by a motion estimate computed at low resolution. Second, the low-resolution guide image that UHD diffusion methods already use to keep tiles globally coherent is computed with a temporally stable video method, which is affordable at low resolution. That makes the guide a temporal anchor as well as a spatial one. The claim is that these two changes together bring tiled gigapixel diffusion close to the temporal stability of a global color lookup table, while keeping the spatially varying expressiveness a lookup table lacks. They should also let static regions reuse cached tiles.

What exists and what is missing

The gigapixel literature fixes seams inside one static image. FreeAdapt restores 4K/8K images with a pretrained latent diffusion model. It guides each tile with the Fourier phase and attention features of a downsampled reference image, and reports PSNR gains "typically exceeding 2 dB" over plain patch-based inference. It is explicitly single-frame and "unsuitable for real-time applications like live video processing." D-VST virtually restains billion-pixel slides. It increases tile overlap only in smooth, low-frequency regions and runs 8× faster than uniform high-overlap blending. Bao et al. remove CycleGAN tiling artifacts by averaging several passes with random window shifts and center-weighted masks. The code is at RandomWalkSlidingWindow.

These papers supply our spatial machinery. They also motivate the proposal, because each one makes a per-image random or content-dependent choice about where seams go. That choice changes whenever the frame changes. This is the tension the research file calls Gap 2: random shifting turns static seams into moving ones, while a rigid grid leaves seams fixed and visible.

The flicker literature addresses stability but never at this scale. Ruder et al. traced flicker to non-convex optimization, where tiny input changes send the model to different solutions, and fixed it with flow-based warping losses. ReCoNet trains a feed-forward network with feature-level and luminance-aware temporal losses and runs at 640×360. FreeViS stabilizes diffusion video stylization with RAFT optical flow guidance and reference-isolating attention on short web clips. Its memory cost grows with pixel count or its square, so it cannot run on a 25k×14k frame.

D-LUT takes the opposite route. It produces a single global 3D color lookup table, so flicker is impossible by construction, at 90 FPS on Full HD. The price is that it cannot vary the transformation across tissue types or scene regions.

PANDA supplies real gigapixel video: up to 25k×14k at 30 Hz from a static array camera. It is a detection and tracking benchmark with no generative processing, so it constrains our evaluation regime but offers no method to compare against.

No paper in the file applies a tiled generative pipeline to gigapixel frame sequences or measures its flicker. The obvious combination would run FreeViS-style flow guidance tile by tile. That fails on memory and would shear flow fields at tile borders (Gap 1). Our design deliberately avoids full-resolution flow altogether.

Core Claim

Flicker in a tiled gigapixel pipeline has three separable sources. Grid flicker comes from seam positions and blend weights changing between frames. Sampling flicker comes from fresh noise for each tile at each frame. Content flicker is the model's intrinsic instability, as described by Ruder et al. We claim that the first two dominate in the tiled regime, and that scene-locking removes both.

Scene-locking means the following. Let $\phi_t$ map canonical scene coordinates to frame $t$. We estimate $\phi_t$ at low resolution: a global homography, plus a coarse flow field computed on a frame downsampled to roughly 1–2 megapixels. That flow is smooth, so it can be upsampled without tile-border shearing. The initial noise, the tile layout, and the blend masks are then defined once on the canvas and pulled back to each frame:

$$\epsilon_t(x) = \epsilon_{\mathrm{c}}\big(\phi_t^{-1}(x)\big), \qquad G_t = G_{\mathrm{c}}\circ\phi_t^{-1}.$$

Here $\epsilon_{\mathrm{c}}$ is a single noise field on the canvas, $G_{\mathrm{c}}$ is the canvas tile grid and blend-weight map (for example, D-VST's frequency-adaptive overlap computed once), and $x$ is a pixel in frame $t$. Where objects move independently of the background, the coarse flow carries their noise and seam assignments with them. Where the model cannot follow an object, such as pedestrians only a few pixels tall, the pipeline falls back to the anchor described next.

Content flicker is attacked through the guide. FreeAdapt's phase and feature guidance come from a low-resolution reference. We produce that reference as a temporally stabilized low-resolution video, using ReCoNet-style warping losses or FreeViS-style guidance, both of which are tractable at that scale. The tiles therefore inherit temporal consistency from a small video rather than enforcing it themselves.

Finally, where the low-resolution change map shows no change and $\phi_t$ is the identity, as in most of PANDA's static background, tiles are reused from cache instead of recomputed. This addresses the per-frame diffusion cost (Gap 4) without any new model.

Research Program

The first step is diagnostic. We build a measurement protocol that attributes flicker to its three sources. We run FreeAdapt and D-VST per frame under four conditions: all randomness free, grid fixed, noise fixed, and both fixed. Each run uses sequences whose true inter-frame motion is known exactly. The attribution uses the difference in warped temporal error between conditions.

We then add the scene-locked noise and grid, the stabilized guide, and caching one at a time. The base diffusion models stay frozen; everything is training-free, which keeps the comparison to FreeAdapt and D-VST clean.

A secondary analysis tests the frequency tension in Gap 5. Low-resolution phase guidance may erase small moving objects. We measure this directly by detecting PANDA pedestrians in processed frames and asking whether scene-locked noise alone, without phase guidance, preserves them better.

Baselines

The naive reference point is per-frame inference with each method's published single-image procedure. That means FreeAdapt with fresh noise per frame, and the random-walk sliding window of Bao et al. with fresh offsets per frame. This is what a practitioner would do today, and it defines how much flicker the tiling itself adds.

The strongest existing spatial method is FreeAdapt. Its reported advantage is a PSNR gain "typically exceeding 2 dB" over standard patch-based inference on the UHD restoration benchmarks. For restaining, the strongest existing method is D-VST, with a reported 94.17% cancer-classifier accuracy on virtually stained images against 95% on real IHC. No reported result was found on Lacuna for the temporal error of any tiled gigapixel method, so all temporal comparisons are new measurements. Every variant uses the same frozen weights, the same number of denoising steps, the same tile size, and the same GPU budget. Scene-locking adds only low-resolution motion estimation, and we report that cost separately. To beat these baselines, our method must reduce warped temporal error, especially in bands around seams, by a clear margin across seeds, while keeping per-frame PSNR and DISTS within noise of the per-frame method and keeping D-VST's classifier accuracy.

The strongest simple competitor is fixed grid plus fixed global seed. The same random seed and the same tile grid are used for every frame, noise is attached to screen pixels, and the frame-by-frame pipeline is otherwise unchanged. This baseline takes one line of code and genuinely removes grid and sampling flicker when the camera is static. We run it on everything. A second competitor is post-hoc deflickering: process frames independently, then temporally smooth the outputs using the upsampled low-resolution flow.

For color-only tasks, D-LUT serves as a stability ceiling. It has zero flicker by construction, so it shows how much stability our spatially varying pipeline gives up in exchange for local adaptation.

Ablations remove, one at a time, the scene-locked noise, the scene-locked grid, the temporally stabilized guide (replacing it with per-frame low-resolution references), and the cache. Together they isolate which ingredient carries the gain.

Why not just…?

Why not just fix the random seed and the tile grid across frames? This is the real threat. On a static camera it may match us on background flicker. Its failure should appear under camera motion, zoom, or registration drift between serial sections. There, screen-attached noise and seams stay fixed on the display while content slides beneath them, producing a "texture sticking" artifact. It also does nothing for content flicker. Our synthetic-motion sequences separate the two cases. If fixed-seed matches scene-locking even under motion, we will report that the contribution reduces to the stabilized guide and caching.

Why not just downsample, stabilize at low resolution, and super-resolve? This throws away the reason for capturing gigapixel data in the first place. A 25k×14k frame reduced to Full HD loses more than 160× in pixel area, and PANDA already reports small-object detection AP of only about 0.20 at full resolution. A tiled super-resolution step would also reintroduce the seams FreeAdapt was built to remove. We still test this as a baseline and score it with pedestrian detection AP on processed PANDA frames.

Why not just use a global lookup table like D-LUT? It is flicker-free and fast, but a single global color mapping cannot restain epithelium and stroma differently. D-VST's separate structure and tone encoders exist precisely to prevent that kind of pathology leakage. We include D-LUT on color-only tasks to measure the stability we sacrifice, and we expect it to lose on region-specific fidelity.

Why not just smooth the outputs afterwards? Smoothing trades flicker for ghosting and blur, the stability-versus-richness trade-off documented in the video stylization literature, and it cannot remove seams that move. The seam-band temporal error and DISTS measurements will show whether this trade-off holds here.

Proposed Experiments

We use three sequence types. The first is controlled pans, zooms, and small affine jitters over UHD-LL and UHD-Haze images and over RegH2I slides. Here the true warp is known exactly, which makes warped temporal error an exact measurement. The second is serial-motion simulations of slide stacks, created with small rotations and translations, to mimic registered neighboring sections. The third is real PANDA clips of 16–32 frames at 8k×8k crops and at full frame, with synthetic haze or low-light degradation for restoration.

The metrics are warped temporal error, computed globally and within seam bands; a seam-visibility gradient statistic; PSNR and DISTS per frame; the D-VST pathology classifier accuracy on restained slides; Faster R-CNN small-object AP on processed PANDA frames; and wall-clock time per frame, including the fraction of tiles served from cache.

The decisive checks follow from the design. First, under motion, scene-locking must beat fixed-seed/fixed-grid on seam-band flicker. Second, removing the stabilized guide must raise content flicker without affecting seams, which confirms that the three sources are separable. Third, per-frame fidelity must not fall below FreeAdapt's single-image numbers. Fourth, on static PANDA background, caching must cut compute without a measurable increase in flicker.