Diffusion models are image generators that learn to turn random noise into pictures by removing noise step by step. They can only work on a fixed-size canvas, roughly 512 to 1024 pixels on a side, before GPU memory runs out. To make very large textures, current methods slide a window across a big canvas, denoise each window independently, and average the predictions where windows overlap. This keeps memory bounded, but no window ever sees anything outside itself. On textures with long-range structure, such as brick walls, woven fabric, or tiled floors, the result drifts: lattice rows bend, grain loses its direction, and color and lighting wander slowly across the canvas. The memory limit is the direct cause, because global coherence would require attention over the whole canvas, and that cost grows with canvas size.
We propose to keep memory bounded without giving up long-range coherence. Near-regular textures have a repeating lattice, so a distant region that should match a given window is fully determined by that window's position within one period of the lattice, which we call its lattice phase. We maintain a small attention memory on the GPU, indexed by lattice phase. Every window writes its attention keys and values into the slot for its phase and reads back what other windows, possibly thousands of pixels away, have written for the same phase. The memory's size depends on the lattice period and not on the canvas size. Two sentences summarize the idea: windows in a tiled diffusion run talk to each other through a fixed-size bank organized by where they sit in the repeating pattern. Long-range structural agreement then costs constant GPU memory instead of memory that scales with canvas area.
Core Claim
For near-regular and slowly deformed lattice textures, a canvas-written, phase-indexed key-value bank, shared across all windows of a training-free tiled diffusion sampler, will cut long-range geometric regularity error and low-frequency drift relative to independent window averaging. Its peak GPU memory will stay flat as the canvas grows from about 4 to over 100 megapixels. The new ingredient is neither tiling nor reference attention, since both exist. It is that the bank is written by the canvas during sampling, not copied from the exemplar, and that it is addressed by lattice phase, which turns a memory-capacity problem into a coherence mechanism.
Relation to Prior Work
Three papers supply the machinery for large canvases, and the proposal sits on top of them. MultiDiffusion shows that averaging overlapping window predictions is the exact solution of a quadratic reconciliation objective, and it reports a panorama FID of 10.3. Infinite Texture applies random-window score aggregation to textures above 85 megapixels after roughly 10 minutes of per-texture fine-tuning on an A100. Its own analysis motivates this work: it reports that directional structure loses orientation and that lighting and color drift across wide canvases. Tiled Diffusion enforces seam continuity between neighboring latent tiles (Tiling Score 0.03 versus 0.29). It also documents a context-window trade-off: large windows give coherence but less variety, and small windows give variety but break structure. All three couple windows only through their overlaps. Our bank couples windows that never overlap.
Two papers supply the structural signal. NPP-Net warps coordinates modulo detected periods and proposes Top-K candidate periods for robustness, reaching LPIPS 0.188 on the PSU Near-Regular Texture Database. It also constrains us: it needs the period estimate within about 10% of the truth and assumes planar geometry. Quantitative Evaluation of Near Regular Texture Synthesis Algorithms shows that lattice awareness matters. Lattice-based synthesis preserves regularity in 95% of cases, versus 55% for Graph Cut and 23% for patch copying, and the paper defines the geometric and appearance regularity scores we reuse.
Generating Non-Stationary Textures Using Self-Rectification supplies the attention mechanism. It treats attention keys and values from a reference image as a texture sampler and injects them into generation. It is bounded at 512×512 and takes about 3 minutes per image. We reuse its injection operation but change what is injected and how it is addressed.
The systems literature sets our posture rather than our method. Ren et al.'s out-of-core Cholesky shows that when the dependency structure is known in advance, statically scheduled transfers beat demand paging. That work reports a 20% gain over cuSOLVER. Our sampler likewise knows its window order and phase lookups in advance. We must be honest, though: at 8× latent downsampling, even a gigapixel canvas's latents occupy only hundreds of megabytes. Moving latents in and out of the GPU is therefore engineering, not the contribution. The binding constraint is attention context, not latent storage.
Research Program
The sampler first estimates a lattice from the exemplar. It uses NPP-Net-style Top-K period proposals, plus an optional smooth deformation field in the spirit of the deformed-lattice tiles of Hays et al. Each latent position $x$ receives a phase
$$\phi(x) = \left( \frac{u\cdot w(x)}{p_u} \bmod 1,\ \frac{v\cdot w(x)}{p_v} \bmod 1 \right).$$
Here $w(x)$ is the deformation-corrected coordinate, $u$ and $v$ are the lattice direction vectors, and $p_u$ and $p_v$ are the two periods. We quantize phase space into $B$ bins. Each bin holds a running, exponentially averaged key-value pair for each chosen self-attention layer of a pretrained Stable Diffusion U-Net, refreshed at every denoising step. When a window is denoised, each query token attends jointly over its own window tokens and over the bank entries for its own phase bin and the neighboring bins. Bank reads use soft bilinear weights across bins, so that a small period error degrades smoothly instead of snapping. After the window is denoised, its keys and values are written back to the bank. The bank is seeded from the exemplar's own phase-binned keys and values. This keeps the method training-free and removes Infinite Texture's per-texture fine-tuning.
Bank memory is about $B \cdot L \cdot d$, where $L$ is the number of attention layers used and $d$ is the feature width. This is independent of canvas size, which is the whole memory argument.
Drift has a low-frequency cause that phase alone cannot fix. We therefore also keep one global channel statistic per phase bin and regress window latent means toward it. This is a cheap stand-in for the low-frequency anchor that Infinite Texture lacks.
Two research questions accompany the build. The first is how sensitive the bank is to period error: we will sweep injected error from 0 to 20% and see whether online re-estimation from canvas-written entries extends NPP-Net's roughly 10% tolerance. The second is how far the method reaches beyond strict lattices. We will test whether the approach degrades gracefully, or harmfully, on non-stationary textures without a lattice, such as veins and rings. On those inputs a phase bank could impose false repetition. We report this as a stated limitation, not a hidden one.
Baselines
The naive reference point is independent random-window score aggregation in the style of MultiDiffusion and Infinite Texture. It uses the same pretrained model, the same window size and overlap, the same number of denoising steps, and the same exemplar conditioning through self-rectification-style reference injection. Our method must reduce geometric regularity error and drift at matched wall-clock time and matched peak memory.
The strongest existing large-canvas texture method is Infinite Texture with its per-texture fine-tuning. It is the strongest competitor because it is the only method in the file that generates textures above 85 megapixels. We run it with its reported roughly 10-minute fine-tuning budget, and we give our training-free method the same total GPU time spent on more sampling steps if that helps. The file reports a 45% realism preference for Infinite Texture versus 22% for adversarial expansion in its human study. No reported result was found on Lacuna for Infinite Texture's geometric regularity or drift metrics, so we will measure them ourselves. Beating it means lower regularity error at equal or greater preference.
The strongest regularity baseline is lattice-based near-regular synthesis from Hays et al., which reports a 95% regularity success rate. It may well match us on regularity, so we must beat it on appearance variety (non-repetition) while staying within a few points of its success rate.
On the near-regular texture database, NPP-Net reports LPIPS 0.188 for extrapolation and inpainting. We compare at the resolutions where it fits in memory, and report where its dense coordinate queries exhaust GPU memory.
Two ablations isolate the key ingredient. The first is a static exemplar bank: the same phase-indexed bank filled once from the exemplar and never written by the canvas. The second is a phase-free bank: canvas-written entries addressed by nearest spatial position instead of by phase, at matched memory.
Why not just…?
Why not just use bigger windows or a coarse-to-fine pass? Generating a low-resolution global layout and then refining it tile by tile, as SinGAN's pyramid does in SinGAN, is the most serious threat for lighting and color drift. It may fully solve low-frequency drift. Its weakness is that lattice alignment is decided at coarse scale, and mortar lines a few pixels wide are not resolvable there. Tiled Diffusion documents the corresponding trade-off for window size. We run a coarse-to-fine baseline at matched memory and report drift and regularity separately. If it matches us on both, the phase bank is unnecessary, and we will say so.
Why not just inject the exemplar's attention into every window, as self-rectification does? This is the static-bank ablation, and it is a real competitor. A fixed exemplar bank gives every window the same appearance cues, but it cannot enforce agreement between windows on the variations they invent. Those variations include weathering, slow color gradients, and accumulated small lattice deformation. Our method claims that canvas-written entries trade some independence for this agreement. If the static bank scores within noise of ours on regularity and drift, the core claim fails.
Why not just tile the lattice classically? Lattice tiling from Hays et al. is cheap in memory and nearly perfect in regularity. Its failure mode is visible repetition, which its appearance score and human preference capture. We report repetition directly by measuring how often distant output patches nearly match one another.
Why not just rely on unified virtual memory paging? Our working set is small and its access pattern is predictable, so paging is not the bottleneck. The systems evidence in HyTGraph against demand paging matters only for the image decoder at gigapixel scale. We handle that with a statically scheduled, double-buffered tile decode and report its throughput against managed memory as a secondary result.
Proposed Experiments
The main evaluation uses the PSU Near-Regular Texture Database and the near-regular set of Hays et al. We synthesize each exemplar at 4, 16, 64 and 128 megapixels. We report the geometric and appearance regularity scores and the regularity success rate, LPIPS against held-out crops, an orientation-coherence score for directional textures, and low-frequency drift. Drift is measured as the variance of heavily blurred canvas color, relative to the exemplar. We also report the distant-patch repetition rate, peak GPU memory, and time per megapixel. Directional and non-lattice exemplars from the Describable Textures Dataset (the striped, veined and grid classes) serve as the stress set, where we expect the method to help less or to hurt. A paired human preference study follows Infinite Texture's protocol.
Three checks decide the outcome. First, peak memory must stay flat from 4 to 128 megapixels while geometric regularity error stays flat. The independent-window baseline's regularity error should instead grow with canvas size; if it does not grow, the problem we target does not exist. Second, the canvas-written phase bank must beat the static exemplar bank on regularity and drift. Third, under 10–20% injected period error, soft phase lookup with online re-estimation must degrade more gently than NPP-Net's reported tolerance. If the coarse-to-fine baseline wins on drift but loses on lattice alignment, we will combine the two and narrow the claim to alignment.
References
- MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation
- Infinite Texture: Text-guided High Resolution Diffusion Texture Synthesis
- Tiled Diffusion
- Learning Continuous Implicit Representation for Near-Periodic Patterns
- Quantitative Evaluation of Near Regular Texture Synthesis Algorithms
- Generating Non-Stationary Textures using Self-Rectification
- Accelerating Mixed-Precision Out-of-Core Cholesky Factorization with Static Task Scheduling
- SinGAN: Learning a Generative Model From a Single Natural Image
- HyTGraph: GPU-Accelerated Graph Processing with Hybrid Transfer Management