Resolution Agnostic Generative Modeling

50 concepts46 papers1 proposal

Most deep generative models treat images as discrete matrices of pixels. When training a model on a dataset like ImageNet or CelebA, the architecture learns spatial priors that are tightly coupled to a specific training resolution (e.g., $512 \times 512$). This coupling happens because convolutional kernels, self-attention matrices, and positional encodings are inherently dependent on the spatial dimensions of the input tensor.

When we attempt to sample from these models at resolutions or aspect ratios outside their training distribution, this spatial coupling breaks down. For instance, forcing a model trained on square images to generate a wide panorama or a 4K canvas often results in structural failures, such as the "Frankenstein effect" where objects are unnaturally duplicated or cropped (Lee et al.; Shi et al.). This fixed-resolution constraint limits the utility of generative models across real-world tasks, from image super-resolution (Wang et al.) and inpainting (Liu et al.) to zero-shot image compression (Vonderfecht et al.) and unconstrained face manipulation (Yang et al.).

To overcome this limitation, two primary paradigms have emerged: shifting from discrete grids to continuous function spaces, and adapting pre-trained grid models at inference time via latent synchronization and attention scaling.

Shifting from Grids to Continuous Function Spaces

To achieve true resolution independence, one prominent line of research redefines how data is represented. Instead of generating a discrete grid of pixels, the model learns a distribution over continuous functions. In this paradigm, an image is treated as an Implicit Neural Representation (INR) or a neural field—a continuous mapping $f: \mathbb{R}^2 \to \mathbb{R}^3$ that takes a spatial coordinate $(x, y)$ and outputs a color value $(R, G, B)$.

Dupont et al. introduced this concept via Generative Adversarial Stochastic Processes (GASP), demonstrating that models could generate continuous signals queryable at any arbitrary coordinate. Similarly, Rahman et al. extended generative adversarial networks to infinite-dimensional function spaces using neural operators, allowing the modeling of continuous physical phenomena independent of grid discretization.

Lifting this concept to diffusion and flow-based generative models requires redefining the noising and generative pathways within infinite-dimensional Hilbert spaces. Corallo et al. formulated Functional Diffusion Processes (FDPs) to enable resolution-independent generation, while Park et al. and Pieper-Sethmacher et al. utilized Doob's $h$-transform and stochastic optimal control to guarantee that the generative process remains robust across varying sampling densities.

By operating in the continuous domain, models like $\infty$-Diff by Bond-Taylor et al. can synthesize high-fidelity details beyond the training resolution via sparse coordinate subsampling. Furthermore, Ranjan and Wang et al. demonstrated that training flow transformers directly in the ambient coordinate space bypasses the fixed-resolution bottlenecks imposed by traditional discrete latent encoders (such as VAEs), which Chen identified as a primary bottleneck in continuous generative pipelines.

Inference-Time Adaptation and Latent Synchronization

While continuous functional representations offer elegant mathematical foundations, retraining models from scratch is computationally expensive. An alternative paradigm focuses on adapting large, pre-trained grid-based models (like Stable Diffusion or FLUX) to arbitrary resolutions at inference time without further tuning. This approach relies on two primary insights: attention scaling and latent synchronization.

Attention Entropy Scaling

When a model trained on a fixed resolution is given a larger canvas, the number of image tokens increases quadratically. This expansion dilutes the attention entropy, causing the model to lose global context and repeat objects. To fix this, Jin et al. introduced a dynamic scaling factor that adjusts the attention matrix based on token count, preserving the original structural composition.

Latent Synchronization

To generate large panoramas or high-definition images, the canvas can be broken down into overlapping local patches. However, independently denoising these patches leads to boundary seams and semantic drift. Lee et al. solved this in SyncDiffusion by introducing a perceptual similarity loss that synchronizes overlapping local diffusion paths.

Quattrini et al. advanced this by merging overlapping latent views inside the self-attention layers rather than post-hoc, ensuring global semantic consistency. Lee et al. provided a formal probabilistic foundation for this technique via SyncSDE, framing diffusion synchronization as a system of coupled Stochastic Differential Equations.

By combining these mechanisms with frequency-domain structural guidance (ZHANG et al.; Shi et al.) or multi-aspect ratio training strategies (Zheng et al.; Gao et al.), modern generative frameworks can seamlessly scale to 4K resolutions and arbitrary aspect ratios while preserving both global coherence and fine-grained local textures.

Key Foundational Work

Solving Inverse Problems by Joint Posterior Maximization with Autoencoding Prior (2022) — contributed 2 concept elements to this direction.
INR-V: A Continuous Representation Space for Video-based Generative Tasks (2022) — contributed 1 concept elements to this direction.
Continuous-Time Functional Diffusion Processes (2023) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_fixed_resolution_constraint_fixed_resolu_d195809a,
  title = {fixed-resolution constraint (fixed-resolution training bias)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/fixed-resolution-constraint-fixed-resolution-training-bias/rdr_d195809a37de4eabb458460cb2e817ee},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_fixed_resolution_constraint_fixed_resolu_d195809a_v1,
  title = {fixed-resolution constraint (fixed-resolution training bias)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/fixed-resolution-constraint-fixed-resolution-training-bias/rdr_d195809a37de4eabb458460cb2e817ee/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake