Standard generative models—whether they are Generative Adversarial Networks (GANs), Diffusion Models, or Flow Matching frameworks—typically treat data as discrete, fixed-dimensional tensors. An image is represented as an $H \times W \times C$ grid of pixels; a 3D shape is a fixed voxel grid.
This discrete framing introduces a fundamental limitation: the model's architecture is tightly coupled to a specific resolution. If you train a model on $64 \times 64$ images, scaling up to $512 \times 512$ requires either retraining a completely new model with quadratically larger memory requirements, or relying on heuristic interpolation methods that inevitably introduce blur.
A rapidly growing paradigm shifts generative modeling from discrete grids to infinite-dimensional function spaces. Instead of generating an array of pixel values, these models learn to generate a continuous function $f: \mathcal{X} \to \mathcal{Y}$ that maps a coordinate (such as a spatial position $x, y$) to its corresponding value (such as an RGB color). Because the generative process happens in function space, resolution becomes an inference-time choice rather than a training-time constraint, enabling true zero-shot super-resolution.
The Core Intuition: Coordinate-In, Value-Out
To understand why this approach works, we must first look at how data is represented. Instead of storing an image as a matrix, we can represent it as an Implicit Neural Representation (INR) or a Neural Field.
Early foundations in this area, such as Attentive Neural Processes introduced by Hyunjik Kim et al. in Attentive Neural Processes, demonstrated that treating structured data as continuous coordinate mappings allows models to generalize effectively to unseen, higher-resolution grids. Rather than learning fixed pixel dependencies, the model learns the underlying functional relationship across the continuous domain. Similarly, the framework proposed in Variational Mixture of HyperGenerators for Learning Distributions over Functions combines the flexibility of INRs with the efficient inference of Variational Autoencoders (VAEs) to map latent variables directly to continuous functional distributions.
Discrete Grid Model:
Latent Vector z ───> [ Convolutional Layers ] ───> Fixed Pixel Grid (e.g., 64x64)
Function Space Model:
Latent Vector z ───┐
├───> [ Function Space Model ] ───> Continuous Function f(x,y) ───> Query any arbitrary grid density
Coordinates (x, y) ─┘
When a generative model learns a distribution over these continuous functions, super-resolution changes from an ill-posed upscaling problem into a simple matter of coordinate querying. To generate a higher-resolution output, you do not alter the model; you simply evaluate the learned continuous function on a finer, denser grid of coordinates.
Lifting Generative Frameworks to Hilbert Spaces
To generate novel, high-fidelity functions, standard generative paradigms must be mathematically reformulated to operate within infinite-dimensional Hilbert spaces.
Generative Adversarial Stochastic Processes
To bypass the "curse of discretization" in traditional GANs, Emilien Dupont et al. proposed the Generative Adversarial Stochastic Process (GASP) in Generative Models as Distributions of Functions. GASP defines the generator and discriminator over continuous domains, treating data points as sample paths of a stochastic process. This allows the model to generate structurally coherent data on arbitrary geometries, such as climate data on a sphere, at resolutions far beyond those seen during training.
Functional Diffusion
Diffusion models can also be lifted into function space. Biao Zhang et al. introduced Functional Diffusion, shifting the diffusion process from discrete arrays to infinite-dimensional functional spaces. This concept was expanded by Giulio Corallo et al. in Continuous-Time Functional Diffusion Processes, which decouples the generative trajectory entirely from fixed pixel coordinates.
To provide a rigorous mathematical foundation for this behavior, Byoungwoo Park et al. connected stochastic optimal control theory with Doob’s $h$-transform in Hilbert spaces (Stochastic Optimal Control for Diffusion Bridges in Function Spaces), proving that resolution-free generation can be achieved via mathematically sound diffusion bridges over continuous spaces.
Functional Flow Matching
Flow matching offers an alternative to diffusion that can similarly be adapted for continuous representations. Yuyang Wang developed INRFlow: Flow Matching for INRs in Ambient Space, a domain-agnostic framework that applies flow matching directly to the coordinate space. By learning a continuous probability density field rather than operating in a compressed latent space, it handles images, 3D point clouds, and proteins under a single training recipe.
Concurrently, Joshua M. Susskind et al. demonstrated in Coordinate In and Value Out: Training Flow Transformers in Ambient Space that this coordinate-based flow formulation eliminates the need for domain-specific compressors like VAEs. To ensure smooth transitions across scales, Jianxin Zhang et al. proposed Flow Straight and Fast in Hilbert Space: Functional Rectified Flow, leveraging the superposition principle for continuity equations to guarantee contour coherence when scaling up outputs.
Why It Works: Discretization Invariance
The core mechanism that enables zero-shot super-resolution without architectural failure is discretization invariance.
Standard convolutional layers rely on fixed stride and kernel sizes relative to a pixel grid. If you pass a high-resolution grid into a standard CNN trained on low-resolution data, the receptive fields shift, destroying global structural coherence.
To solve this, function-space models utilize Neural Operators. Neural operators are designed to map between infinite-dimensional function spaces using operations (like integration or continuous attention) that remain invariant to how the underlying domain is discretized.
As shown by Yaozhong Shi et al. in Mesh-Informed Neural Operator : A Transformer Generative Approach, an operator-based architecture treats data as continuous functions independent of any underlying mesh or sensor network geometry. This property allows the model to process irregular domains seamlessly.
Traditional Convolution:
[Pixel Input] ──> [Discrete Kernel Matrix Multiply] ──> Dependent on Grid Size
Neural Operator:
[Function Input] ──> [Continuous Integral / Attention Kernel] ──> Independent of Grid Size
This architectural independence is vividly leveraged by Alexandros Graikos et al. in ∞-Brush: Controllable Large Image Synthesis with Diffusion Models in Infinite Dimensions. By introducing a cross-attention neural operator, their model decouples internal generation from the target pixel grid. As a result, it can synthesize images up to $4096 \times 4096$ pixels while maintaining a completely constant memory footprint and parameter count during its core generative steps.
Taming the Infinite: Practical Training Mechanics
Training a model to operate on an infinite-dimensional space presents an obvious paradox: how do you compute a loss function over an infinite number of points without running out of memory?
The solution lies in sparse coordinate subsampling combined with continuous noise processes. In ∞-Diff: Infinite Resolution Diffusion with Subsampled Mollified States (arxiv 2023), Sam Bond-Taylor demonstrated that function-space diffusion can be trained efficiently by optimizing over a small, randomly sampled subset of coordinates at each training step. To prevent the high-frequency noise from collapsing the functional representations, the framework introduces "mollified" (smoothed) noise states.
In a subsequent work, Sam Bond-Taylor et al. clarified an important intuition regarding what happens during zero-shot upscaling in ∞-Diff: Infinite Resolution Diffusion with Subsampled Mollified States (iclr 2024):
While continuous-function models do not magically hallucinate entirely new high-frequency features outside of their training distribution, they maintain significantly higher detail, sharpness, and contour coherence compared to traditional interpolation methods because they preserve the underlying functional boundaries.
Beyond Images: Physical Sciences and Spectral Bias
Because physical phenomena naturally exist as continuous fields, this paradigm is highly effective for scientific surrogate modeling. Jan Hagnberger et al. developed Vectorized Conditional Neural Fields (Vectorized Conditional Neural Fields: A Framework for Solving Time-dependent Parametric Partial Differential Equations), combining continuous coordinate inputs with Transformers to achieve zero-shot super-resolution across complex 1D, 2D, and 3D fluid dynamics and PDE simulations.
One persistent challenge when scaling to extreme resolutions is spectral bias—the tendency of deep networks to learn low-frequency, smooth functions before high-frequency details, which can lead to slight blurriness at extreme upscaling factors. To combat this, Xihaier Luo et al. designed the Hierarchical Neural Operator Transformer (Hierarchical Neural Operator Transformer with Learnable Frequency-aware Loss Prior for Arbitrary-scale Super-resolution). By incorporating a learnable, frequency-aware loss prior, their framework forces the model to capture fine-grained high-frequency details, enabling structural integrity and sharpness even at extreme, arbitrary extrapolation scales.
Key Foundational Work
Generative Models as Distributions of Functions (2022) — contributed 1 concept elements to this direction.
Attentive Neural Processes (2019) — contributed 1 concept elements to this direction.