Exploiting Sub-Pixel Aliasing for Multi-Frame Super-Resolution

61 concepts49 papers

Digital imaging systems are bound by a fundamental physical constraint: the spatial sampling rate of the sensor. When a scene contains high-frequency spatial details—such as fine textures, sharp edges, or distant text—that vary faster than the distance between adjacent pixels (the pixel pitch), the sensor cannot resolve them accurately. According to the Nyquist-Shannon sampling theorem, these frequencies exceed the sensor's native capacity and fold back into the lower-frequency spectrum. This phenomenon is known as aliasing, and it manifests visually as jagged edges, moiré patterns, or irreversible blur.

In single-image processing, aliasing is treated as destructive distortion. However, in multi-frame computational imaging, aliasing is treated as a rich, deterministic source of hidden information. By capturing multiple low-resolution views of a scene with slight fractional offsets—sub-pixel displacements—imaging systems can untangle these folded frequencies. This article explains the intuition behind why sub-pixel aliasing can be computationally inverted to reconstruct high-resolution images that break the single-frame Nyquist barrier.


The Intuition: Aliasing as an Encoded Signal

To understand why multi-frame super-resolution works, we must first shift our perspective on aliasing. Imagine a high-frequency sinusoidal grating pattern projected onto a discrete pixel grid. If the grating's peaks and troughs occur faster than the pixel spacing, an individual pixel can only record the average intensity over its surface area. A single static capture permanently loses the true frequency, mapping it instead to an incorrect, lower-frequency pattern.

However, if we shift the camera slightly to the right by exactly half a pixel, the sensor elements sample a completely different phase of that same high-frequency grating. The resulting aliased pattern in this second image will look distinctly different from the first.

As Bruno Lecouat et al. demonstrate in Lucas-Kanade Reloaded: End-to-End Super-Resolution from Raw Image Bursts, these sub-pixel shifts cause high-frequency details to alias into the low-resolution grid in predictable ways. Because the aliasing is deterministic and depends strictly on the spatial phase of the sensor, a sequence of shifted frames provides a set of linearly independent equations describing the underlying continuous scene. If we can precisely determine the sub-pixel offsets between frames, we can solve this system of equations to recover the true high-frequency textures that are completely invisible in any individual source image.


Mechanisms for Generating Sub-Pixel Displacement

For multi-frame fusion to succeed, the captured frames must contain non-redundant information. If the camera is perfectly stationary and the scene is static, every frame will feature identical aliasing, yielding no new data. Non-redundant sampling requires sub-pixel displacement, which can be introduced through several distinct physical mechanisms:

  1. Natural Jitter: In handheld photography, natural hand tremors introduce random, sub-pixel camera motion between frames in a burst. Goutam Bhat leverages this in Deep Burst Super-Resolution, showing that uncontrolled human movement provides the exact spatial diversity needed to reconstruct fine architectural details and legible text.
  2. Controlled Rotational Sub-Pixel Displacement: Instead of lateral translation, sub-pixel shifts can be generated by rotating a sensor around its optical axis relative to a fixed rectangular pixel grid. Hajime Nagahara et al. demonstrate this mechanism in Resolution Improving Method from a Sub-pixel Displaced Omnidirectional Image Sequence, using sensor rotation to achieve uniform multi-frame super-resolution across a 360-degree field of view without complex mechanical shifting stages.
  3. Orbital and Push-Frame Motion: Satellite imaging systems naturally experience sub-pixel offsets as they sweep across the Earth's surface. Ngoc Long Nguyen et al. exploit this in Self-Supervised Multi-Image Super-Resolution for Push-Frame Satellite Images, using the inherent satellite trajectory to collect aliased frames and synthesize high-resolution imagery without requiring ground-truth training data.

Regardless of how the displacement is generated, the foundational requirement remains what Mohan Shankar et al. describe in Thin infrared imaging systems through multichannel sampling: capturing multiple subaperture views, each with a distinct sampling phase, to ensure the captured dataset contains non-redundant high-frequency information.


The Computational Pipeline: Registration and Fusion

Reconstructing a high-resolution image from an aliased sequence requires two primary computational steps: sub-pixel registration and high-resolution grid fusion.

Low-Res Frames       Sub-Pixel Registration          Dense Grid Fusion          Super-Res Output
  [ Frame 1 ]  --->   (Frequency Domain /   )  --->  [ .

* .
* .
* ]  --->   [ Sharp, Unaliased ]
  [ Frame 2 ]  --->   (Low-Frequency Phase  )        [ * .

* .
* . ]         [ High-Frequency   ]
  [ Frame 3 ]  --->   (Alignment            )        [ .

* .
* .
* ]         [ Detail Recovered ]

Sub-Pixel Registration

If the computational system miscalculates the alignment of the input frames by even a fraction of a pixel, the fusion process will introduce destructive artifacts and blur rather than recovering detail. However, registering images that suffer from heavy aliasing is a chicken-and-egg problem: the aliasing distorts the very spatial features (like edges and corners) normally used for alignment.

To bypass this obstacle, Patrick Vandewalle et al. introduce a frequency-domain framework in A Frequency Domain Approach to Registration of Aliased Images with Application to Super-resolution. Their method relies on a key mathematical insight: while the high-frequency spectrum is heavily corrupted by aliasing, the low-frequency portion of the Fourier spectrum remains entirely alias-free. By isolating this low-frequency band, the authors compute highly accurate sub-pixel horizontal, vertical, and rotational shifts, ensuring robust alignment despite severe spatial distortion.

Grid Fusion and Deconvolution

Once the exact sub-pixel trajectories are known, the pixels from all low-resolution frames are mapped onto a single, dense virtual grid. A classic approach to this is the "shift-and-add" algorithm, which interleaves the sparse samples from different times or channels to fill in the missing spatial coordinates, as detailed by Antony C. S. Chan in All-passive pixel super-resolution of time-stretch imaging.

Because physical lenses introduce optical blur (the Point Spread Function, or PSF), this grid mapping is typically paired with an inverse convolution or deconvolution framework. To prevent amplification of noise during deconvolution, the system response must be characterized on a grid finer than the physical sensor pitch. Mauricio Delbracio et al. show in Non-parametric Sub-pixel Local Point Spread Function Estimation that calculating the PSF at a sub-pixel resolution is necessary to accurately model light diffraction and lens aberrations that occur below the sensor's native sampling rate.

Modern implementations often unroll these optimization steps into deep neural networks. For example, Mazen Mel demonstrates in HoloADMM: High-Quality Holographic Complex Field Recovery that integrating physics-based optimization with convolutional layers allows networks to explicitly exploit aliasing patterns across sub-pixel displaced captures, recovering sub-micron features that are completely invisible in single-frame baselines.


Modality Independence: A Universal Signal Principle

The capacity to invert aliasing through sub-pixel exploitation is not a quirk of optical camera sensors; it is a universal principle of digital signal processing. Consequently, this framework is widely applied across diverse imaging modalities:

By treating aliasing as an encoded signal rather than destructive noise, multi-frame super-resolution shifts the burden of resolution enhancement from expensive hardware manufacturing to elegant mathematical inversion.

Key Foundational Work

Reconstruction of high resolution 3D visual information (1994) — contributed 2 concept elements to this direction.
Resolution improving method from multi-focal omnidirectional images (2001) — contributed 2 concept elements to this direction.
Resolution Improving Method from a Sub-pixel Displaced Omnidirectional Image Sequence (2001) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_aliasing_based_resolution_enhancement_7fd8c920,
  title = {aliasing-based resolution enhancement},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/aliasing-based-resolution-enhancement/rdr_7fd8c920ee6f4c3cb46daafe50fa5e65},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_aliasing_based_resolution_enhancement_7fd8c920_v1,
  title = {aliasing-based resolution enhancement},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/aliasing-based-resolution-enhancement/rdr_7fd8c920ee6f4c3cb46daafe50fa5e65/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake