Sampling Signals with a Finite Rate of Innovation

35 concepts11 papers1 proposal

The classical Shannon-Nyquist sampling theorem is the bedrock of modern digital signal processing. It states that to perfectly reconstruct a signal, we must sample it at twice its highest frequency. But this framework encounters a fundamental problem when dealing with signals containing sharp transitions, such as streams of Dirac impulses, step functions, or piecewise polynomial splines. Because these signals possess infinite bandwidth, the Nyquist theorem dictates that they require an infinite sampling rate to be captured perfectly.

Intuitively, this feels wrong. A signal consisting of three sharp pulses over a given time interval is fundamentally sparse. It does not contain infinite information; it contains exactly six degrees of freedom—the locations and amplitudes of the three pulses.

To resolve this paradox, a paradigm shift emerged: Finite Rate of Innovation (FRI) sampling. Instead of characterizing a signal by its maximum frequency, the FRI framework characterizes a signal by its rate of innovation ($\rho$), which represents the number of degrees of freedom or parameters defining the signal per unit of time. As explored by Onativia et al. in Sparse sampling: theory, methods and an application in neuroscience, bandwidth is not the sole determinant of information content. By shifting our perspective from the frequency domain to a parametric one, we can sample and perfectly reconstruct non-bandlimited signals at their rate of innovation rather than their non-existent Nyquist rate (Michaeli et al. in Xampling at the Rate of Innovation).

How It Works: The Role of the Sampling Kernel

We cannot sample a continuous stream of impulses directly with an ideal analog-to-digital converter; the impulses would fall between samples and be missed entirely. Instead, the continuous signal must first pass through a physical filter—a sampling kernel—before being digitized. The kernel smooths or "blurs" the signal, spreading the information of the discrete events across multiple neighboring samples.

Continuous Signal (e.g., Spikes) ---> [ Sampling Kernel ] ---> Smoothed Signal ---> Discrete Samples

In traditional sampling, the kernel is a low-pass filter designed to bandlimit the signal. In FRI sampling, the kernel serves a different mathematical purpose: it must map the continuous parameters (like pulse locations) into a structured algebraic form.

The foundational mechanism relies on the kernel's ability to reproduce polynomials or exponentials, a property known as the Strang-Fix condition. As demonstrated by Dragotti in Exact sampling results for 1-D and 2-D signals with finite rate of innovation using Strang-Fix conditions and local reconstruction algorithms, if a compact-support kernel can reproduce exponentials, a linear combination of the discrete samples can yield the exact moments of the continuous signal.

Different families of kernels offer unique advantages:

From Samples to Parameters: The Annihilating Filter

Once the sampling kernel has spread the signal's innovation across the discrete samples, how do we extract the exact, off-the-grid continuous locations of the pulses?

The magic lies in a classical algebraic tool called the annihilating filter (also known as Prony's method). When a kernel successfully extracts the exponential moments of a signal, the resulting sequence of values satisfies a linear recurrence relation. We can construct a digital filter whose coefficients are chosen specifically to "annihilate" this sequence (forcing the output to zero).

The roots of the polynomial formed by this annihilating filter correspond exactly to the continuous-time locations of the innovations (such as the spike times). Once the locations are found, solving for the amplitudes becomes a straightforward linear least-squares problem.

This relationship reveals a deep duality between signal sparsity and matrix structure. As Ye et al. explain in Compressive Sampling Using Annihilating Filter-Based Low-Rank Interpolation, because the sequence can be annihilated by a short filter, a structured matrix (like a Hankel matrix) built from the samples must be inherently low-rank. This structural insight allows researchers to use low-rank matrix completion to recover missing or compressed samples before performing the final parametric reconstruction.

Bridging the Gap to the Real World

While the pure algebraic formulation of FRI is mathematically elegant, it suffers from two major vulnerabilities in practice:

  1. Kernel Sensitivity: Real hardware cannot always implement the mathematically perfect kernels required for exact exponential reproduction.
  2. Noise Instability: Root-finding algorithms are notoriously sensitive to noise; a tiny perturbation in the samples can cause the reconstructed pulse locations to shift wildly or disappear.

To make FRI practical, modern research has focused on relaxing kernel requirements and introducing statistical robustness.

Arbitrary and Realizable Kernels

Instead of forcing hardware to implement strict exponential-reproducing filters, Oñativia et al. introduced the "Approximate Strang-Fix" framework in Approximate Strang-Fix: sampling infinite streams of Diracs with any kernel. By allowing a controlled approximation error, this method enables the use of virtually any practical sampling kernel, making the theory compatible with off-the-shelf analog-to-digital converters.

Optimization and Stochastic Reconstruction

To combat noise, researchers have shifted away from fragile algebraic root-finding toward optimization and Bayesian estimation:

Application: Tracking Neural Activity

A compelling validation of the FRI framework is found in computational neuroscience, specifically in calcium imaging. To track the activity of thousands of neurons simultaneously, scientists use fluorescent indicators that light up when a neuron fires. However, the recorded signal is not a clean set of spikes; it is a noisy, blurred time-series where each action potential triggers a calcium transient characterized by a non-instantaneous rise and a slow exponential decay.

Spikes (Innovation) ---> [ Calcium Dynamics ] ---> Blurred, Noisy Fluorescent Signal

This is fundamentally an FRI problem. The underlying signal is a stream of impulses, and the calcium dynamics act as a natural sampling kernel. By extending the FRI framework to accommodate these generalized, non-instantaneous pulse shapes, Reynolds et al. demonstrated in AN EXTENSION OF THE FRI FRAMEWORK FOR CALCIUM TRANSIENT DETECTION that mathematical models can accurately deconvolve the blur and noise, pinpointing exact spike locations with remarkable precision.

By moving beyond the constraints of bandwidth and anchoring sampling rates to the true information density of a signal, FRI sampling provides a mathematically rigorous and physically viable bridge between the continuous and digital worlds.

Key Foundational Work

Exact sampling results for 1-D and 2-D signals with finite rate of innovation using Strang-Fix conditions and local reconstruction algorithms (2005) — contributed 8 concept elements to this direction.
Resolution enhancement and sampling with wavelets and footprints (2003) — contributed 4 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_innovation_rate_sampling_69ad429e,
  title = {innovation-rate sampling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/innovation-rate-sampling/rdr_69ad429ec0394b489d779a4522df2e6e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_innovation_rate_sampling_69ad429e_v1,
  title = {innovation-rate sampling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/innovation-rate-sampling/rdr_69ad429ec0394b489d779a4522df2e6e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake