Shannon Meets Nyquist: Capacity Limits of Sampled Analog Channels

Yuxin Chen, Yonina C. Eldar, Andrea J. Goldsmith

Abstract

We explore two fundamental questions at the intersection of sampling theory and information theory: how channel capacity is affected by sampling below the channel's Nyquist rate, and what sub-Nyquist sampling strategy should be employed to maximize capacity. In particular, we derive the capacity of sampled analog channels for three prevalent sampling strategies: sampling with filtering, sampling with filter banks, and sampling with modulation and filter banks. These sampling mechanisms subsume most nonuniform sampling techniques applied in practice. Our analyses illuminate interesting connections between under-sampled channels and multiple-input multiple-output channels. The optimal sampling structures are shown to extract out the frequencies with the highest SNR from each aliased frequency set, while suppressing aliasing and out-of-band noise. We also highlight connections between undersampled channel capacity and minimum mean-squared error (MSE) estimation from sampled data. In particular, we show that the filters maximizing capacity and the ones minimizing MSE are equivalent under both filtering and filter-bank sampling strategies. These results demonstrate the effect upon channel capacity of sub-Nyquist sampling techniques, and characterize the tradeoff between information rate and sampling rate.

Introduction

In classical communication theory, the receiver is typically assumed to sample the incoming signal at or above the Nyquist rate—twice the signal's bandwidth. This ensures that the continuous-time signal can be perfectly reconstructed in the discrete domain without information loss. However, modern wideband systems, such as cognitive radio or ultra-wideband sensing, often encounter a "sampling bottleneck" where hardware limitations like power consumption and ADC speed make Nyquist-rate sampling prohibitively expensive.

This work addresses two core questions:

  1. What is the Shannon capacity loss when forced to sample below the Nyquist rate?
  2. What is the mathematically optimal design for the analog front end (filters and modulators) to maximize the information rate under sub-Nyquist constraints?

Methodology: The MIMO Equivalence

The authors analyze three increasingly complex sampling architectures: single-branch filtering, multi-branch filter banks, and a general architecture involving modulation followed by filter banks. The latter is similar to the "Modulated Wideband Converter" used in compressed sensing.

The key innovation lies in treating the aliasing caused by sub-Nyquist sampling as a mathematical feature. By analyzing the system in the frequency domain, the "folding" of frequencies due to undersampling is shown to be equivalent to a Multiple-Input Multiple-Output (MIMO) or Multiple-Input Single-Output (MISO) channel.

Equivalent MIMO frequency-domain model showing the folding of spectral components due to sampling.

Using asymptotic spectral properties of Toeplitz operators, the authors derive closed-form capacity expressions for these "folded" channels. This theoretical framework allows for the evaluation of capacity across various sampling rates and architectures.

Key Insights and Optimal Strategies

A primary finding is that alias suppression is optimal for capacity maximization. While many sub-Nyquist reconstruction techniques (like Compressed Sensing) aim to "mix" frequencies for later recovery, maximizing information rate requires the receiver to use filters that select the "cleanest" (highest SNR) frequency bins and suppress others to prevent noise from aliasing into the signal.

Furthermore, the paper establishes a "Shannon-MMSE Connection," proving that the filters maximizing Shannon capacity are identical to those minimizing the Minimum Mean-Squared Error (MMSE) of signal reconstruction. This reconciles information-theoretic rate goals with sampling-theoretic accuracy goals.

Results and Performance

The study provides a parametric "water-filling" formula for the capacity of sampled Gaussian channels. For multiband signals, it is shown that the full analog channel capacity can be achieved by sampling at the Landau rate (the sum of the widths of the active sub-bands), provided a multi-branch filter bank is employed.

Analysis of channel response and capacity: (a) Original channel, (b) Optimal prefilter, (c) Filtered channel, and (d) Capacity vs. sampling rate.

As seen in the results, capacity is not always monotonic with the sampling rate. A slight increase in sampling rate could potentially decrease capacity if it causes a high-noise region to alias onto a high-signal region. However, with optimal filtering, the capacity generally increases until it plateaus at the Nyquist rate (the "super-Nyquist regime").

Discussion and Future Work

While this work provides a robust theoretical foundation, it assumes perfect Channel State Information (CSI) at the receiver. In practical sub-Nyquist systems, estimating the channel from undersampled data remains a significant challenge. Additionally, the current framework focuses on Linear Time-Invariant (LTI) channels and periodic modulation sequences. Future research directions include extending these results to multiuser interference channels and scenarios where the spectral support of the signal is unknown a priori.

Summary updated

Abstract

We explore two fundamental questions at the intersection of sampling theory and information theory: how channel capacity is affected by sampling below the channel's Nyquist rate, and what sub-Nyquist sampling strategy should be employed to maximize capacity. In particular, we derive the capacity of sampled analog channels for three prevalent sampling strategies: sampling with filtering, sampling with filter banks, and sampling with modulation and filter banks. These sampling mechanisms subsume most nonuniform sampling techniques applied in practice. Our analyses illuminate interesting connections between under-sampled channels and multiple-input multiple-output channels. The optimal sampling structures are shown to extract out the frequencies with the highest SNR from each aliased frequency set, while suppressing aliasing and out-of-band noise. We also highlight connections between undersampled channel capacity and minimum mean-squared error (MSE) estimation from sampled data. In particular, we show that the filters maximizing capacity and the ones minimizing MSE are equivalent under both filtering and filter-bank sampling strategies. These results demonstrate the effect upon channel capacity of sub-Nyquist sampling techniques, and characterize the tradeoff between information rate and sampling rate.
BibTeX
@inproceedings{chen2011shannon,
  author = {Yuxin Chen and Yonina C. Eldar and Andrea J. Goldsmith},
  title = {{Shannon Meets Nyquist: Capacity Limits of Sampled Analog Channels}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2011},
  doi = {10.1109/icassp.2011.5946352},
}
Figure