Abstract
We consider the transmission of a Gaussian vector source over a multi-dimensional Gaussian channel where a random or a fixed subset of the channel outputs are erased. Within the setup where the only encoding operation allowed is a linear unitary transformation on the source, we investigate the MMSE performance, both in average, and also in terms of guarantees that hold with high probability as a function of the system parameters. Under the performance criterion of average MMSE, necessary conditions that should be satisfied by the optimal unitary encoders are established and explicit solutions for a class of settings are presented. For random sampling of signals that have a low number of degrees of freedom, we present MMSE bounds that hold with high probability. Our results illustrate how the spread of the eigenvalue distribution and the unitary transformation contribute to these performance guarantees. The performance of the discrete Fourier transform (DFT) is also investigated. As a benchmark, we investigate the equidistant sampling of circularly wide-sense stationary (c.w.s.s.) signals, and present the explicit error expression that quantifies the effects of the sampling rate and the eigenvalue distribution of the covariance matrix of the signal. These findings may be useful in understanding the geometric dependence of signal uncertainty in a stochastic process. In particular, unlike information theoretic measures such as entropy, we highlight the basis dependence of uncertainty in a signal with another perspective. The unitary encoding space restriction exhibits the most and least favorable signal bases for estimation.
Introduction
In classical information theory, uncertainty measures like entropy are fundamentally basis-invariant. Whether a Gaussian signal is represented in its original coordinate system or rotated via a unitary transformation, its total entropy remains unchanged. However, this invariance does not hold in practical sensing and communication scenarios where the channel is imperfect. When a signal is transmitted over a Gaussian erasure channel—where a subset of the components is lost or unsampled—the basis in which the signal is represented becomes a critical factor in determining the reconstruction error.
This paper investigates the role of unitary precoding in minimizing the Minimum Mean-Square Error (MMSE) for Gaussian vector sources. By rotating the signal's basis before transmission, we can potentially protect the most significant information from being lost due to random erasures.
System Model and the Precoding Problem
The researchers model the system as a linear noisy measurement process:
$$y = Hx + n$$
where $x$ is the precoded signal, $H$ is a diagonal sampling matrix representing the erasure pattern, and $n$ is additive Gaussian noise. The precoding operation is defined as $x = Uw$, where $w$ is the original Gaussian source and $U$ is an $N \times N$ unitary matrix.
The core objective is to find the optimal unitary matrix $U$ that minimizes the MMSE. The study approaches this from two perspectives:
- Average Performance: Minimizing the expected MMSE across all possible random erasure patterns.
- High-Probability Guarantees: Ensuring the MMSE remains below a certain threshold with high probability for specific signal families, particularly those with low degrees of freedom.
The Significance of Basis Choice
A key insight of the study is that the "spread" of signal energy across components is vital for robustness. In a noiseless scenario, the Identity transform represents the worst-case precoder. Because the Identity matrix preserves the signal's original alignment, its energy remains concentrated in specific components. If the channel erases a high-energy component, that information is irrecoverably lost.
Conversely, unitary precoders like the Discrete Fourier Transform (DFT) or the Walsh-Hadamard transform "spread" the signal's energy across all transmitted components. This ensures that even if several components are erased, the remaining samples still contain enough information to reconstruct the signal with relatively low error. This highlights a fundamental difference between information-theoretic uncertainty and estimation-theoretic uncertainty.
Optimality Conditions and the DFT
The paper establishes necessary first-order conditions that any optimal unitary encoder must satisfy using variational equalities. A common heuristic in compressive sensing is that the DFT is the ideal basis for sparse signals. The authors validate this for a specific case: when all non-zero eigenvalues of the signal's covariance matrix are equal (referred to as "flat support"). In this scenario, any unitary matrix with constant-magnitude entries (like the DFT) is indeed optimal.
However, the authors provide a significant theoretical contribution by proving that the DFT is not universally optimal. For signals with general eigenvalue distributions, they demonstrate through Lemma 2.7 that other unitary transformations can outperform the DFT, challenging the standard intuition that the DFT is always the best choice for sparse signal recovery.
High-Probability Bounds and Effective Degrees of Freedom
Moving beyond strict sparsity, the research introduces the concept of effective degrees of freedom (DOF). This metric characterizes signals where power is concentrated in a few eigenvalues, even if the signal is not strictly sparse.
The authors derive a rigorous bound showing that the MMSE is small with high probability if the number of samples $M$ is sufficiently large relative to the signal's mutual coherence and its effective DOF. This provides a practical guideline for system designers: as long as the precoder sufficiently "spreads" the signal and enough samples are collected, reliable reconstruction is guaranteed even under random erasures.
Benchmarking and SNR Penalty
To provide a baseline, the study compares random sampling against deterministic equidistant sampling for circularly wide-sense stationary (c.w.s.s.) signals. This comparison reveals an "SNR penalty"—a loss in effective Signal-to-Noise Ratio when moving from structured sampling to random erasures. This penalty is driven by the randomness of the erasure process and the presence of "insignificant" eigenvalues (small but non-zero) that contribute to noise without providing substantial signal power.
Conclusion
This work highlights the geometric dependence of signal uncertainty. Unlike entropy, MMSE under erasures is highly sensitive to the signal's basis. While spreading transforms like the DFT offer significant robustness, the search for a general, closed-form optimal precoder for arbitrary signals remains an open challenge. These findings offer a new perspective on how to design resilient communication and sensing systems by carefully choosing the representation basis to match the channel's characteristics and the signal's underlying structure.
Summary updated