Minimax Linear Smoothing for Capacities

H. Vincent PoorThe Annals of Probability 1982 · DOI

Abstract

Minimax linear smoothers are considered for the problem of estimating a homogeneous signal field in an additive orthogonal noise field. A minimax game with the quadratic-mean estimation error as an objective function is used to formulate this problem. Uncertainty in signal and noise field spectra is modeled using general nonparametric classes of measures proposed by Huber and Strassen for the problem of minimax hypothesis testing. These classes, which are described in terms of Choquet alternating capacities of order 2, include the conventional models for spectral uncertainty and admit a general solution to the minimax linear smoothing problem.

Introduction: The Challenge of Spectral Uncertainty

In classical signal processing, the Wiener filter represents the optimal solution for estimating a signal obscured by additive noise, provided the power spectral densities (PSDs) of both the signal and the noise are known exactly. However, in practical applications, such precise knowledge is rarely available. Engineers often work with nominal models that are subject to uncertainty due to measurement errors, modeling approximations, or outliers.

If a filter is optimized for a specific nominal spectrum but the actual signal or noise deviates from this model, performance can degrade significantly. This paper, published by H. Vincent Poor in The Annals of Probability (1982), addresses this vulnerability by formulating the minimax linear smoothing problem. The goal is to design a linear filter that minimizes the maximum possible estimation error across a defined class of potential spectral models.

Theoretical Framework: 2-Alternating Capacities

Poor's primary contribution is the generalization of robust filtering techniques using the theory of Choquet capacities. While previous work had addressed specific uncertainty models like $\epsilon$-contamination, Poor utilizes "2-alternating capacities" to define neighborhoods of measures. This mathematical framework is highly flexible, encompassing common models such as Prohorov neighborhoods and total variation neighborhoods.

The problem is framed as a minimax game between two players:

  1. The Designer: Chooses a filter transfer function $h$ to minimize the mean square error.
  2. Nature: Chooses spectral measures $m_S$ (signal) and $m_N$ (noise) from uncertainty sets $\mathcal{M}_S$ and $\mathcal{M}_N$ to maximize the error.

The Minimax Solution: Finding the Least-Favorable Pair

The core of the paper's methodology relies on adapting the Huber-Strassen theory—originally developed for minimax hypothesis testing—to the context of signal estimation. Poor proves that a saddle-point exists for this game.

The search for a robust filter is shown to be equivalent to identifying a "least-favorable" pair of spectral measures $(q_S, q_N)$ within the allowed uncertainty sets. Once this worst-case scenario is identified, the minimax-optimal filter is simply the standard Wiener filter designed for that specific pair.

Key Theorems

  • Theorem 2.2: Establishes that if a specific derivative exists between the capacities of the signal and noise, a minimax linear smoother $h_0$ can be constructed.
  • Theorem 2.3: Characterizes the least-favorable pair as the one that maximizes the minimum mean square error, confirming the saddle-point property.

Practical Implications: The "Clipped" Filter Structure

For many practical uncertainty models, the resulting minimax filter exhibits an intuitive "clipped" or "saturated" structure. In Section 3, Poor provides a concrete formula for the transfer function $h_0(\omega)$ when uncertainty is modeled as a neighborhood around nominal measures:

$$h_0(\omega) = \max{k', \min{k'', h'(\omega)}}

In this expression, $h'(\omega)$ is the nominal optimal filter, and $k', k''$ are constants determined by the degree of uncertainty. This structure implies that in high signal-to-noise ratio (SNR) regions, the filter "trusts" the signal less than the nominal filter would, protecting against cases where the signal is weaker than expected. Conversely, in low SNR regions, the filter "trusts" the noise less, protecting against cases where the noise is stronger than expected.

Conclusion and Limitations

Poor’s work provides a unified, rigorous foundation for robust linear smoothing. By leveraging the language of capacities, the paper offers a "one size fits all" solution for various types of spectral uncertainty. However, several limitations remain. The study is restricted to linear smoothers, whereas non-linear estimators might offer superior performance in certain non-Gaussian environments. Additionally, the results assume homogeneous (stationary) signal and noise fields; extending these concepts to non-stationary environments would require significant further development. Despite these constraints, the paper remains a seminal work in robust statistics and signal processing, bridging the gap between abstract measure theory and practical filter design.

Summary updated

Abstract

Minimax linear smoothers are considered for the problem of estimating a homogeneous signal field in an additive orthogonal noise field. A minimax game with the quadratic-mean estimation error as an objective function is used to formulate this problem. Uncertainty in signal and noise field spectra is modeled using general nonparametric classes of measures proposed by Huber and Strassen for the problem of minimax hypothesis testing. These classes, which are described in terms of Choquet alternating capacities of order 2, include the conventional models for spectral uncertainty and admit a general solution to the minimax linear smoothing problem.
BibTeX
@article{poor1982minimax,
  author = {H. Vincent Poor},
  title = {{Minimax Linear Smoothing for Capacities}},
  journal = {The Annals of Probability},
  year = {1982},
  doi = {10.1214/aop/1176993874},
}
Figure