On the Equivalence of f-Divergence Balls and Density Bands in Robust Detection

Michael Fauß, Abdelhak M. Zoubir, H. Vincent PoorICASSP 2018

Abstract

The paper deals with minimax optimal statistical tests for two composite hypotheses, where each hypothesis is defined by a non-parametric uncertainty set of feasible distributions. It is shown that for every pair of uncertainty sets of the f-divergence ball type, a pair of uncertainty sets of the density band type can be constructed, which is equivalent in the sense that it admits the same pair of least favorable distributions. This result implies that robust tests under $f$-divergence ball uncertainty, which are typically only minimax optimal for the single sample case, are also fixed sample size minimax optimal with respect to the equivalent density band uncertainty sets.

Introduction

In robust detection, the primary objective is to design statistical tests that remain effective even when the actual data distribution deviates from a nominal model. This is achieved by defining an uncertainty set—a neighborhood of distributions around the nominal model—and identifying a minimax optimal test that performs best in the worst-case scenario within that set.

Two common methods for defining these sets are $f$-divergence balls and density bands. $f$-divergence balls, which use measures like Kullback-Leibler (KL) divergence or Hellinger distance, are mathematically elegant but often lack optimality for sequences of observations (fixed sample size). Conversely, density bands define pointwise upper and lower bounds on probability density and are well-behaved across multiple observations. This paper provides a theoretical bridge between these two models, justifying the use of $f$-divergence-based tests in multi-sample scenarios.

Theoretical Framework and Equivalence

The authors represent the robust detection problem as a saddle-point optimization where a statistician chooses a test and "nature" chooses the Least Favorable Distributions (LFDs) within the uncertainty sets to maximize error. By applying Karush-Kuhn-Tucker (KKT) conditions, the researchers analyzed the structure of LFDs for $f$-divergence balls and discovered they share the same mathematical form as LFDs produced by density band models.

Specifically, the paper proves a Theorem of Equivalence: for every pair of $f$-divergence balls, there exists a pair of equivalent density bands that yield the exact same worst-case distributions. This implies that $f$-divergence-based tests, while typically derived for single samples, are also fixed-sample-size optimal when viewed through the lens of their equivalent density bands.

Numerical Verification

The authors validated their findings using a synthetic numerical example with two Gaussian nominal models: $\mathcal{N}(-1, 1)$ and $\mathcal{N}(1, 2)$. They applied KL-divergence balls with radii $\epsilon_0 = 0.03$ and $\epsilon_1 = 0.02$ to calculate the equivalent density band parameters ($a_0, b_0, a_1, b_1$).

Least favorable distributions (LFDs) $q_i(x)$ and their corresponding density bands $a_i p_i(x), b_i p_i(x)$ for the Gaussian example.

As illustrated in the figure, the LFDs $q_0(x)$ and $q_1(x)$ (solid lines) are effectively "clipped" by the equivalent density bands (dashed lines). This mapping allows abstract divergence values to be interpreted as intuitive "contamination ratios"; for example, a KL radius of 0.03 for $H_0$ corresponded to approximately a 10% contamination ratio.

Conclusion

This research provides a rigorous theoretical justification for using single-sample robust tests in multi-sample scenarios. By establishing this equivalence, researchers can leverage the extensive literature on $f$-divergence balls while maintaining minimax optimality for fixed sample sizes. Future work may explore whether this equivalence extends to other uncertainty models, such as those based on Wasserstein distance.

Summary updated

Abstract

The paper deals with minimax optimal statistical tests for two composite hypotheses, where each hypothesis is defined by a non-parametric uncertainty set of feasible distributions. It is shown that for every pair of uncertainty sets of the f-divergence ball type, a pair of uncertainty sets of the density band type can be constructed, which is equivalent in the sense that it admits the same pair of least favorable distributions. This result implies that robust tests under $f$-divergence ball uncertainty, which are typically only minimax optimal for the single sample case, are also fixed sample size minimax optimal with respect to the equivalent density band uncertainty sets.
BibTeX
@inproceedings{fau2018equivalence,
  author = {Michael Fauß and Abdelhak M. Zoubir and H. Vincent Poor},
  title = {{On the Equivalence of f-Divergence Balls and Density Bands in Robust Detection}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2018},
}
Figure