Concentration Inequalities and Laws of Large Numbers under Irrelevance of Lower and Upper Expectations

Fábio Gagliardi CozmanarXiv 2009 · arXiv

Abstract

This paper presents concentration inequalities and laws of large numbers under weak assumptions of irrelevance, expressed through lower and upper expectations. The results extend de Cooman and Miranda’s recent results concerning epistemic irrelevance. The proofs indicate connections between concepts of irrelevance for lower/upper expectations and the standard theory of martingales.

Introduction

In classical probability theory, we typically assume the existence of a single, precisely known probability distribution. However, in many real-world applications—ranging from robust statistics and decision-making under disagreement to expert systems—this assumption is often too rigid. In such cases, we may only possess a range of possible expectations, known as lower and upper expectations. This "imprecise probability" framework provides a more robust way to model uncertainty.

A significant challenge in this field is defining independence. Standard stochastic independence is often too restrictive for imprecise models. Researchers have previously developed the concept of epistemic irrelevance, where learning about one variable does not alter the upper or lower expectations of another. While progress has been made—notably by de Cooman and Miranda—their proofs for Laws of Large Numbers (LLN) were often complex and restricted to specific bounded conditions. This paper simplifies these proofs, tightens the resulting bounds, and extends the theory to unbounded variables.

Theoretical Framework: Bridging Imprecision and Martingales

The core contribution of this work is the bridge built between imprecise probability and classical martingale theory. The author introduces a relaxation of "forward irrelevance" termed weak irrelevance. Essentially, this condition requires that the conditional upper and lower expectations of a variable, given past observations, remain equal to the unconditional expectations.

The most significant technical insight is that sequences of variables satisfying these irrelevance conditions can be transformed into martingales. By demonstrating that epistemic irrelevance is the natural imprecise counterpart to the martingale property, the author is able to leverage decades of progress in standard probability theory to solve problems in the imprecise domain.

Concentration Inequalities

By mapping these sequences to martingales, the paper derives imprecise versions of fundamental concentration inequalities:

  1. Hoeffding Inequality: Derived for independent-like bounded variables.
  2. Azuma Inequality: Derived for dependent sequences.

The paper provides explicit exponential bounds (Theorems 1 and 2). For a sequence of $n$ bounded variables, the probability that the average deviates from the upper expectation by more than $\epsilon$ vanishes at a rate of $e^{-2n\epsilon^2/B^2}$. These results show that the sample mean will eventually stay within the interval defined by the lower and upper expectations, even under the looser assumption of weak irrelevance.

Laws of Large Numbers

The paper establishes both Weak and Strong Laws of Large Numbers for the imprecise setting:

  • Weak Law of Large Numbers: The author proves that the sample mean converges in probability to the interval $[\underline{\mu}, \overline{\mu}]$.
  • Strong Law for Unbounded Variables: While previous work focused on variables within a fixed range (e.g., $[0, 1]$), this paper utilizes the Kolmogorov-Hajek-Renyi inequality to extend the Strong Law of Large Numbers to unbounded variables with finite variance. This extension assumes countable additivity, showing that the sample mean almost surely enters the expectation interval provided the "imprecision" (the gap between upper and lower expectations) remains finite.

Discussion and Limitations

While these results represent a significant step forward in making imprecise probability more practically applicable, several theoretical hurdles remain:

  • Disintegrability: The proofs rely on the property of disintegrability, which ensures that conditional expectations behave predictably. It remains an open question whether these laws hold in systems where disintegrability fails.
  • Countable Additivity: The extension to unbounded variables requires the assumption of countable additivity. Some frameworks of imprecise probability, such as Walley’s, prefer to avoid this assumption, suggesting a need for future research to bridge this gap.
  • Tightness of Bounds: While the paper provides functional concentration inequalities, further investigation is needed to determine if these are the tightest possible bounds for the imprecise setting.

Conclusion

By demonstrating that "weak irrelevance" is sufficient to ensure stable long-term averages, this paper simplifies and generalizes the foundations of imprecise probability. The connection to martingale theory not only provides more elegant proofs but also opens the door for further integration of classical probabilistic tools into robust statistical frameworks.

Summary updated

Abstract

This paper presents concentration inequalities and laws of large numbers under weak assumptions of irrelevance, expressed through lower and upper expectations. The results extend de Cooman and Miranda’s recent results concerning epistemic irrelevance. The proofs indicate connections between concepts of irrelevance for lower/upper expectations and the standard theory of martingales.
BibTeX
@misc{cozman2009concentration,
  author = {Fábio Gagliardi Cozman},
  title = {{Concentration Inequalities and Laws of Large Numbers under Irrelevance of Lower and Upper Expectations}},
  year = {2009},
  eprint = {0810.2821},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/0810.2821},
}

Loading related content...

Figure