Information-theoretic upper and lower bounds for statistical estimation

Tong ZhangIEEE Trans. Information Theory 2006 · OpenReview

Abstract

In this paper, we establish upper and lower bounds for some statistical estimation problems through concise information-theoretic arguments. Our upper bound analysis is based on a simple yet general inequality which we call the information exponential inequality. We show that this inequality naturally leads to a general randomized estimation method, for which performance upper bounds can be obtained. The lower bounds, applicable for all statistical estimators, are obtained by original applications of some well known information-theoretic inequalities, and approximately match the obtained upper bounds for various important problems. Moreover, our framework can be regarded as a natural generalization of the standard minimax framework, in that we allow the performance of the estimator to vary for different possible underlying distributions according to a predefined prior
BibTeX
@article{zhang2006information,
  author = {Tong Zhang},
  title = {{Information-theoretic upper and lower bounds for statistical estimation}},
  journal = {IEEE Trans. Information Theory},
  year = {2006},
  url = {https://openreview.net/forum?id=Q7tXIvrw6g},
}

Loading related content...

Figure