Quantization of Prior Probabilities for Hypothesis Testing

Kush R. Varshney, Lav R. VarshneyIEEE TSP 2008 · DOI

Abstract

In classical Bayesian hypothesis testing, decision-makers are assumed to possess exact knowledge of prior probabilities. However, real-world agents—both biological and artificial—operate under finite memory and processing constraints. This paper proposes a framework for quantizing prior probabilities, modeling how limited "mental slots" lead to categorical information processing. We introduce the Mean Bayes Risk Error (MBRE) as a distortion metric and derive optimal quantization conditions using an adapted Lloyd-Max algorithm. Our analysis reveals that optimal categories are highly dependent on error costs and provides a mathematical explanation for systematic biases and social discrimination in resource-constrained agents.

Introduction

Decision-making often relies on pre-existing beliefs or prior probabilities. In many practical scenarios, such as a referee judging a player's conduct or a sensor network classifying signals, the number of possible priors is vast, yet the capacity to store and process them is limited. This necessitates the quantization of the prior space into a finite set of representative levels. While standard quantization techniques focus on minimizing Mean Squared Error (MSE), hypothesis testing requires a metric that accounts for the impact of quantization on decision quality.

Methodology: Mean Bayes Risk Error

The core contribution of this work is the definition of Mean Bayes Risk Error (MBRE). Unlike traditional distortion measures, MBRE quantifies the difference between the Bayes risk achieved with the true prior and the risk achieved using a quantized, mismatched prior.

To find the optimal quantization levels, we adapt the Lloyd-Max conditions:

  1. Nearest Neighbor Condition: The boundary between two categories is defined by the intersection of the mismatched Bayes risk functions of adjacent representation points.
  2. Centroid Condition: The optimal representative prior for a given interval is determined by a specific ratio of the conditional means of the prior distribution.
Quantization of the Bayes risk function for K = 1, 2, 3, and 5 levels, illustrating how increasing 'mental slots' improves the approximation of the true risk function.

As shown in Figure 4, as the number of quantization levels $K$ increases, the piecewise linear approximation (the quantized risk) more closely tracks the true Bayes risk function. This visualization demonstrates the "decision-aware" nature of the quantization process, where categories are formed based on the risk profile rather than simple distance.

High-Resolution Theory and Results

For systems with a large number of categories, we employ high-resolution theory to derive a distortion-rate function. This approximation shows that the MBRE decreases at a rate proportional to $1/K^2$, similar to standard quantization but with a scaling factor dependent on the geometry of the Bayes risk function.

The Mean Bayes Risk Error (MBRE) as a function of the number of quantization levels K. The error decreases monotonically, with the optimal quantizer outperforming standard metrics.

Numerical simulations using Uniform and Beta distributions validate these theoretical findings. Figure 3 illustrates the monotonic decrease in distortion $D$ as the number of quantization levels $K$ increases. Notably, the MBRE-optimal quantizer consistently outperforms quantizers designed using standard Mean Absolute Error (MAE), particularly when the costs of false alarms and misses are asymmetric.

Social Implications: Resource Allocation and Bias

A significant insight from this framework is its application to social discrimination. If a decision-maker interacts primarily with a majority group, they naturally allocate more quantization levels to that group to minimize overall error. Consequently, minority groups are represented by coarser categories, leading to higher decision errors—a phenomenon known as "own-race bias." This suggests that systematic discrimination can emerge from rational resource allocation under segregation, even in the absence of explicit prejudice.

Conclusion

This paper provides a rigorous mathematical foundation for the quantization of prior probabilities in hypothesis testing. By shifting the focus from signal reconstruction to decision performance, the MBRE framework captures the nuances of resource-constrained decision-making. Future work will extend these concepts to $M$-ary hypothesis testing and explore the dynamics of learning quantized priors over time.

Summary updated

BibTeX
@article{varshney2008quantization,
  author = {Kush R. Varshney and Lav R. Varshney},
  title = {{Quantization of Prior Probabilities for Hypothesis Testing}},
  journal = {IEEE Trans. Signal Process.},
  year = {2008},
  doi = {10.1109/tsp.2008.928164},
}
Figure