Abstract
It is revealed that there is a link between the quantization approach employed and the dimension of the vector parameter which can be accurately estimated by a quantized estimation system. A critical quantity called inestimable dimension for quantized data (IDQD) is introduced, which doesn't depend on the quantization regions and the statistical models of the observations but instead depends only on the number of sensors and on the precision of the vector quantizers employed by the system. It is shown that the IDQD describes a quantization induced fundamental limitation on the estimation capabilities of the system. To be specific, if the dimension of the desired vector parameter is larger than the IDQD of the quantized estimation system, then the Fisher information matrix for estimating the desired vector parameter is singular, and moreover, there exist infinitely many nonidentifiable vector parameter points in the vector parameter space. Furthermore, it is shown that under some common assumptions on the statistical models of the observations and the quantization system, a smaller IDQD can be obtained, which can specify an even more limiting quantization induced fundamental limitation on the estimation capabilities of the system.
Introduction
In modern distributed systems like the Internet of Things (IoT) or wireless sensor networks, sensors collect continuous physical data and convert it into discrete bits (quantization) before sending it to a central "fusion center." This process inherently loses information. While researchers have long studied how to design better quantizers, a deeper question remains: Is there an absolute upper limit on how many parameters we can estimate from quantized data? This paper establishes a mathematical "hard wall," termed the Inestimable Dimension for Quantized Data (IDQD), that prevents accurate estimation when the parameter dimension is too high relative to the system's quantization resolution.
System Model and Methods
The researchers approach the problem using two classical criteria from estimation theory: Information-Regularity (whether the Fisher Information Matrix is non-singular) and Identifiability (whether unique parameter values produce unique probability distributions).
The system model, as illustrated above, involves $N$ sensors observing a parameter $\theta$. Each sensor performs quantization before transmitting data to a fusion center. The derived IDQD metric depends solely on the number of sensors ($N$) and the number of quantization levels ($R$). Using tools from linear algebra and algebraic topology, the authors prove that exceeding this dimension leads to a total breakdown of estimation criteria.
Key Results and Insights
-
The General Limit: For a system with $N$ sensors where each sensor $j$ uses a sequence of quantizers with $R_{jl}$ levels, the IDQD is defined as: $$\lambda = \sum_{j=1}^N \left( \prod_{l=1}^{L_j} R_{jl} \right) - N$$ If the dimension of your parameter vector $D_\theta > \lambda$, the Fisher Information Matrix (FIM) is guaranteed to be singular for any quantization scheme.
-
The Refined IDQD: If observations are independent, the limit drops significantly to a "refined" version: $$\lambda_{Indep} = \sum_{j=1}^N \sum_{l=1}^{L_j} R_{jl} - \sum_{j=1}^N L_j$$ This shows that independence, while simplifying math, actually reduces the maximum number of parameters you can track.
-
Model-Agnostic Nature: Surprisingly, the basic IDQD limit does not depend on the statistical distribution of noise or specific quantizer thresholds; it is a structural limitation of the system's "bit budget."
Numerical Examples and Nonidentifiability
While exceeding the IDQD makes the entire space nonidentifiable, it doesn't mean every single point is nonidentifiable. However, infinitely many nonidentifiable parameter points emerge.
Numerical simulations using Gaussian observation models visualize the "contours" of nonidentifiability. As shown in the figure, different parameter pairs become observationally equivalent once the IDQD limit is crossed, as they lie on the same probability contour for the quantized output.
Limitations and Future Work
The current study assumes error-free transmission to the fusion center. In real-world wireless networks, channel noise would likely degrade these limits further. Additionally, while this work focuses on static parameters, extending these findings to dynamic state estimation (e.g., Kalman filtering) and exploring local estimation accuracy in high-dimensional manifolds remain important areas for future research.
Summary updated