Sensitivity-Aware Density Estimation in Multiple Dimensions

Aleix Boquet-Pujadas, Pol del Aguila Pla, Michael Unser

Abstract

Estimating probability densities in multi-dimensional spaces is a fundamental challenge, particularly when data is sampled with uneven sensitivity. We present Regularized-Density Splines (RDS), a framework that models log-density as a spline and employs Hessian-Schatten regularization. This approach achieves spatial adaptivity and high computational efficiency, addressing the curse of dimensionality and sensitivity heterogeneity. RDS demonstrates significant performance gains in medical imaging tasks, such as PET rebinning, outperforming standard techniques like Kernel Density Estimation (KDE) in both accuracy and speed.

Introduction

Density Estimation (DE) is critical for statistics and signal processing but faces significant hurdles in real-world applications. Three primary challenges include the curse of dimensionality, where data sparsity makes standard methods like KDE computationally prohibitive; uneven detector sensitivity, which leads to noise instability in traditional weighted DE methods; and complex boundary conditions that standard tools often ignore, leading to edge bias.

Methodology: Regularized-Density Splines (RDS)

The RDS framework is built on four key pillars designed to overcome these challenges:

  1. Exponential Spline Family: By modeling density as the exponential of a spline on a uniform grid, RDS ensures non-negativity and enables fast, separable convolutions for density evaluation.
  2. Sensitivity-Aware Likelihood: Detector sensitivity is incorporated directly into the likelihood function's normalization. This avoids the "division by zero" problem found in traditional weighted KDE by treating sensitivity as a change of measure.
  3. Hessian-Schatten Regularization: This core innovation uses the nuclear norm (Schatten $p=1$ norm) of the Hessian of the log-density. This promotes "locally affine" structures, acting as an automatic, spatially adaptive bandwidth selector that manages the bias-variance tradeoff.
  4. Accelerated Optimization: A proximal-gradient descent algorithm with an adaptive Lipschitz stepsize ensures stable and fast convergence, even though the log-likelihood is not globally Lipschitz.
The effect of the regularization parameter $\lambda$ on the shape of the estimated log-density, demonstrating the transition from highly curved to locally affine structures.

Experimental Evaluation

The framework was evaluated on synthetic distributions (Uniform, Gaussian, and Laplacian mixtures) and realistic medical imaging datasets. These included simulated PET phantoms (Brain Phantom, Digimouse) and real-world Amyloid PET scans characterized by highly heterogeneous sensitivity and detector gaps.

Key Results and Insights

RDS provides several advantages over traditional methods:

  • Efficiency: By using a grid-based representation, the cost of evaluating the density is independent of the number of samples ($N$). In 2D, RDS optimized in roughly 1 second, while its evaluation time remained constant at $10^{-3}$ seconds even as $N$ increased.
  • Accuracy: RDS consistently outperforms Histogram Estimation (HE) and KDE. In 3D synthetic tests, it achieved a 15 dB improvement in Mean Squared Error (MSE) over HE.
  • Robustness: The Hessian regularization allows the model to remain "flat" where data is scarce and capture features only where supported by data.
MSE performance as a function of sample size for different spline degrees, demonstrating the significant accuracy gains of linear ($n=1$) and cubic ($n=3$) splines over the degree-0 baseline.

Qualitatively, RDS significantly reduces streak artifacts in PET rebinning compared to Filtered Back-Projection (FBP) and preserves contrast better than KDE in low-sensitivity regions.

Conclusion

Regularized-Density Splines offer a robust and efficient solution for multi-dimensional density estimation under uneven sensitivity. While the method relies on an underlying grid, its regularization properties and computational speed make it a powerful tool for medical imaging. Future work will explore the use of Box Splines for directional features and integrating RDS estimates into uncertainty-aware reconstruction frameworks.

Summary updated

Abstract

We formulate an optimization problem to estimate probability densities in the context of multidimensional problems that are sampled with uneven probability. It considers detector sensitivity as an heterogeneous density and takes advantage of the computational speed and flexible boundary conditions offered by splines on a grid. We choose to regularize the Hessian of the spline via the nuclear norm to promote sparsity. As a result, the method is spatially adaptive and stable against the choice of the regularization parameter, which plays the role of the bandwidth. We test our computational pipeline on standard densities and provide software. We also present a new approach to PET rebinning as an application of our framework.
BibTeX
@article{boquetpujadas2024sensitivity,
  author = {Aleix Boquet-Pujadas and Pol del Aguila Pla and Michael Unser},
  title = {{Sensitivity-Aware Density Estimation in Multiple Dimensions}},
  journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
  year = {2024},
}
Figure