Alternating Direction Method of Multipliers for Negative Binomial Model with The Weighted Difference of Anisotropic and Isotropic Total Variation

Yu Lu, Kevin Bui, Roummel F. Marcia

Abstract

In many applications such as medical imaging, the measurement data represent counts of photons hitting a detector. Such counts in low-photon settings are often modeled using a Poisson distribution. However, this model assumes that the mean and variance of the signal's noise distribution are equal. For overdispersed data where the variance is greater than the mean, the negative binomial distribution is a more appropriate statistical model. In this paper, we propose an optimization approach for recovering images corrupted by overdispersed Poisson noise. In particular, we incorporate a weighted anisotropic-isotropic total variation regularizer, which avoids staircasing artifacts that are introduced by a regular total variation penalty. We use an alternating direction method of multipliers, where each subproblem has a closed-form solution. Numerical experiments demonstrate the effectiveness of our proposed approach, especially in very photon-limited settings.

Introduction

In fields like medical imaging and astronomy, researchers often deal with "photon-limited" data, where they are essentially counting individual particles hitting a sensor. The standard statistical tool for this is the Poisson distribution. However, the Poisson model has a strict limitation: it assumes the mean and the variance of the noise are identical.

In many real-world scenarios, the data is "overdispersed," meaning the variance is actually higher than the mean. Using a Poisson model here leads to poor image recovery. This research addresses this by using the Negative Binomial (NB) distribution, which is a more flexible generalization of the Poisson distribution that can account for this extra variance. Furthermore, the common "staircasing" problem in image processing—where smooth gradients in an image are turned into blocky, artificial steps—is addressed by using a sophisticated regularization technique.

Proposed Methodology

The proposed optimization model minimizes the negative log-likelihood of the Negative Binomial distribution combined with a weighted difference of anisotropic and isotropic total variation (AITV).

Key Technical Components

  1. The NB Model: The objective function is derived from the NB probability mass function, resulting in a loss function that better fits overdispersed data.
  2. AITV Regularization: Instead of standard Total Variation (TV), which often causes blocky artifacts, the model uses the difference between Anisotropic TV (which favors horizontal/vertical edges) and Isotropic TV (which is rotationally invariant). This "weighted difference" helps preserve edges while keeping smooth regions smooth.
  3. ADMM Algorithm: To solve the resulting complex optimization problem, the Alternating Direction Method of Multipliers (ADMM) is employed. This breaks the problem into three smaller subproblems:
  4. The $f$-subproblem: Solved efficiently in the frequency domain using Fast Fourier Transforms (FFT).
  5. The $v$-subproblem: Involves solving a cubic equation for each pixel, for which a closed-form solution is provided.
  6. The $w$-subproblem: Solved using a "proximal operator," which also has a known closed-form solution.

By ensuring every subproblem has a closed-form solution, the algorithm remains computationally efficient despite the complexity of the NB model.

Experimental Setup

The method was evaluated using five standard test images from the Berkeley Segmentation Dataset: Airplane, Train, Elephant, Elk, and Skier.

Standard test images used for evaluation: (a) Airplane, (b) Train, (c) Elephant, (d) Elk, and (e) Skier.

Various noise levels were simulated by adjusting the dispersion parameter $r$ (where lower $r$ indicates higher overdispersion/noise). Testing included $r$ values ranging from 1 (extremely noisy/overdispersed) to 1000 (nearly Poisson).

Results and Analysis

The NB model significantly outperformed the Poisson model in low-photon, high-overdispersion settings ($r=1$ and $r=10$):

  • PSNR (Peak Signal-to-Noise Ratio): For the "Airplane" image at $r=1$, the NB model achieved a PSNR of 23.85 dB, compared to just 13.24 dB for the Poisson model.
  • SSIM (Structural Similarity Index): In the same setting, the NB model scored 0.84 vs. the Poisson model's 0.40, indicating much better preservation of structural details.
  • Visual Quality: Qualitatively, the Poisson reconstructions appeared "speckled" and noisy in the sky regions, whereas the NB reconstructions were significantly smoother and more natural-looking.

As expected, as $r$ increased to 1000, the performance of both models became nearly identical, confirming that the NB model successfully generalizes the Poisson case.

Conclusion and Future Work

The research demonstrates that the NB model is a "safe bet"—it performs better on overdispersed data and matches Poisson performance on standard data. The AITV regularizer effectively penalizes artifacts that make recovered images look blocky.

However, some limitations remain:

  • Parameter Estimation: In these experiments, the exact value of the dispersion parameter $r$ was used as a prior. In real-world applications, $r$ is often unknown and must be estimated.
  • Sensitivity: While the algorithm isn't highly sensitive to the exact value of $r$, a robust method for automatic $r$-estimation would improve usability.
  • Computational Complexity: While the subproblems have closed-form solutions, the ADMM approach requires multiple iterations and FFTs, which might be a bottleneck for real-time high-resolution processing.

Summary updated

Abstract

In many applications such as medical imaging, the measurement data represent counts of photons hitting a detector. Such counts in low-photon settings are often modeled using a Poisson distribution. However, this model assumes that the mean and variance of the signal's noise distribution are equal. For overdispersed data where the variance is greater than the mean, the negative binomial distribution is a more appropriate statistical model. In this paper, we propose an optimization approach for recovering images corrupted by overdispersed Poisson noise. In particular, we incorporate a weighted anisotropic-isotropic total variation regularizer, which avoids staircasing artifacts that are introduced by a regular total variation penalty. We use an alternating direction method of multipliers, where each subproblem has a closed-form solution. Numerical experiments demonstrate the effectiveness of our proposed approach, especially in very photon-limited settings.
BibTeX
@misc{lu2024alternating,
  author = {Yu Lu and Kevin Bui and Roummel F. Marcia},
  title = {{Alternating Direction Method of Multipliers for Negative Binomial Model with The Weighted Difference of Anisotropic and Isotropic Total Variation}},
  year = {2024},
  eprint = {2408.16117},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.16117},
}
Figure