Difference of Anisotropic and Isotropic TV for Segmentation under Blur and Poisson Noise

Kevin Bui, Yifei Lou, Fredrick Park, Jack Xin

TL;DR

This paper introduces a robust image segmentation framework designed specifically for images degraded by blur and Poisson noise. By combining a Poisson-specific data fidelity term with a nonconvex "Anisotropic-Isotropic Total Variation" (AITV) regularizer, the authors achieve superior edge preservation and segmentation accuracy compared to traditional Gaussian-based models.

What Problem Does This Paper Address?

Most classical image segmentation models, like the famous Mumford-Shah model, assume that image noise follows a Gaussian distribution. However, in many high-stakes fields—such as medical imaging (PET scans) and astronomy—noise is often Poisson-distributed, arising from the discrete nature of photon counting. Furthermore, images are frequently blurred by the optics of the capturing device.

Standard Total Variation (TV) regularization is often used to handle noise while keeping edges sharp, but it has a tendency to "over-smooth" or create "staircase" artifacts. The challenge addressed here is creating a mathematically sound and computationally efficient way to segment images that are simultaneously blurry, noisy (Poisson), and contain intricate, thin structures that standard methods often erase.

Methods

The authors utilize a Smoothing-and-Thresholding (SaT) framework. This is a two-stage process: first, produce a clean, "piecewise-smooth" version of the image; second, apply $k$-means clustering to define the final segments.

The core innovation lies in the smoothing step's objective function:

  1. Poisson Fidelity: Instead of the standard least-squares (Gaussian) approach, they use a Maximum A Posteriori (MAP) term based on the Poisson distribution ($\langle Au - f \log Au, 1 \rangle$).
  2. AITV Regularization: They incorporate the weighted difference of anisotropic ($\ell_1$) and isotropic ($\ell_2$) total variation. This nonconvex term acts as a better proxy for sparsity in image gradients than standard TV, helping to preserve thin lines and sharp boundaries.
  3. ADMM Solver: To solve this complex nonconvex optimization problem, the authors developed a customized Alternating Direction Method of Multipliers (ADMM) algorithm. They utilize a specific "proximal operator" for the $\ell_1 - \alpha\ell_2$ term, which allows the algorithm to converge efficiently despite the model's nonconvexity.

For color images, they use a Smoothing, Lifting, and Thresholding (SLaT) approach, where the image is "lifted" into the Lab color space to better capture perceptual differences before clustering.

Datasets

The researchers evaluated their method on three distinct types of data:

  • DRIVE Dataset: 20 images of retinal blood vessels, used for binary segmentation of thin, branch-like structures.
  • BrainWeb Dataset: 20 simulated brain MRI images, used for "multiphase" segmentation (distinguishing between background, cerebrospinal fluid, grey matter, and white matter).
  • PASCAL VOC 2010: 10 natural color images used to test the SLaT framework on complex, real-world scenes.

Key Insights and Ideas

  • The Power of the Difference: The "AITV" term ($\ell_1 - \alpha\ell_2$) is the "secret sauce." While standard TV ($\ell_1$) promotes sparsity, the subtraction of the isotropic term ($\ell_2$) makes the regularizer "tighter" and more aggressive at preserving the specific geometry of edges without the typical blurring of thin lines.
  • Poisson vs. Gaussian: The paper demonstrates that using the correct statistical model for noise (Poisson MAP) is non-negotiable for low-light or photon-limited scenarios. Using Gaussian-based models on Poisson noise leads to significant artifacts.
  • Mathematical Rigor: Unlike many papers that propose nonconvex models and hope for the best, the authors provide a formal proof for the existence of a global minimizer and a convergence analysis for their ADMM scheme.

Key Results

The proposed AITV-SaT/SLaT method was compared against several heavy hitters, including the original TV-SaT, T-ROF, and the Potts model.

  • Accuracy: On the DRIVE dataset (vessels), AITV achieved the highest DICE index (a measure of overlap with ground truth), particularly in cases with high blur and noise.
  • Detail Preservation: Qualitatively, the method was significantly better at capturing thin vessels that other methods ignored or disconnected.
  • Multiphase Performance: In brain imaging tests, it outperformed others in segmenting Grey Matter (GM) and White Matter (WM), achieving higher PSNR (Peak Signal-to-Noise Ratio) values.
  • Efficiency: While slightly slower than the simplest TV models due to the nonconvex calculations, it remained orders of magnitude faster than complex global solvers like Pock or Storath, making it practical for real-world use.

Limitations and Future Work

  • Parameter Sensitivity: The model relies on a few hyperparameters ($\lambda, \mu, \alpha$). While the authors provide a sensitivity analysis, these currently need to be tuned manually for different noise levels.
  • Convergence Assumptions: The convergence proof for the ADMM algorithm relies on some relatively strong assumptions (like the boundedness of Lagrange multipliers) that might not always hold in every edge case.
  • Future Directions: The authors suggest developing an adaptive $\alpha$ parameter that could automatically adjust the sparsity constraint for different parts of an image. They also aim to adapt the framework for other noise types, such as impulsive (salt-and-pepper) noise.

Summary updated

Abstract

In this paper, we aim to segment an image degraded by blur and Poisson noise. We adopt a smoothing-and-thresholding (SaT) segmentation framework that finds a piecewise-smooth solution, followed by $k$-means clustering to segment the image. Specifically for the image smoothing step, we replace the least-squares fidelity for Gaussian noise in the Mumford-Shah model with a maximum posterior (MAP) term to deal with Poisson noise and we incorporate the weighted difference of anisotropic and isotropic total variation (AITV) as a regularization to promote the sparsity of image gradients. For such a nonconvex model, we develop a specific splitting scheme and utilize a proximal operator to apply the alternating direction method of multipliers (ADMM). Convergence analysis is provided to validate the efficacy of the ADMM scheme. Numerical experiments on various segmentation scenarios (grayscale/color and multiphase) showcase that our proposed method outperforms a number of segmentation methods, including the original SaT.
BibTeX
@article{bui2023difference,
  author = {Kevin Bui and Yifei Lou and Fredrick Park and Jack Xin},
  title = {{Difference of Anisotropic and Isotropic TV for Segmentation under Blur and Poisson Noise}},
  journal = {Frontiers of Computer Science},
  year = {2023},
  doi = {10.3389/fcomp.2023.1131317},
}
Figure