Difference of anisotropic and isotropic TV for segmentation under blur and Poisson noise

Kevin Bui, Yifei Lou, Fredrick Park, Jack Xin

TL;DR

Researchers have developed a new image segmentation framework specifically designed for images corrupted by blur and Poisson noise (common in medical and astronomical imaging). By combining a Poisson-specific data fidelity term with a nonconvex "AITV" regularizer, the method achieves significantly sharper edges and better detail preservation than standard Total Variation (TV) techniques.

What Problem Does This Paper Address?

Image segmentation—the process of partitioning an image into meaningful regions—is a cornerstone of computer vision. However, real-world images are rarely perfect. In fields like medical imaging (MRI/PET) or microscopy, images are often degraded by two major factors: blur (from lens optics or motion) and Poisson noise (inherent in photon-counting devices).

Most traditional models, like the classic Mumford-Shah model, are optimized for Gaussian noise. When applied to Poisson-corrupted data, these models often produce "blocky" artifacts or fail to capture fine, thin structures. The authors aim to bridge this gap by refining the "Smoothing-and-Thresholding" (SaT) framework to handle these specific degradations while maintaining computational efficiency.

Methods

The authors propose a two-stage approach:

  1. The Smoothing Step: Instead of a standard least-squares approach, they solve a variational problem using a Maximum A Posteriori (MAP) fidelity term tailored for Poisson noise.
  2. The Innovation: They replace standard Total Variation (TV) with AITV (Weighted Difference of Anisotropic and Isotropic Total Variation). While standard TV uses the $\ell_1$ norm of image gradients, AITV subtracts a weighted $\ell_2$ norm ($\ell_1 - \alpha\ell_2$). This nonconvex regularizer is much better at promoting "sparsity" in the gradients, which translates to sharper, more distinct edges.
  3. The Solver: To handle the nonconvexity, they developed a custom ADMM (Alternating Direction Method of Multipliers) algorithm. This includes a specific "proximal operator" that allows the math to be solved efficiently in the frequency domain using Fast Fourier Transforms (FFTs).
  4. The Thresholding Step: Once the image is smoothed and the noise/blur are reduced, they apply k-means clustering to finalize the segmentation. For color images, they use a "lifting" technique, converting the image into the Lab color space to better separate color information before clustering.

Datasets

The method was evaluated on three distinct types of data:

  • DRIVE Dataset: Retinal vessel images used for binary segmentation. This is a high-stakes test because blood vessels are thin and easily lost in noise.
  • BrainWeb Dataset: Simulated MRI brain scans used for "multiphase" segmentation (separating cerebrospinal fluid, gray matter, and white matter).
  • PASCAL VOC 2010: A standard benchmark for natural color images, testing the algorithm's versatility across different scenes and objects.

Key Insights and Ideas

  • The Power of Nonconvexity: The "aha moment" here is that while convex models (like standard TV) are easier to solve, they often over-smooth images. By embracing the nonconvex AITV, the model can "punish" small gradients more effectively, leading to cleaner regions and sharper boundaries.
  • Statistical Alignment: By using a Poisson-specific log-likelihood term ($Au - f \log Au$) instead of a simple squared difference, the model respects the underlying physics of how the noisy image was created.
  • Efficiency Matters: Despite using a more complex nonconvex model, the authors' ADMM implementation remains nearly as fast as standard convex methods, making it practical for large datasets.

Key Results

  • Superior Detail: On the DRIVE dataset, the AITV-based method successfully segmented thin vessels that standard TV-based methods (like SaT and T-ROF) completely missed.
  • Quantitative Gains:
    • In binary segmentation (DRIVE), AITV SaT achieved the highest DICE indices (a measure of overlap with ground truth) across all noise levels.
  • In color segmentation (PASCAL VOC), it consistently outperformed competitors in PSNR (Peak Signal-to-Noise Ratio), indicating a much cleaner reconstruction of the underlying regions.
  • Speed: The proposed method was significantly faster (often by a factor of 5x to 10x) than other high-performance models like the Potts model or Pock’s algorithm.

Limitations and Future Work

  • Parameter Sensitivity: The model relies on three main parameters ($\lambda, \mu, \alpha$). While the authors provide a sensitivity analysis, these currently require manual tuning to get optimal results for different types of images.
  • Convergence Theory: While the authors proved "subsequential convergence" (meaning the algorithm eventually settles on a stable solution), proving global convergence for this specific nonconvex model remains an open theoretical challenge.
  • Adaptive Sparsity: The authors suggest that future versions of the algorithm could automatically determine the best value for $\alpha$ (the sparsity weight) based on the local characteristics of the image.

Summary updated

Abstract

In this paper, we aim to segment an image degraded by blur and Poisson noise. We adopt a smoothing-and-thresholding (SaT) segmentation framework that finds a piecewise-smooth solution, followed by $k$-means clustering to segment the image. Specifically for the image smoothing step, we replace the least-squares fidelity for Gaussian noise in the Mumford-Shah model with a maximum posterior (MAP) term to deal with Poisson noise and we incorporate the weighted difference of anisotropic and isotropic total variation (AITV) as a regularization to promote the sparsity of image gradients. For such a nonconvex model, we develop a specific splitting scheme and utilize a proximal operator to apply the alternating direction method of multipliers (ADMM). Convergence analysis is provided to validate the efficacy of the ADMM scheme. Numerical experiments on various segmentation scenarios (grayscale/color and multiphase) showcase that our proposed method outperforms a number of segmentation methods, including the original SaT.
BibTeX
@article{bui2023difference,
  author = {Kevin Bui and Yifei Lou and Fredrick Park and Jack Xin},
  title = {{Difference of anisotropic and isotropic TV for segmentation under blur and Poisson noise}},
  journal = {Frontiers of Computer Science},
  year = {2023},
  doi = {10.3389/fcomp.2023.1131317},
}
Figure