Morphology on categorical distributions

Silas Nyboe Ørting, Hans Jacob Teglbjærg Stephensen, Jon Sporring

Abstract

Mathematical morphology is a cornerstone of image processing, yet its application to probabilistic multi-class outputs is hindered by the lack of a natural mathematical order among categories. This paper introduces a novel framework for applying morphological operations directly to categorical probability distributions. By targeting specific classes and proportionally rescaling others, the proposed method enables nuanced, uncertainty-aware post-processing. Furthermore, "protected" operations are introduced to leave specified classes untouched, providing a robust toolset for cleaning neural network predictions in complex domains like medical imaging.

Introduction

Deep learning models for image segmentation typically produce a probability distribution over multiple classes for each pixel. While these raw predictions are powerful, they often require post-processing to remove noise, close holes, or smooth boundaries. Mathematical Morphology (MM) is the standard tool for such tasks in grayscale and binary images. However, extending MM to categorical data is non-trivial because categories like "background," "cell membrane," and "mitochondria" do not possess the inherent ordering required by the complete lattices upon which classic MM is built.

The Challenge of Categorical Data

Traditional MM relies on a strict ordering of pixel values. Previous attempts to adapt MM for multi-class images often forced an arbitrary ranking, thresholded the image (losing probabilistic information), or failed to maintain idempotence—a key property where repeated application of an operation yields no further change. The authors address these shortcomings by defining mathematically sound morphological operations that operate directly on unordered probability distributions.

Proposed Methodology: Direct Categorical Morphology

The core contribution of this work is a method that targets a single category for dilation or erosion. Instead of attempting to order all classes, the operation is applied to the probability of the target class. To maintain a valid probability distribution (summing to 1.0), the probabilities of all non-target classes are rescaled proportionally. This approach elegantly preserves the relative conditional probabilities of the remaining classes, maintaining the model's original uncertainty.

Illustration of categorical morphological operations using a structuring element.

Protected Operations

A significant advancement is the introduction of "protected" categories. In many real-world scenarios, one might wish to dilate a specific class (e.g., a tumor core) without encroaching upon another critical class (e.g., healthy tissue). By utilizing the Fast Marching Method (FMM) to calculate geodesic distances around protected regions, the algorithm ensures that morphological operations do not overwrite or "bleed" into specified classes.

Demonstration of protected morphological operations on a grid with multiple categories.

Applications and Results

The utility of these methods was demonstrated on two challenging datasets:

  • Hippocampus Electron Microscopy (EM): Categorical "opening" operations successfully removed false-positive mitochondria predictions that bridged unrelated structures, restoring correct cytosol separation without retraining the underlying network.
  • QUBIQ Brain Tumor Dataset (MRI): Protected dilations were used to model inter-rater annotator bias. By dilating the active tumor core while protecting the edema and background, the authors generated realistic alternative ground-truths that reflect the variability seen in human expert labeling.

Conclusion

This framework provides a mathematically rigorous way to apply morphological filters to probabilistic categorical data. By avoiding arbitrary rankings and preserving uncertainty through rescaling, it offers a versatile tool for post-processing neural network outputs. Future work will explore more complex operations, such as morphological gradients, and the integration of these techniques into generative sampling processes to produce smoother, more realistic discrete segmentation masks.

Summary updated

BibTeX
@article{rting2023morphology,
  author = {Silas Nyboe Ørting and Hans Jacob Teglbjærg Stephensen and Jon Sporring},
  title = {{Morphology on categorical distributions}},
  journal = {J. Math. Imaging Vis.},
  year = {2023},
  doi = {10.1007/s10851-023-01146-x},
}
Figure