Directional latent space representation for medical image segmentation

Xintao Liu, Yan Gao, Changqing Zhan, Qiao Wangr, Yu Zhang, Yi He, Hongyan Quan

Abstract

Efficient medical image segmentation plays an important role in computer-aided diagnosis (CAD). Deep mining of pixel semantics is crucial for medical image segmentation. However, previous works on medical semantic segmentation usually overlook the importance of embedding subspace, and lacked the mining of latent space direction information. In this work, we constructed global orthogonal basis and channel orthogonal basis in the latent space, which can significantly enhance the feature representation. We propose a novel distance-based segmentation method that decouples the embedding space into sub-embedding spaces of different classes, and then implements pixel level classification based on the distance between its embedding features and the origin of the subspace. Experiments on various public medical image segmentation benchmarks show the effectiveness of our model as compared to the state-of-the-art methods. The code is available at https://github.com/lxt0525/LSDENet.

Introduction

Medical image segmentation is a fundamental task in computer-aided diagnosis, yet it remains challenging due to the complex shapes and ambiguous boundaries of anatomical structures. While architectures like U-Net have set a strong baseline, they often struggle with high intra-class variance and the subtle semantic nuances of medical data.

Current research typically overlooks two critical aspects of feature representation:

  1. Latent Directionality: Most models focus on feature magnitude or local dependencies, ignoring the rich semantic information encoded in the direction of vectors within the latent space.
  2. Prototype Limitations: Standard segmentation methods often rely on a single "prototype" or center for each class. In clinical settings, a single average representation cannot capture the diversity of real-world patient data, such as varying organ shapes across different demographics.

To address these gaps, we introduce LSDENet (Latent Space Directional Enhancement Network), which explicitly models latent directions and decouples the embedding space into class-specific subspaces.

Methods

LSDENet enhances standard encoder-decoder architectures with two primary "plug-and-play" modules: the Directional Latent Representation (DLR) module and the Decoupled Pixel Embedding Space (DPES) module.

Overview of the LSDENet architecture, highlighting the DLR and DPES modules.

Directional Latent Representation (DLR)

The DLR module utilizes two sets of orthogonal bases to represent directions in the latent space. Global Orthogonal Bases help the model learn inter-image differences, improving robustness across different scans. Channel Orthogonal Bases explore intra-image dependencies. By leveraging these bases, the model can "edit" latent representations more efficiently than high-complexity self-attention mechanisms, highlighting salient features without excessive computational overhead.

Decoupled Pixel Embedding Space (DPES)

Instead of a shared embedding space, the DPES module splits the space into $N$ distinct subspaces, where $N$ corresponds to the number of target classes. This decoupling prevents feature interference between different anatomical structures.

Distance-Based Segmentation Head

Departing from standard Softmax-based classification, LSDENet projects pixel embeddings into these class-specific subspaces. Classification is determined by the proximity of a pixel's embedding to the origin of a specific subspace. This "origin proximity" approach better handles intra-class variance compared to traditional prototype-based methods, as it does not rely on a single "average" representation for a class.

Experimental Results

We evaluated LSDENet on three diverse public benchmarks: Synapse (Multi-organ CT), ACDC (Cardiac MRI), and TN3k (Thyroid Ultrasound).

Performance on Synapse

On the Synapse multi-organ CT dataset, LSDENet achieved a mean Dice Coefficient of 86.94%, outperforming recent state-of-the-art models like nnFormer and TransUNet. The model showed particularly strong performance in segmenting the kidney (KL/KR) and liver.

Comparison of LSDENet against state-of-the-art methods on the Synapse multi-organ CT dataset.

Performance on ACDC and TN3k

  • ACDC: The model reached a Dice score of 93.11%, surpassing the previous best (FCT) at 93.02%.
  • TN3k: LSDENet showed a significant improvement on challenging ultrasound images, achieving a Dice score of 90.19% compared to AttnUNet's 87.94%.

Beyond accuracy, LSDENet's CNN-based design offers greater flexibility than Transformer-based counterparts, as it can process arbitrary input resolutions—a critical requirement for clinical workflows where image sizes vary.

Conclusion

LSDENet introduces a novel paradigm for medical image segmentation by focusing on latent space directionality and subspace decoupling. By moving away from single-prototype representations toward origin-based classification in decoupled spaces, the model achieves superior performance across multiple modalities. Future work will focus on refining the segmentation of subtle, minute anatomical targets that remain challenging for current automated systems.

Summary updated

BibTeX
@inproceedings{liu2024directional,
  author = {Xintao Liu and Yan Gao and Changqing Zhan and Qiao Wangr and Yu Zhang and Yi He and Hongyan Quan},
  title = {{Directional latent space representation for medical image segmentation}},
  booktitle = {Research Square},
  year = {2024},
  doi = {10.21203/rs.3.rs-4444261/v1},
}
Figure