GLOWin: A Flow-based Invertible Generative Framework for Learning Disentangled Feature Representations in Medical Images

Aadhithya Sankar, Matthias Keicher, Rami Eisawy, Abhijeet Parida, Franz Pfister, Seong-Tae Kim, Nassir NavabarXiv 2021 · arXiv

Abstract

GLOWin is a novel flow-based generative framework designed to achieve disentangled representations in medical imaging. By adapting Factor Loss to spatial embeddings within an end-to-end invertible architecture, GLOWin separates independent features—such as anatomical shape and pathological anomalies—without the fidelity loss common in VAE-based approaches. Evaluated on brain MRI data, the model demonstrates superior performance in downstream tasks like anomaly classification and anatomical regression while maintaining high generative quality.

Introduction

Deep learning in medical imaging often operates as a "black box," hindering clinical trust and interpretability. Furthermore, medical datasets frequently suffer from class imbalances, such as a scarcity of rare pathologies compared to healthy scans. Disentangled representation learning seeks to address these issues by ensuring that individual units in a model's latent space correspond to specific physical factors, like tumor size or organ shape.

While Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) have been employed for this task, they face significant hurdles: VAEs often produce blurry reconstructions, and GANs are notoriously difficult to invert (mapping an image back to its latent code). GLOWin provides a high-fidelity, fully invertible, and end-to-end solution that works directly on raw images to isolate semantic features.

Methodology

GLOWin builds upon the Glow architecture, a type of "Normalizing Flow" model. Unlike standard neural networks, flow models utilize a series of invertible transformations, allowing them to map an image to a latent space and back with zero information loss.

The GLOWin architecture featuring multi-scale flow blocks and the Spatial Factor Loss for disentanglement.

The framework introduces three key innovations:

  1. Spatial Factor Loss: The authors adapted the standard Factor Loss to work with 3D spatial feature maps ($C \times H \times W$) produced by flow models.
  2. Latent Factorization: The final latent space is partitioned into $M+1$ components. For instance, specific dimensions are dedicated to anomalies, others to slice indices (representing brain shape), and the remainder to residual features.
  3. Pairwise Training: The model is trained using pairs of images sharing a semantic concept (e.g., two different slices containing tumors). By enforcing high correlation between the "anomaly" factors of these pairs while keeping other factors uncorrelated, the model learns to isolate specific features effectively.

Experimental Setup

The researchers evaluated GLOWin using the BraTS 2019 (Multimodal Brain Tumor Segmentation Challenge) dataset, consisting of 335 co-registered, skull-stripped Brain MRI scans (FLAIR modality). The primary task was to disentangle two generative factors: the presence of anomalies (tumors) and the slice index (a proxy for anatomical brain shape at different depths).

Results and Discussion

GLOWin achieved a bits-per-dimension (BPD) score of 2.602, nearly identical to the baseline Glow's 2.588. This confirms that enforcing disentanglement does not degrade the model's ability to capture the data distribution.

In downstream tasks, GLOWin demonstrated significant improvements:

  • Anomaly Classification: GLOWin achieved 77.23% accuracy, outperforming both the baseline Glow (77.04%) and the previous state-of-the-art IIN model (70.62%).
  • Slice Index Regression: The model achieved a Mean Absolute Error (MAE) of 0.048, significantly better than Glow (0.065) and IIN (0.066).
Latent space interpolation demonstrating independent control over anatomical shape and pathological features.

Qualitatively, the model's invertibility allows for precise image manipulation. As shown in the interpolation results, the model can smoothly transition the shape of a brain while keeping a tumor fixed in place, or vice versa, proving the factors are truly independent. This capability enables the generation of realistic, synthetic anomalies within a specific patient's healthy scan.

Conclusion and Future Work

GLOWin demonstrates that flow-based models can achieve high-quality disentanglement without the need for separate pre-trained encoders. However, challenges remain regarding memory constraints, as flow models are dimension-preserving and intensive to train on high-resolution images. Future work will focus on multi-scale disentanglement—enforcing feature separation at earlier layers—and applying the framework to a wider variety of medical imaging modalities for data augmentation and rare-case synthesis.

Summary updated

Abstract

Disentangled representations can be useful in many downstream tasks, help to make deep learning models more interpretable, and allow for control over features of synthetically generated images that can be useful in training other models that require a large number of labelled or unlabelled data. Recently, flow-based generative models have been proposed to generate realistic images by directly modeling the data distribution with invertible functions. In this work, we propose a new flow-based generative model framework, named GLOWin, that is end-to-end invertible and able to learn disentangled representations. Feature disentanglement is achieved by factorizing the latent space into components such that each component learns the representation for one generative factor. Comprehensive experiments have been conducted to evaluate the proposed method on a public brain tumor MR dataset. Quantitative and qualitative results suggest that the proposed method is effective in disentangling the features from complex medical images.
BibTeX
@misc{sankar2021glowin,
  author = {Aadhithya Sankar and Matthias Keicher and Rami Eisawy and Abhijeet Parida and Franz Pfister and Seong-Tae Kim and Nassir Navab},
  title = {{GLOWin: A Flow-based Invertible Generative Framework for Learning Disentangled Feature Representations in Medical Images}},
  year = {2021},
  eprint = {2103.10868},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2103.10868},
}
Figure