A Knowledge Distillation Framework for Multi-Organ Segmentation of Medaka Fish in Tomographic Image

Jwalin Bhatt, Yaroslav Zharov, Sungho Suh, Tilo Baumbach, Vincent Heuveline, Paul Lukowicz

TL;DR

This paper introduces a novel self-training framework for segmenting multiple organs in Medaka fish CT scans, significantly reducing the need for extensive manual annotation. By leveraging knowledge distillation and a quality classifier to refine pseudo-labels, their method achieves superior segmentation performance with substantially less labeled data.

What Problem Does This Paper Address?

Creating detailed anatomical atlases of organisms, particularly from high-resolution 3D imaging like CT scans, is crucial for biological research. A key bottleneck in this process is accurate organ segmentation. While deep learning models excel at image segmentation, they typically require vast amounts of meticulously hand-annotated data, which is time-consuming and expensive to generate, especially for complex 3D structures. This paper tackles the challenge of achieving high-quality multi-organ segmentation with significantly reduced annotation effort, specifically for Medaka fish, a model organism in vertebrate research.

Methods

The proposed framework employs a three-pronged approach:

  1. Teacher-Student Model: A "Teacher" model, trained on a small set of labeled data, generates initial "pseudo-labels" for a larger pool of unlabeled data. A "Student" model is then trained using these pseudo-labels and the original labeled data.
  2. Quality Classifier: To mitigate the noise and inaccuracies inherent in pseudo-labels, a dedicated "Quality Classifier" is introduced. This classifier learns to distinguish between high-quality and low-quality pseudo-labels, allowing the Student model to focus on more reliable annotations.
  3. Pixel-wise Knowledge Distillation: To prevent the Student model from overfitting to the potentially noisy pseudo-labeled data and to retain valuable information from the Teacher's initial training, a pixel-wise knowledge distillation loss is applied. This loss encourages the Student to mimic the softened predictions of the Teacher model, effectively transferring "dark knowledge."

The Student model is trained with a combined loss function that includes segmentation loss (comparing predictions to ground truth) and the knowledge distillation loss, weighted by a hyperparameter. The Student model is also initialized with the Teacher's checkpoint to accelerate convergence.

Datasets

  • Medaka CT-scan dataset: This dataset consists of tomographic images of Medaka fish.
  • Labeled data: 30 samples were used for training the Teacher model. This was split into 75% for training and 25% for validation (23 volumes for training, 7 for validation).
  • Unlabeled data: 582 scans from three different experiments were used for generating pseudo-labels.
  • Quality Classifier training data: Approximately 1000 slices were manually labeled into "good" and "bad" categories to train the Quality Classifier.

Key Insights and Ideas

  • Pseudo-label refinement is critical: Simply using all generated pseudo-labels can be detrimental. A mechanism to filter or refine these labels (like the Quality Classifier) is essential for improving performance.
  • Knowledge distillation prevents overfitting: The pixel-wise knowledge distillation loss is key to preventing the Student model from memorizing noisy pseudo-labels and helps it generalize better.
  • Synergistic component improvement: The authors note that the combined improvements from their different components (pseudo-label filtering and knowledge distillation) are greater than the sum of their individual contributions, suggesting a synergistic effect.
  • Significant annotation reduction: The framework demonstrates that high-quality segmentation can be achieved with a fraction of the labeled data typically required for fully supervised methods.

Key Results

  • Improved mIoU and Dice scores: The proposed method achieved a mean Intersection over Union (mIoU) of 82.4% and a Dice score of 89.4% on the full dataset, outperforming the fully supervised baseline (74.8% mIoU, 82.2% Dice). This represents a 5.9% improvement in mIoU and a 3.4% improvement in Dice.
  • Reduced annotation burden: With only two labeled volumes, the proposed method achieved better results (72.5% mIoU) than the fully supervised method using 12 labeled volumes (60.1% mIoU). This highlights the framework's ability to achieve comparable or better performance with significantly less manual annotation.
  • Quality Classifier contribution: The Quality Classifier alone contributed a substantial improvement of +4.7% mIoU.

Limitations and Future Work

  • Quality Classifier data dependency: The authors note that the Quality Classifier's performance could benefit from more training data.
  • Future Work: The authors plan to extend their method by collecting more tomographic data on Medaka and other fish species. They also aim to evaluate and improve the framework on a wider variety of organisms, believing it can contribute to a better understanding of future-oriented genetics.

Summary updated

Abstract

Morphological atlases are an important tool in organismal studies, and modern high-throughput Computed Tomography (CT) facilities can produce hundreds of full-body high-resolution volumetric images of organisms. However, creating an atlas from these volumes requires accurate organ segmentation. In the last decade, machine learning approaches have achieved incredible results in image segmentation tasks, but they require large amounts of annotated data for training. In this paper, we propose a self-training framework for multi-organ segmentation in tomographic images of Medaka fish. We utilize the pseudo-labeled data from a pretrained Teacher model and adopt a Quality Classifier to refine the pseudo-labeled data. Then, we introduce a pixel-wise knowledge distillation method to prevent overfitting to the pseudo-labeled data and improve the segmentation performance. The experimental results demonstrate that our method improves mean Intersection over Union (IoU) by 5.9% on the full dataset and enables keeping the quality while using three times less markup.
BibTeX
@inproceedings{bhatt2023knowledge,
  author = {Jwalin Bhatt and Yaroslav Zharov and Sungho Suh and Tilo Baumbach and Vincent Heuveline and Paul Lukowicz},
  title = {{A Knowledge Distillation Framework for Multi-Organ Segmentation of Medaka Fish in Tomographic Image}},
  booktitle = {ISBI},
  year = {2023},
  url = {https://openreview.net/forum?id=3Gp54CFjug},
}
Figure