Neural Annotation Refinement: Development of a New 3D Dataset for Adrenal Gland Analysis

Jiancheng Yang, Rui Shi, Udaranga Wickramasinghe, Qikui Zhu, Bingbing Ni, Pascal Fua

Sure, here's a blog-style summary of the paper "Neural Annotation Refinement: Development of a New 3D Dataset for Adrenal Gland Analysis":

TL;DR

This paper introduces Neural Annotation Refinement (NeAR), a novel method to automatically correct imperfect human annotations in medical imaging datasets. By leveraging appearance-aware implicit neural representations, NeAR significantly improves the quality of annotations, leading to better performance in downstream tasks like adrenal gland diagnosis. The authors also introduce the ALAN dataset, a new benchmark for adrenal gland analysis featuring 1,584 3D models with refined annotations.

What Problem Does This Paper Address?

Medical image analysis heavily relies on accurate annotations to train deep learning models. However, obtaining high-quality annotations is a significant challenge due to the time-consuming and expensive nature of expert annotation, especially for 3D medical data. This often leads to imperfect annotations with artefacts, false positives, and false negatives. While multi-expert consensus is considered the gold standard, it's often impractical. This paper addresses the critical need for an automated method to refine these imperfect human annotations, making them more reliable for training robust medical imaging models.

Methods

The core of the paper is the Neural Annotation Refinement (NeAR) method. NeAR builds upon the concept of deep implicit surface modeling, which uses neural networks to represent 3D shapes as continuous functions.

Here's a breakdown of the key techniques:

  • Implicit Surface Modeling: Standard implicit models learn a mapping from spatial coordinates to a shape representation (e.g., occupancy probability). This acts as a "deep prior" that can smooth out noisy annotations.
  • Appearance-Awareness: A crucial innovation is making the implicit model "appearance-aware." NeAR integrates the image appearance (e.g., Hounsfield Units in CT scans) at a query point as an input to the implicit function. This helps the model align the repaired surface with the actual anatomical boundaries, fixing misalignments caused by shape-only models.
  • Convolutional Decoder with Multi-Scale Features: To address the data-scarcity typical in medical imaging, NeAR employs a convolutional decoder that generates multi-scale feature maps. These features are then aggregated to represent the shape, making the model more data-efficient than purely MLP-based approaches.
  • Auto-Decoding: The training utilizes an auto-decoding approach, where a learnable latent vector for each shape is optimized directly, avoiding the need for an explicit encoder.

Datasets

The paper uses and introduces the following datasets:

  • AbdomenCT-1K: A public abdominal CT organ segmentation dataset. The authors use the adrenal gland subset (100 cases) to synthesize distorted "golden standard" annotations for evaluating NeAR's repair quality.
  • ALAN (Adrenal gLand ANalysis) Dataset: A new, large-scale dataset introduced by the authors. It comprises 1,584 3D adrenal gland models derived from CT scans. Each case includes:
  • A segmentation mask (annotated by a single clinician, hence imperfect).
  • A diagnosis label (normal vs. abnormal) assigned by two clinicians and confirmed by a senior endocrinologist (considered reliable).
  • NeAR-repaired shapes of the adrenal glands.

The ALAN dataset is intended to be open-source and serve as a benchmark for medical shape analysis and classification tasks.

Key Insights and Ideas

  • Appearance is Key for Accurate Refinement: The most significant insight is that incorporating image appearance into the implicit surface model dramatically improves annotation repair quality. Shape-only models can smooth out noise but may misalign with actual anatomy, whereas appearance-aware models achieve more precise boundary adherence.
  • Implicit Surfaces as Powerful Priors: Deep implicit surfaces act as strong priors that can effectively denoise and regularize segmentation masks, going beyond simple morphological operations.
  • Data Efficiency in Medical Imaging: The use of convolutional decoders with multi-scale features is a practical solution for training implicit models on limited medical datasets.
  • Improved Downstream Performance: The paper demonstrates that repairing annotations with NeAR not only improves the visual quality of the segmentation but also leads to better performance in subsequent tasks, such as classifying adrenal glands as normal or abnormal.

Key Results

  • Segmentation Repairing on Distorted Golden Standards:
    • NeAR (with appearance) achieved the highest Dice Similarity Coefficient (DSC) of 81.07% and Normalized Surface Dice (NSD) of 91.22% on the synthesized distorted dataset, outperforming standard segmentation counterparts (Seg-FCN, Seg-UNet) and the shape-only NeAR.
  • Qualitative results showed NeAR (S+A) repairing distorted annotations more accurately and fixing issues that other methods missed.
  • Adrenal Diagnosis on the Repaired ALAN Dataset:
    • Models trained on NeAR-repaired shapes achieved significantly better diagnostic performance (measured by Area Under the ROC Curve - AUC) compared to models trained on human-annotated imperfect shapes or shapes repaired by standard segmentation methods.
  • For example, ResNet-18 (ACS) trained on NeAR (S+A) repaired shapes achieved an AUC of 91.11%, outperforming human-annotated shapes (90.10%) and shapes repaired by Seg-UNet (86.83%).

Limitations and Future Work

  • Limited to Adrenal Glands: The current study and dataset are focused specifically on adrenal glands.
  • Future Work: The authors suggest extending NeAR to handle sparse annotations and small objects in future research. They also plan to test the method on other anatomical structures.

Summary updated

Abstract

The human annotations are imperfect, especially when produced by junior practitioners. Multi-expert consensus is usually regarded as golden standard, while this annotation protocol is too expensive to implement in many real-world projects. In this study, we propose a method to refine human annotation, named Neural Annotation Refinement (NeAR). It is based on a learnable implicit function, which decodes a latent vector into represented shape. By integrating the appearance as an input of implicit functions, the appearance-aware NeAR fixes the annotation artefacts. Our method is demonstrated on the application of adrenal gland analysis. We first show that the NeAR can repair distorted golden standards on a public adrenal gland segmentation dataset. Besides, we develop a new Adrenal gLand ANalysis (ALAN) dataset with the proposed NeAR, where each case consists of a 3D shape of adrenal gland and its diagnosis label (normal vs. abnormal) assigned by experts. We show that models trained on the shapes repaired by the NeAR can diagnose adrenal glands better than the original ones. The ALAN dataset will be open-source, with 1,594 shapes for adrenal gland diagnosis, which serves as a new benchmark for medical shape analysis. Code and dataset are available at https://github.com/M3DV/NeAR .
BibTeX
@inproceedings{yang2022neural,
  author = {Jiancheng Yang and Rui Shi and Udaranga Wickramasinghe and Qikui Zhu and Bingbing Ni and Pascal Fua},
  title = {{Neural Annotation Refinement: Development of a New 3D Dataset for Adrenal Gland Analysis}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2022},
  url = {https://openreview.net/forum?id=MSzUtHTMW5},
}
Figure