ElixirNet: Relation-Aware Network Architecture Adaptation for Medical Lesion Detection

Chenhan Jiang, Shaoju Wang, Xiaodan Liang, Hang Xu, Nong Xiao

Introduction

The application of deep learning to medical imaging has historically leaned on architectures optimized for natural image datasets like ImageNet or COCO. However, medical lesion detection in CT scans is fundamentally distinct from general object detection. Lesions often exhibit high visual similarity to surrounding healthy tissue, appear as extremely small targets, and occur within a context of massive class imbalance where background pixels vastly outnumber lesion pixels.

Comparison of object detection in natural images (left) versus medical CT scans (right), highlighting the domain gap.

Standard detectors like Feature Pyramid Networks (FPN) often struggle with these nuances. ElixirNet is proposed as a specialized framework to bridge this "domain gap" by customizing the detection head and leveraging relational context.

Methodology

ElixirNet enhances the standard two-stage detection pipeline with three novel modules designed for the specificities of medical data.

Truncated RPN (TRPN)

Traditional Region Proposal Networks (RPN) sample anchors uniformly, which in medical imaging leads to a high volume of false positives from background tissue. TRPN learns a probability distribution to identify likely lesion areas and "truncates" the search space by filtering out low-confidence regions. This sparsity improves both efficiency and precision.

Auto-lesion Block (ALB)

Rather than using a generic, hand-crafted RCNN head, ElixirNet employs Differentiable Neural Architecture Search (NAS) to discover an optimal block structure. The search space includes operations like dilated convolutions and non-local operators, allowing the model to automatically adapt to the scale and spatial relationships inherent in CT scans.

Relation Transfer Module (RTM)

Medical findings are often correlated (e.g., the presence of one lesion may increase the likelihood of another in a related organ). The RTM uses an interpretable graph to encode these semantic relationships. This allows the network to transfer knowledge from common, well-annotated lesions to rarer types, enhancing the model's ability to generalize across diverse pathologies.

The ElixirNet architecture featuring the Truncated RPN, Auto-lesion Block (ALB), and Relation Transfer Module (RTM).

Experimental Results

The model was rigorously evaluated on the DeepLesion dataset (32,000+ axial CT slices) and the Kits19 kidney tumor dataset.

  • Detection Performance: ElixirNet achieved a mean Average Precision (mAP) of 35.2% on DeepLesion, a significant jump from the 31.5% baseline set by standard FPN.
  • Sensitivity: At a threshold of 3 false positives per image, ElixirNet demonstrated a 2.3% improvement in sensitivity.
  • Efficiency: Despite the performance gains, the NAS-optimized architecture used one-third fewer parameters than the baseline models.
  • Generalization: The RTM provided a consistent 2% sensitivity boost across all lesion types, proving especially valuable for rare cases.

Discussion and Future Work

While ElixirNet is highly efficient in terms of parameters, the complex nature of the searched ALB blocks results in slightly higher inference latency (90ms vs 68ms). Future iterations could focus on hardware-aware NAS to optimize for real-time clinical deployment. Additionally, while the current "2.5D" approach (using three adjacent slices) is effective, extending the architecture to full 3D volumetric data remains a promising avenue for further improving detection accuracy.

Conclusion

ElixirNet demonstrates that specialized architecture adaptation is crucial for medical AI. By focusing on the detection head and incorporating relational context, it achieves superior performance and parameter efficiency, offering a robust framework for automated lesion detection in clinical settings.

Summary updated

Abstract

Most advances in medical lesion detection network are limited to subtle modification on the conventional detection network designed for natural images. However, there exists a vast domain gap between medical images and natural images where the medical image detection often suffers from several domain-specific challenges, such as high lesion/background similarity, dominant tiny lesions, and severe class imbalance. Is a hand-crafted detection network tailored for natural image undoubtedly good enough over a discrepant medical lesion domain? Is there more powerful operations, filters, and sub-networks that better fit the medical lesion detection problem to be discovered? In this paper, we introduce a novel ElixirNet that includes three components: 1) TruncatedRPN balances positive and negative data for false positive reduction; 2) Auto-lesion Block is automatically customized for medical images to incorporate relation-aware operations among region proposals, and leads to more suitable and efficient classification and localization. 3) Relation transfer module incorporates the semantic relationship and transfers the relevant contextual information with an interpretable the graph thus alleviates the problem of lack of annotations for all types of lesions. Experiments on DeepLesion and Kits19 prove the effectiveness of ElixirNet, achieving improvement of both sensitivity and precision over FPN with fewer parameters.
BibTeX
@inproceedings{jiang2020elixirnet,
  author = {Chenhan Jiang and Shaoju Wang and Xiaodan Liang and Hang Xu and Nong Xiao},
  title = {{ElixirNet: Relation-Aware Network Architecture Adaptation for Medical Lesion Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i07.6765},
}
Figure