Abstract
In this study, we propose a new activation function, called Adaptive Smooth Activation Unit (ASAU), tailored for optimized gradient propagation, thereby enhancing the proficiency of convolutional networks in medical image analysis. We apply this new activation function to two important and commonly used general tasks in medical image analysis: automatic disease diagnosis and organ segmentation in CT and MRI. Our rigorous evaluation on the RadImageNet abdominal/pelvis (CT and MRI) dataset and Liver Tumor Segmentation Benchmark (LiTS) 2017 demonstrates that our ASAU-integrated frameworks not only achieve a substantial (4.80%) improvement over ReLU in classification accuracy (disease detection) on abdominal CT and MRI but also achieves 1%-3% improvement in dice coefficient compared to widely used activations for 'healthy liver tissue' segmentation. These improvements offer new baselines for developing a diagnostic tool, particularly for complex, challenging pathologies. The superior performance and adaptability of ASAU highlight its potential for integration into a wide range of image classification and segmentation tasks.
Introduction
In the domain of deep learning, activation functions serve as the critical gatekeepers of information, determining which signals are propagated through a neural network. While the Rectified Linear Unit (ReLU) has long been the industry standard due to its simplicity and computational efficiency, its inherent limitations—specifically its non-differentiability at the origin and the potential for "dead neurons"—can hinder performance in high-stakes fields like medical imaging.
Medical image analysis requires the detection of subtle tissue variations and precise anatomical boundaries. The "sharp edges" or kinks in traditional activations like ReLU can lead to information loss in negative input regions and suboptimal gradient flow. To address these challenges, researchers at the Machine & Hybrid Intelligence Lab at Northwestern University have introduced the Adaptive Smooth Activation Unit (ASAU), a novel, differentiable activation function designed to improve gradient propagation and model adaptability.
The Adaptive Smooth Activation Unit (ASAU)
ASAU is a generalized, smooth activation function that can adaptively approximate several traditional functions, including ReLU, Leaky ReLU, and Maxout. By utilizing a combination of hyperbolic tangent ($\tanh$) and SoftPlus functions, ASAU creates a smooth transition across the origin, ensuring the function is differentiable everywhere.
Key technical advantages of ASAU include:
- Differentiability: Unlike ReLU, ASAU is differentiable at all points. This allows for more refined gradient updates during backpropagation, which is essential for learning the intricate features found in CT and MRI scans.
- Learnable Parameters: By tuning internal parameters ($\alpha$ and $\beta$), ASAU can dynamically adjust its shape during training to best fit the specific requirements of the dataset and architecture.
- Seamless Integration: ASAU can be integrated into existing state-of-the-art architectures, such as ResNet and UNet, by simply replacing the standard activation layers.
Experimental Evaluation
The efficacy of ASAU was validated using two large-scale medical benchmarks:
- RadImageNet: A comprehensive dataset for medical transfer learning. The study focused on abdominal and pelvis scans, encompassing approximately 140,000 CT slices across 28 disease classes and 91,000 MRI slices across 26 classes.
- LiTS (Liver Tumor Segmentation Benchmark) 2017: A multi-center dataset of 201 CT scans used to evaluate the precision of organ boundary delineation and lesion detection.
Results and Discussion
The quantitative results demonstrate that ASAU consistently outperforms standard activation functions across both classification and segmentation tasks.
Disease Diagnosis (Classification)
On the RadImageNet CT dataset, integrating ASAU into a ResNet-50 architecture resulted in a significant performance boost. The model achieved an accuracy of 72.47%, compared to 67.67% for the standard ReLU-based model. Furthermore, the Matthews Correlation Coefficient (MCC)—a robust metric for evaluating classification quality—rose from 52.84 to 61.10, indicating a more reliable diagnostic capability.
Organ Segmentation
In liver segmentation tasks, ASAU improved the Mean Dice Similarity Coefficient (mDSC) by 1% to 3% across various models, including UNet and DoubleUNet. In clinical practice, these incremental gains in Dice scores translate to more precise surgical planning and more accurate radiation therapy targeting.
Conclusion and Future Work
The Adaptive Smooth Activation Unit represents a significant step forward in optimizing neural networks for medical radiology. By replacing rigid thresholds with smooth, learnable curves, ASAU enables models to better capture the subtle nuances of human anatomy and pathology.
Future research will explore the application of ASAU in true 3D volumetric processing and expand the scope of segmentation to a broader suite of abdominal organs, including the kidneys, pancreas, and spleen. While the computational complexity of smooth functions is slightly higher than ReLU, the substantial gains in diagnostic accuracy and segmentation precision suggest that ASAU is a powerful tool for the next generation of medical AI.
Reference: Biswas, K., Jha, D., et al. (2023). Adaptive Smooth Activation for Improved Disease Diagnosis and Organ Segmentation from Radiology Scans. arXiv:2312.11480v1.
Summary updated