Abstract
Fast and accurate diagnosis is essential for the efficient and effective control of the COVID-19 pandemic. Despite the prevalence of the outbreak, relatively few diagnostic images are openly available to develop automatic diagnosis algorithms. Traditional deep learning methods often struggle when data is highly unbalanced, with many cases in one class and only a few in another. We propose a novel activation function based on the generalized extreme value (GEV) distribution from extreme value theory, which improves performance over the traditional sigmoid activation function when one class significantly outweighs the other. We demonstrate the proposed activation function on a publicly available dataset and externally validate on a dataset consisting of 1,909 healthy chest X-rays and 84 COVID-19 X-rays. The proposed method achieves an improved area under the receiver operating characteristic (DeLong's p-value < 0.05) compared to the sigmoid activation. Our method is also demonstrated on a dataset of healthy and pneumonia vs. COVID-19 X-rays and a set of computerized tomography images, achieving improved sensitivity. The proposed GEV activation function significantly improves upon the previously used sigmoid activation for binary classification, offering a new paradigm for fighting diseases with limited training data.
The Challenge of Unbalanced Medical Data
Deep learning models typically require large, balanced datasets to achieve high accuracy. However, in medical imaging—especially during the early stages of a pandemic—datasets are often highly unbalanced. A typical scenario might involve thousands of "healthy" images but only a few dozen "positive" cases.
Standard diagnostic models rely on the sigmoid activation function for binary classification. The sigmoid function assumes a symmetric logistic distribution of errors. When data is heavily skewed, the rare class (the disease) effectively lives in the "tail" of the distribution. Sigmoid functions struggle to model these extreme tails accurately, often leading to models biased toward the majority class, resulting in dangerously high false-negative rates.
Methodology: The GEV Activation Function
To address this "needle in a haystack" problem, researchers introduced the GEV Activation Function, derived from Extreme Value Theory (EVT). EVT is a statistical branch designed specifically to model rare, extreme events such as natural disasters or financial crashes.
Mathematical Innovation
Unlike the fixed, symmetric sigmoid curve, the GEV function offers mathematical flexibility through three trainable parameters:
- $\xi$ (shape): Allows the network to learn the specific skewness of the data.
- $\mu$ (location): Adjusts the center of the distribution.
- $\sigma$ (scale): Adjusts the spread.
Depending on the learned value of $\xi$, the function can adapt to take the form of Gumbel, Fréchet, or Weibull distributions, making it uniquely suited for unbalanced datasets.
Integration into CNNs
The GEV function is implemented as a custom layer at the end of a Convolutional Neural Network (CNN). In this study, the authors utilized Inception V3 and MobileNet architectures. As shown in the pipeline above, the CNN extracts features from X-ray or CT images, which are then processed by the GEV activation function to produce a final diagnosis.
Experimental Results
The GEV activation was rigorously tested against the standard sigmoid function across multiple datasets:
- Controlled Imbalance (Toy Dataset): At a 1:50 imbalance ratio (Pneumonia vs. Healthy), GEV achieved an AUC of 0.941 and sensitivity of 0.828, while the sigmoid function's performance plummeted to an AUC of 0.609 and sensitivity of 0.419.
- COVID-19 X-rays: On a dataset of 1,909 healthy and 84 COVID-19 images, GEV reached an AUC of 0.820, significantly outperforming the sigmoid's 0.750 (DeLong’s p-value < 0.05).
- CT Scans: In a striking test on CT data, the sigmoid model failed completely, classifying every image as non-COVID (0% sensitivity). The GEV model, however, achieved a sensitivity of 0.628, demonstrating its ability to distinguish rare positive cases.
Discussion and Future Directions
The core insight of this research is that rare disease detection is mathematically analogous to predicting extreme environmental events. By using a distribution built for "extremes," the model becomes more robust without the need for artificial data inflation (oversampling), which can lead to overfitting.
While the results are promising, future work will focus on extending the GEV implementation to multiclass scenarios—such as distinguishing between COVID-19, bacterial pneumonia, and healthy lungs—and validating the models on raw, uncurated clinical data to ensure real-world reliability. This new paradigm is expected to play a significant role in the fight against COVID-19 and other rare diseases where training data is scarce.
Summary updated