Introducing the GEV Activation Function for Highly Unbalanced Data to Develop COVID-19 Diagnostic Models

Joshua Bridge, Yanda Meng, Yitian Zhao, Yong Du, Mingfeng Zhao, Renrong Sun, Yalin ZhengIEEE JBHI 2020 · OpenReview

Abstract

Fast and accurate diagnosis is essential for the efficient and effective control of the COVID-19 pandemic. Despite the prevalence of the outbreak, relatively few diagnostic images are openly available to develop automatic diagnosis algorithms. Traditional deep learning methods often struggle when data is highly unbalanced, with many cases in one class and only a few in another. We propose a novel activation function based on the generalized extreme value (GEV) distribution from extreme value theory, which improves performance over the traditional sigmoid activation function when one class significantly outweighs the other. We demonstrate the proposed activation function on a publicly available dataset and externally validate on a dataset consisting of 1,909 healthy chest X-rays and 84 COVID-19 X-rays. The proposed method achieves an improved area under the receiver operating characteristic (DeLong's p-value < 0.05) compared to the sigmoid activation. Our method is also demonstrated on a dataset of healthy and pneumonia vs. COVID-19 X-rays and a set of computerized tomography images, achieving improved sensitivity. The proposed GEV activation function significantly improves upon the previously used sigmoid activation for binary classification, offering a new paradigm for fighting diseases with limited training data.

The Challenge of Unbalanced Medical Data

Deep learning models typically require large, balanced datasets to achieve high accuracy. However, in medical imaging—especially during the early stages of a pandemic—datasets are often highly unbalanced. A typical scenario might involve thousands of "healthy" images but only a few dozen "positive" cases.

Standard diagnostic models rely on the sigmoid activation function for binary classification. The sigmoid function assumes a symmetric logistic distribution of errors. When data is heavily skewed, the rare class (the disease) effectively lives in the "tail" of the distribution. Sigmoid functions struggle to model these extreme tails accurately, often leading to models biased toward the majority class, resulting in dangerously high false-negative rates.

Methodology: The GEV Activation Function

To address this "needle in a haystack" problem, researchers introduced the GEV Activation Function, derived from Extreme Value Theory (EVT). EVT is a statistical branch designed specifically to model rare, extreme events such as natural disasters or financial crashes.

Mathematical Innovation

Unlike the fixed, symmetric sigmoid curve, the GEV function offers mathematical flexibility through three trainable parameters:

  • $\xi$ (shape): Allows the network to learn the specific skewness of the data.
  • $\mu$ (location): Adjusts the center of the distribution.
  • $\sigma$ (scale): Adjusts the spread.

Depending on the learned value of $\xi$, the function can adapt to take the form of Gumbel, Fréchet, or Weibull distributions, making it uniquely suited for unbalanced datasets.

The proposed diagnostic pipeline integrates Inception-V3 for feature extraction with a custom GEV activation layer to handle data imbalance.

Integration into CNNs

The GEV function is implemented as a custom layer at the end of a Convolutional Neural Network (CNN). In this study, the authors utilized Inception V3 and MobileNet architectures. As shown in the pipeline above, the CNN extracts features from X-ray or CT images, which are then processed by the GEV activation function to produce a final diagnosis.

Experimental Results

The GEV activation was rigorously tested against the standard sigmoid function across multiple datasets:

  1. Controlled Imbalance (Toy Dataset): At a 1:50 imbalance ratio (Pneumonia vs. Healthy), GEV achieved an AUC of 0.941 and sensitivity of 0.828, while the sigmoid function's performance plummeted to an AUC of 0.609 and sensitivity of 0.419.
  2. COVID-19 X-rays: On a dataset of 1,909 healthy and 84 COVID-19 images, GEV reached an AUC of 0.820, significantly outperforming the sigmoid's 0.750 (DeLong’s p-value < 0.05).
  3. CT Scans: In a striking test on CT data, the sigmoid model failed completely, classifying every image as non-COVID (0% sensitivity). The GEV model, however, achieved a sensitivity of 0.628, demonstrating its ability to distinguish rare positive cases.

Discussion and Future Directions

The core insight of this research is that rare disease detection is mathematically analogous to predicting extreme environmental events. By using a distribution built for "extremes," the model becomes more robust without the need for artificial data inflation (oversampling), which can lead to overfitting.

While the results are promising, future work will focus on extending the GEV implementation to multiclass scenarios—such as distinguishing between COVID-19, bacterial pneumonia, and healthy lungs—and validating the models on raw, uncurated clinical data to ensure real-world reliability. This new paradigm is expected to play a significant role in the fight against COVID-19 and other rare diseases where training data is scarce.

Summary updated

Abstract

Fast and accurate diagnosis is essential for the efficient and effective control of the COVID-19 pandemic that is currently disrupting the whole world. Despite the prevalence of the COVID-19 outbreak, relatively few diagnostic images are openly available to develop automatic diagnosis algorithms. Traditional deep learning methods often struggle when data is highly unbalanced with many cases in one class and only a few cases in another; new methods must be developed to overcome this challenge. We propose a novel activation function based on the generalized extreme value (GEV) distribution from extreme value theory, which improves performance over the traditional sigmoid activation function when one class significantly outweighs the other. We demonstrate the proposed activation function on a publicly available dataset and externally validate on a dataset consisting of 1,909 healthy chest X-rays and 84 COVID-19 X-rays. The proposed method achieves an improved area under the receiver operating characteristic (DeLong's p-value <; 0.05) compared to the sigmoid activation. Our method is also demonstrated on a dataset of healthy and pneumonia vs. COVID-19 X-rays and a set of computerized tomography images, achieving improved sensitivity. The proposed GEV activation function significantly improves upon the previously used sigmoid activation for binary classification. This new paradigm is expected to play a significant role in the fight against COVID-19 and other diseases, with relatively few training cases available.
BibTeX
@article{bridge2020introducing,
  author = {Joshua Bridge and Yanda Meng and Yitian Zhao and Yong Du and Mingfeng Zhao and Renrong Sun and Yalin Zheng},
  title = {{Introducing the GEV Activation Function for Highly Unbalanced Data to Develop COVID-19 Diagnostic Models}},
  journal = {IEEE J. Biomed. Health Informatics},
  year = {2020},
  url = {https://openreview.net/forum?id=vWvFr_Uki5M},
}
Figure