SMOGAN: Synthetic Minority Oversampling with GAN Refinement for Imbalanced Regression

Shayan Alahyari, Mike Domaratzki

Introduction

In many real-world regression tasks—such as predicting air quality spikes, rare medical outcomes, or extreme financial shifts—the data is heavily skewed. Most observations fall into a "normal" range, while the most critical values (the "minority" samples) are few and far between. Standard machine learning models are biased toward these dense, majority regions, often failing to predict rare extremes accurately.

While techniques like SMOTE have been adapted for regression (e.g., SMOGN), they often rely on simple linear interpolation or Gaussian noise. In complex, non-linear datasets, these methods can produce "unrealistic" synthetic samples that do not follow the true relationship between features and targets, potentially degrading model performance.

The SMOGAN Framework

To address these limitations, the authors propose SMOGAN, a novel two-stage framework that combines traditional oversampling with generative adversarial networks (GANs). Instead of asking a GAN to generate minority samples from scratch—a difficult task when data is scarce—SMOGAN uses the GAN as a "polishing" layer for existing oversampling techniques.

The SMOGAN architecture: A two-stage process involving initial synthetic generation followed by adversarial refinement using a Critic and Generator.

Stage 1: Initial Synthetic Generation

A base oversampler (such as SMOGN) identifies sparse target regions and generates an initial pool of synthetic candidates. This stage provides the raw material for the subsequent refinement.

Stage 2: Adversarial Refinement (DistGAN)

The core innovation lies in the distribution-aware GAN (DistGAN) used to refine the initial samples:

  • WGAN-GP: The framework utilizes a Wasserstein GAN with Gradient Penalty to ensure stable training and avoid mode collapse.
  • MMD Regularization: The generator’s loss is augmented with Maximum Mean Discrepancy (MMD). This forces synthetic samples to match the statistical moments and distribution shape of the real minority samples.
  • Filtering: The discriminator (or "critic") is trained exclusively on real minority samples, pushing the generator to refine synthetic points until they are indistinguishable from real data.

Experimental Evaluation

The researchers evaluated SMOGAN on 23 imbalanced regression datasets spanning domains such as housing prices, fuel consumption, and medical data. The framework's modular design allows users to swap the Stage 1 oversampler, making it highly adaptable.

Statistical and Predictive Results

SMOGAN achieved a 72% reduction in the discrepancy of feature-target correlation matrices compared to SMOGN and a 90% improvement in matching the mean of univariate statistics. Across all datasets, SMOGAN consistently outperformed state-of-the-art competitors (like WSMOTER and G-SMOTE) using both TabNet and Random Forest regressors. The method showed particularly strong gains in SERA (Squared Error Relevance Area) and F-measure for regression, metrics specifically designed to penalize errors on rare target values.

Visualizing Synthetic Quality

Qualitative analysis using PCA reveals the impact of the refinement stage. While traditional methods often create synthetic data that forms distinct "blobs" or linear artifacts, SMOGAN-refined samples intermix naturally with real minority points, respecting the underlying joint feature-target distribution.

PCA visualization (PC1 vs PC2) comparing real minority samples, standard SMOGN synthetic samples, and SMOGAN refined samples.

Conclusion and Future Work

SMOGAN demonstrates that adversarial refinement can significantly improve the fidelity of synthetic data in imbalanced regression. While the method introduces some computational overhead and requires GAN hyperparameter tuning, the gains in predictive accuracy on high-impact rare values are substantial. Future research will focus on automated fine-tuning strategies for relevance thresholds and extending the framework to handle high-dimensional multimodal datasets.

Summary updated

BibTeX
@article{alahyari2026smogan,
  author = {Shayan Alahyari and Mike Domaratzki},
  title = {{SMOGAN: Synthetic Minority Oversampling with GAN Refinement for Imbalanced Regression}},
  journal = {Expert Systems with Applications},
  year = {2026},
  doi = {10.1016/j.eswa.2026.133032},
}
Figure