Fairness- and uncertainty-aware data generation for data-driven design

Lijun Sun (孙立君), Jiarui Xie, Chonghui Zhang, Lijun Sun, Yaoyao Fiona ZhaoarXiv 2023 · DOI · arXiv

Introduction

In data-driven engineering design, the performance of machine learning models is fundamentally limited by the quality and distribution of the training data. Traditional approaches often rely on Design of Experiments (DOE) to sample the geometric design space (shape space) uniformly. However, due to the highly non-linear mapping between geometry and performance, uniform sampling in the shape space frequently results in "representation bias" within the property space. This leads to datasets where many designs exhibit similar performance characteristics, while significant "voids" or unexplored regions remain in the performance map.

Comparison between conventional data-driven design and the FairGen pipeline.

To address this, researchers at McGill University have introduced FairGen, an active data generation pipeline designed to identify and fill these performance gaps. Unlike existing methods that merely subsample large datasets, FairGen iteratively generates new, high-information data points to ensure a "fair" representation across the entire property space.

The FairGen Framework

FairGen operates as an iterative active learning loop comprising three core modules:

1. Coverage Module

To mathematically define "fairness" in data distribution, FairGen utilizes Voronoi diagrams. By treating existing property combinations as points in a continuous space, the system partitions the space into cells. The size of these cells indicates the "territory" covered by each point; larger cells highlight regions with sparse data. This geometric approach allows the framework to identify the most significant voids in the performance map.

Voronoi-based coverage metric used to identify data-sparse regions in the property space.

2. Uncertainty Module

Exploring unknown regions requires a balance between discovery and reliability. FairGen employs a Deep Ensemble of Mixture Density Networks (MDNs) to quantify predictive uncertainty. This ensemble approach estimates the model's confidence in generating a shape for a specific set of target properties. High uncertainty signals that the model lacks sufficient data to make a reliable prediction, guiding the system to expand its knowledge base incrementally.

3. Bayesian Optimization (BO)

The Bayesian Optimization module serves as the decision-maker, searching for new target properties that maximize coverage (exploration) while maintaining manageable predictive uncertainty (exploitation). Once a target is selected, the MDNs generate the corresponding geometry, which is then validated through high-fidelity physics simulations (e.g., Abaqus) and integrated back into the training set.

Experimental Validation

The framework was validated using S-slot auxetic metamaterials, structures known for their negative Poisson's ratio and fatigue resistance. Starting with an initial dataset of 1,000 designs from a standard grid search, FairGen generated 799 additional designs over 20 iterations. This was compared against traditional Grid Sampling and Latin Hypercube Sampling baselines with up to 3,000 samples.

Key Results and Insights

  • Enhanced Property Coverage: FairGen achieved a coverage score of 5.8, significantly outperforming traditional sampling methods which plateaued below 4.8. It expanded the known property region at approximately twice the rate of standard techniques.
  • Improved Model Accuracy: Generative models trained on FairGen data demonstrated superior performance. The Mean Absolute Error (MAE) was reduced by 1/3 compared to grid-sampled models and by nearly 1/8 compared to randomized sampling models.
  • Data Efficiency: The "information density" of FairGen-generated samples is notably higher. The study found that 799 FairGen samples provided equivalent model quality to over 930 samples generated via traditional geometric methods.

Conclusion and Future Directions

FairGen demonstrates that prioritizing "fairness" in the property space is more critical for inverse design than simple geometric diversity. By actively hunting for missing performance profiles, the pipeline creates more robust and accurate generative models.

Future work will focus on incorporating mechanisms to ensure geometric diversity alongside property fairness, reducing the computational cost of the iterative ensemble training, and adapting the framework to target specific high-performance property regions rather than general exploration.

Summary updated

BibTeX
@misc{2023fairness,
  author = {Lijun Sun (孙立君) and Jiarui Xie and Chonghui Zhang and Lijun Sun and Yaoyao Fiona Zhao},
  title = {{Fairness- and uncertainty-aware data generation for data-driven design}},
  year = {2023},
  eprint = {2309.05842},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2309.05842},
}
Figure