Introduction
Machine Learning (ML) models are increasingly deployed in sensitive domains such as Human Resources (HR) and healthcare. However, these models are notoriously "data-hungry," often requiring larger datasets than are readily available. While data augmentation—the creation of synthetic samples—is a common solution to data scarcity, it frequently acts as a "bias megaphone." If the original training data contains social prejudices, such as gender pay gaps or racial disparities, standard augmentation techniques like SMOTE (Synthetic Minority Over-sampling Technique) tend to replicate and amplify these inequalities.
A critical gap exists in current methodologies: most augmentation techniques prioritize statistical distribution over fairness. Furthermore, the common practice of removing "sensitive" features (e.g., race or gender) often fails because models can learn these biases through proxy variables. This paper introduces a "fair-by-design" augmentation process that balances subgroup representation while maintaining predictive power, integrated into a continuous assurance framework.
The MUSA Platform and Continuous Assurance
The authors propose a multi-step pipeline integrated into the MUSA platform, a framework designed for continuous assurance of ML services. This architecture ensures that as new data arrives, the system updates its internal representations and re-evaluates fairness metrics to prevent bias drift.
The core of this procedure involves four primary stages:
- Similarity Network Representation: Instead of treating data as a flat table, the dataset is represented as a graph. Each data point is a node, and edges are established based on the Gower distance, which effectively handles mixed numerical and categorical data. This distance is further refined using a scaled Exponential kernel to define the strength of relationships.
- Strategic Synthetic Generation: The system identifies "low-density" regions in the network—areas where specific subgroups are underrepresented. It then strategically generates new synthetic nodes in these gaps to balance the dataset's demographic distribution.
- AVPRA (Agent-Based Vector-Label Propagation): To label these new synthetic points, the authors employ AVPRA. Unlike standard label propagation, AVPRA uses Vector Labels (VLs), allowing the system to propagate weighted labels across the network. This captures the complexity of the data distribution more accurately than simple majority-rule approaches.
- Continuous Monitoring: The process is incremental. The network is updated as new data is ingested, and fairness metrics like Equal Opportunity are monitored to ensure the model remains unbiased over time.
Experimental Evaluation
The study utilized a public Human Resources (HR) dataset containing Curriculum Vitae (CV) data for 301 employees. The features included a mix of numerical and categorical data such as experience, age, gender, and ethnicity. The target variable, "Pay Rate," was categorized into bins to transform the task into a classification problem.
The proposed method was compared against three baselines: the original imbalanced dataset, a class-balanced version using SMOTE, and a group-balanced version. Performance was evaluated using Random Forest (RF) and XGBOOST classifiers.
Key Insights: The Latent Space Shield
A significant conceptual contribution of this work is the use of a similarity network to create a "latent feature space." This space acts as a shield, protecting the model from over-relying on a restricted set of sensitive features. By including sensitive features in the network construction rather than deleting them, the system can explicitly identify and fill representation gaps, leading to a more "depolarized" data distribution.
Interestingly, the authors measured "Collective Feature Efficiency" ($F4$) and found it increased by 34% after augmentation. While an increase in $F4$ typically suggests a harder classification task, in this context, it indicates that the model can no longer rely on simplistic, biased patterns. Instead, it is forced to learn more robust and equitable relationships.
Results and Discussion
The experimental results demonstrate a significant breakthrough in balancing performance and equity:
- Fairness Improvement: The most notable result was a ~50% improvement in fairness metrics. The "Equal Opportunity" gap—the difference in true positive rates between privileged and unprivileged groups—dropped from 0.35 to 0.17.
- Accuracy Trade-off: There was a slight decrease in accuracy, with the F1-score dropping from approximately 67% to 65%. The authors argue this is a necessary trade-off for a model that generalizes better across demographic subgroups and adheres to ethical standards.
- Transparency: SHAP (SHapley Additive exPlanations) visualizations confirmed that in the augmented dataset, the model’s reliance on sensitive features like "Sex" and "Age" was significantly more distributed compared to the original or SMOTE-augmented data.
Limitations and Future Work
While promising, the approach currently faces some limitations. The primary experiment was conducted on a relatively small dataset (301 instances). Future work must validate this method on "big data" scales to assess the computational efficiency of similarity network construction. Additionally, the performance of the Exponential kernel is sensitive to its hyperparameter ($\mu$), requiring further exploration into optimal tuning for diverse tabular datasets. Finally, the authors aim to further refine how "continuous assurance" handles rapid concept drift in live production environments.
Summary updated