Robust Machine Learning by Transforming and Augmenting Imperfect Training Data

Elliot Creager

Abstract

Machine Learning (ML) is an expressive framework for turning data into computer programs. Across many problem domains—both in industry and policy settings—the types of computer programs needed for accurate prediction or optimal control are difficult to write by hand. On the other hand, collecting instances of desired system behavior may be relatively more feasible. This makes ML broadly appealing, but also induces data sensitivities that often manifest as unexpected failure modes during deployment. In this sense, the training data available tend to be imperfect for the task at hand. This thesis explores several data sensitivities of modern machine learning and how to address them. We begin by discussing how to prevent ML from codifying prior human discrimination measured in the training data, where we take a fair representation learning approach. We then discuss the problem of learning from data containing spurious features, which provide predictive fidelity during training but are unreliable upon deployment. Here we observe that insofar as standard training methods tend to learn such features, this propensity can be leveraged to search for partitions of training data that expose this inconsistency, ultimately promoting learning algorithms invariant to spurious features. Finally, we turn our attention to reinforcement learning from data with insufficient coverage over all possible states and actions. To address the coverage issue, we discuss how causal priors can be used to model the single-step dynamics of the setting where data are collected. This enables a new type of data augmentation where observed trajectories are stitched together to produce new but plausible counterfactual trajectories.

Introduction

Modern machine learning excels at pattern recognition, yet it remains fundamentally limited by the quality of its training data. Models are often "underspecified," meaning they may achieve high accuracy by latching onto the wrong signals—such as historical human prejudices or spurious correlations that do not hold outside the training environment. For example, a model might identify a "cow" based on the presence of green grass rather than the animal's features. Furthermore, in Reinforcement Learning (RL), models frequently fail because they lack sufficient exposure to diverse state-action transitions.

This research addresses these "data imperfections" across three primary pillars: algorithmic fairness, shortcut learning, and data coverage in RL. By transforming and augmenting imperfect data, we can produce models that are fairer, more robust to distribution shifts, and more sample-efficient.

Methods for Mitigating Data Imperfections

Algorithmic Fairness: LAFTR and FFVAE

To prevent models from codifying human discrimination, we propose focusing on the data representation itself. If a representation is "clean," any downstream model built upon it inherits its fairness properties.

  • LAFTR (Learned Adversarially Fair and Transferable Representations): This framework utilizes adversarial training to learn representations that are "blind" to sensitive attributes (e.g., race or gender). An encoder creates a representation while an adversary attempts to recover the sensitive attribute. The resulting "fairness bottleneck" ensures that the representation can be safely reused for multiple downstream tasks.
  • FFVAE (Flexibly Fair VAE): This approach employs a Variational Autoencoder with disentanglement priors. It isolates sensitive information into specific dimensions of the latent space, allowing practitioners to simply "turn off" those dimensions at test time to ensure equitable outcomes.

Robustness to Spurious Features: EIIL

Most robust learning methods require data to be pre-labeled into different "environments" (e.g., photos taken in different lighting). However, such labels are rarely available in real-world datasets. Environment Inference for Invariant Learning (EIIL) removes this requirement by leveraging the failure of standard Empirical Risk Minimization (ERM). Since ERM is prone to latching onto shortcuts, EIIL uses a reference model to automatically infer which data points belong to "worst-case" environments. It then trains a new model to be invariant across these discovered partitions, effectively automating the discovery of robust features.

Data Coverage in RL: CoDA and MoCoDA

In RL, the state space grows exponentially, leading to sparse data coverage. We introduce Counterfactual Data Augmentation (CoDA), which utilizes "local causal models." If two objects in a scene (such as two independent robot arms) do not interact, their dynamics are independent. CoDA identifies these independent subprocesses and "swaps" their trajectories from different experiences. This allows for the creation of millions of new, plausible training transitions without requiring additional real-world interaction, effectively turning the "curse of dimensionality" into a "blessing of independence."

Experimental Results

The proposed methods were validated across a variety of benchmarks, including UCI Adult, CelebA, Colored MNIST (CMNIST), and complex robotic manipulation tasks like FetchPush and Slide2.

  • Transferable Fairness: LAFTR improved fairness (measured by Equalized Odds) by approximately 20% on average across unseen transfer tasks compared to standard baselines, with minimal impact on predictive accuracy.
  • Unsupervised Robustness: EIIL achieved 68.4% test accuracy on the CMNIST benchmark without requiring environment labels. This outperformed standard Invariant Risk Minimization (IRM) methods that did have access to those labels.
  • RL Efficiency: In the "Slide2" robotic environment, CoDA doubled the sample efficiency of the agent, enabling it to solve complex tasks that were previously unreachable within the same data budget.

Discussion and Future Work

While these methods significantly improve model robustness, several challenges remain. Many fairness techniques still rely on sensitive attribute labels that may be unavailable due to privacy constraints. Additionally, the success of EIIL depends on the initial reference model being exploitable in a specific way. Finally, scaling causal priors like CoDA to raw pixel data remains a significant hurdle, requiring advancements in object-centric computer vision.

In conclusion, because "all data are imperfect," the objective of robust machine learning is not to find a perfect dataset, but to build adaptive processes that identify and mitigate these imperfections throughout the model lifecycle.

Summary updated

Abstract

Machine Learning (ML) is an expressive framework for turning data into computer programs. Across many problem domains -- both in industry and policy settings -- the types of computer programs needed for accurate prediction or optimal control are difficult to write by hand. On the other hand, collecting instances of desired system behavior may be relatively more feasible. This makes ML broadly appealing, but also induces data sensitivities that often manifest as unexpected failure modes during deployment. In this sense, the training data available tend to be imperfect for the task at hand. This thesis explores several data sensitivities of modern machine learning and how to address them. We begin by discussing how to prevent ML from codifying prior human discrimination measured in the training data, where we take a fair representation learning approach. We then discuss the problem of learning from data containing spurious features, which provide predictive fidelity during training but are unreliable upon deployment. Here we observe that insofar as standard training methods tend to learn such features, this propensity can be leveraged to search for partitions of training data that expose this inconsistency, ultimately promoting learning algorithms invariant to spurious features. Finally, we turn our attention to reinforcement learning from data with insufficient coverage over all possible states and actions. To address the coverage issue, we discuss how causal priors can be used to model the single-step dynamics of the setting where data are collected. This enables a new type of data augmentation where observed trajectories are stitched together to produce new but plausible counterfactual trajectories.
BibTeX
@misc{creager2023robust,
  author = {Elliot Creager},
  title = {{Robust Machine Learning by Transforming and Augmenting Imperfect Training Data}},
  year = {2023},
  eprint = {2312.12597},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.12597},
}
Figure