When we build machine learning pipelines, we often treat data preparation and data augmentation as a "bag of tricks." We select a set of operations—such as cropping, blurring, imputing missing values, or adding noise—and apply them to our dataset. However, data operations are fundamentally non-commutative: changing the order in which you apply them changes the final output.
Imagine an image processing pipeline: if you apply a localized pixel mask and then rotate the image, the masked region rotates along with the object. If you rotate the image first and then apply the static mask, the mask covers a completely different semantic part of the data.
In natural language processing, as noted by Qu et al., you cannot easily apply a discrete transformation like back-translation after you have already mapped words into a continuous space and added adversarial embedding noise. The sequence of operations dictates what is mathematically and semantically possible.
Why Order Matters: Non-Linear Interactions
The sensitivity of machine learning models to operation ordering spans across data cleaning, data reduction, and data augmentation.
In data cleaning, Laure Berti-Equille demonstrates that operations like missing-value imputation and outlier removal interact in highly non-linear ways. Imputing values before removing outliers might inadvertently create artificial outliers or mask real ones, completely altering the downstream model's performance. Similarly, in software engineering data reduction, Xuan et al. showed that executing feature selection before instance selection yields entirely different bug triage accuracy than reversing the sequence.
Because the history of applied operations alters the underlying data distribution at each step, treating transformations as independent events fails. As Ratner et al. argue, a system must explicitly track the cumulative history of transformations to prevent generating unrealistic or invalid data.
[Raw Data] ───> (Transformation A) ───> [Intermediate State] ───> (Transformation B) ───> [Final Data]
▲
Fundamentally Different!
▼
[Raw Data] ───> (Transformation B) ───> [Intermediate State] ───> (Transformation A) ───> [Final Data]
Modeling Augmentation as a Trajectory
To move beyond hand-crafted, static heuristics, researchers frame data preparation not as a static pipeline, but as a sequential decision-making problem.
Instead of picking a random subset of transformations, we can model the process as a Markov Decision Process (MDP). In this framing, an input data point represents the initial state. An agent selects a transformation (an action), which transitions the data point to a new state. This creates a trajectory of operations.
Qin et al. utilized this MDP framing to discover complex transformation trajectories for medical image segmentation, showing that learned sequences outperform single random operations. Similarly, Ratner et al. combined a recurrent architecture with reinforcement learning to capture these non-commutative dependencies, allowing the model to learn how operations naturally build upon one another.
Moving Beyond Fixed Heuristics
Historically, automated augmentation frameworks like AutoAugment applied transformations in a rigid, static order. Terauchi et al. point out that ignoring operation sequencing limits the effectiveness of data representations. Similarly, Ruppli et al. note that even advanced differentiable frameworks for contrastive learning typically fix the sequence of operations (e.g., always blurring before cropping), which prevents the model from exploring more challenging or diverse views.
When fixed pipelines are successful, it is often because they hit upon a structured hierarchy that aligns with the data's domain. For instance, Saleem et al. discovered that for plant stress classification, a pipeline that sequences geometric transformations first, followed by color adjustments, and concludes with localized masking yields optimal feature representations. In 3D object detection, Choi et al. also utilize a fixed sequential order specifically to prevent destructive interference between part-aware point cloud manipulations.
The Combinatorial Search Challenge
If optimal sequences exist, why don't we always search for them? The answer lies in the sheer size of the search space. Finding the ideal permutation of $N$ operations is a combinatorial optimization problem that scales factorially.
As highlighted by surveys like Cao et al. and software engineering studies by Yedida et al., finding the optimal stacking order when combining diverse techniques (like instance, label, and boundary engineering) remains an open challenge.
However, the rewards of treating data ordering as an optimization lever are substantial. Even outside of data augmentation, Kim et al. demonstrate that in standard optimization algorithms like incremental gradient descent, the existence of optimal data processing permutations can drastically outperform standard stochastic sampling. By treating the ordering of data operations as a primary lever for optimization rather than an afterthought, we can design data preparation pipelines that actively maximize downstream model performance and robustness.
Key Foundational Work
Learning to Compose Domain-Specific Transformations for Data Augmentation (2017) — contributed 3 concept elements to this direction.
Towards Effective Bug Triage with Software Data Reduction Techniques (2015) — contributed 2 concept elements to this direction.