To Augment or Not to Augment? Diagnosing Distributional Symmetry Breaking

Hannah Lawrence, Elyssa F. Hofgard, Vasco Portilheiro, Yuxuan Chen, Tess E. Smidt, Robin Walters

Abstract

Standard practices in machine learning, such as data augmentation and the use of equivariant neural networks, assume that data distributions are symmetric under certain transformations. This paper challenges that assumption by introducing a metric to quantify "distributional symmetry breaking" (DSB). The authors demonstrate that many common benchmarks are highly canonicalized, and in such cases, forcing model invariance through augmentation can paradoxically degrade performance.

Introduction: The Assumption of Symmetry

In domains ranging from computer vision to molecular modeling, researchers often employ equivariant neural networks or data augmentation (e.g., random rotations) to ensure models treat transformed versions of the same object identically. This approach rests on the implicit assumption that all orientations of a data point are equally likely to occur in the real world.

However, real-world data frequently exhibits distributional symmetry breaking. While a baseball might appear in any orientation, a coffee mug is typically found upright. When a dataset is "canonicalized"—meaning objects consistently appear in specific orientations—forcing a model to be invariant to those orientations may discard valuable information that facilitates learning.

Figure 1: (a) Comparison of isotropic (baseball) vs. canonicalized (mug) distributions. (b) The concept of canonicalization. (c) Task-relevant orientation examples.

Quantifying Symmetry Breaking

To diagnose this phenomenon, the authors propose a novel, interpretable metric, $m(p_X)$, based on a two-sample classifier test. The procedure is straightforward:

  1. Transformation: A copy of the original dataset (Label 0) is created, where every data point is subjected to a random transformation (Label 1).
  2. Classification: A binary classifier is trained to distinguish between the original and transformed sets.
  3. Metric Calculation: If the classifier achieves only 50% accuracy, the dataset is perfectly symmetric. Accuracy approaching 100% indicates a highly canonicalized dataset where the original orientation is distinct and predictable.
Figure 2: Overview of the datasets evaluated (left) and the two-sample classifier test framework (right).

Benchmarking "Secret" Alignment

Applying this metric across diverse benchmarks revealed a surprising trend: almost all major point cloud and molecular datasets are "secretly" aligned. For example, the QM9 molecular dataset scored 97.6%, and the ModelNet40 CAD dataset scored 92.45%. These results suggest that the "equally likely orientations" assumption is frequently violated in practice.

When Augmentation Hurts

The paper provides a theoretical analysis using invariant ridge regression, showing that when data features are strongly correlated with orientation, data augmentation can lead to a "blow-up" in generalization error, particularly in over-parameterized models.

Empirical results support this: on ModelNet40, training without augmentation achieved 78.65% accuracy, significantly outperforming the 61.89% achieved with full train-and-test augmentation. In these cases, the model benefits from the "canonical" information present in the dataset to classify objects more efficiently.

The Locality Loophole in Molecular Data

A key finding concerns the success of equivariant models in chemistry despite high global canonicalization. The authors developed a "local" version of their metric and found that while QM9 is globally ordered (98.5% score), the symmetry-breaking score for local atomic neighborhoods drops to 67.6%. This suggests that equivariant models succeed because they capture local symmetries (isotropic chemical environments) even when the global distribution is biased.

Conclusion

The decision to use data augmentation or equivariant architectures should not be automatic. If a dataset is highly canonicalized and orientation is "task-useful" (e.g., distinguishing a '6' from a '9'), symmetry-aware methods may be counterproductive. By using the $m(p_X)$ metric, practitioners can better diagnose the distributional properties of their data and choose the most effective modeling strategy.

Summary updated

BibTeX
@inproceedings{lawrence2026augment,
  author = {Hannah Lawrence and Elyssa F. Hofgard and Vasco Portilheiro and Yuxuan Chen and Tess E. Smidt and Robin Walters},
  title = {{To Augment or Not to Augment? Diagnosing Distributional Symmetry Breaking}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=ZTb4YmHD9n},
}
Figure