Pseudo-Nonlinear Data Augmentation: A Constrained Energy Minimization Viewpoint

Pingbang Hu, Mahito Sugiyama

Data augmentation is the "secret sauce" of modern machine learning, helping models generalize by artificially expanding small datasets. But as we move toward increasingly complex generative models like GANs and Diffusion for this task, we run into a paradox: to augment a small dataset, you often need a massive amount of data just to train the generator.

In their paper "Pseudo-Nonlinear Data Augmentation: A Constrained Energy Minimization Viewpoint," Pingbang Hu and Mahito Sugiyama propose a way out. By leveraging the elegant math of information geometry, they’ve developed a learning-free, efficient, and highly controllable framework for data augmentation that works across images, audio, and tabular data.

TL;DR

The authors introduce a data augmentation method that replaces deep generative models with principled geometric projections on statistical manifolds. By "designing" rather than "learning" latent spaces, they achieve competitive performance and superior stability on downstream tasks with significantly less computational overhead and better user control.

What Problem Does This Paper Address?

Current state-of-the-art data augmentation typically relies on deep generative models (VAEs, GANs, etc.). While powerful, these methods suffer from three major drawbacks:

  1. The Scarcity Paradox: Augmentation is most needed when data is scarce, yet generative models require large datasets to be effective.
  2. Computational Cost: Generating millions of synthetic samples using deep networks is slow and energy-intensive.
  3. Black-Box Nature: It is difficult to interpret or precisely control the transformations these models apply, which is a dealbreaker for high-stakes applications like medical imaging or finance.

Existing "learning-free" alternatives like PCA-based resampling or simple Mixup are often too linear or heuristic to capture the complex, non-linear structures inherent in real-world data.

Methods

The core idea of the paper is to treat data points not just as vectors, but as probability distributions residing on a curved statistical manifold.

  1. Poset-Based Modeling: The authors represent data structures (like tensors) using partially ordered sets (posets). This allows them to encode prior knowledge about feature relationships—for example, the spatial grid of an image—directly into the geometry of the space.
  2. Dually-Flat Manifolds: They utilize the theory of information geometry, where a manifold has two coordinate systems: one where "straight lines" (geodesics) represent energy-based relationships ($\theta$) and another representing expectations ($\eta$).
  3. The PNL Algorithm:
    • Encoding: Data is projected onto a low-dimensional "base sub-manifold" ($B$). This acts as a non-linear dimensionality reduction.
  4. Augmenting: Simple operations, like linear interpolation, are performed in this latent space.
  5. Decoding (Backward Projection): To map back to the original data space, the algorithm finds a "local data sub-manifold" ($D$) based on the nearest neighbors of the augmented point and projects it back.
Figure 1: Given structured data, we first design a poset Ωthat reflects the structure or the relationship between features. The resulting real-valued poset is then embedded into the statistical manifold S as a discrete probability distribution pθ(x) via an embedding φ. Finally, the log-linear model on posets provides the dually-flat coordinates (θ, η) for pθ, which can be computed efficiently (Section 3.2).

The term "pseudo-nonlinear" comes from a clever mathematical duality: the projection is linear in the manifold's intrinsic coordinates but appears non-linear in the original data space.

Datasets

The authors validated their approach across three distinct modalities:

  • Images: MNIST (handwritten digits) and CIFAR-10 (natural objects).
  • Audio: Speech Commands (downsampled to 4000 dimensions to manage memory).
  • Tabular Data: Connectionist Bench (Sonar), Taiwanese Bankruptcy, and Wine Quality datasets.

Key Insights and Ideas

  • Geometry as a Prior: Instead of hoping a neural network learns the structure of a dataset, we can prescribe it. By choosing the right poset and sub-manifold, we can decide which features (like color or shape) should be preserved and which should be varied.
  • Energy Minimization: The projection process is mathematically equivalent to finding a data configuration that minimizes a modality-specific energy function (specifically, KL divergence). This provides a theoretical guarantee that the augmented data remains "plausible."
  • Stability in Small Data: The method is remarkably stable. On the tiny Connectionist Bench dataset (only 208 points), PNL significantly reduced the variance in model performance compared to all other baselines.
Figure 3: Illustration of forward and backward projection. Here, wi: latent representation of the original data zi, obtained from forward pro- jection to B; w∗: augmented latent represen- tation; w∗7→z′∗: backward projection to D, obtained from the original data of the nearest neighbor(s) of w∗in the latent space.

Key Results

  • Superior Performance: PNL consistently outperformed or matched baselines like standard augmentation (STD), Autoencoders (AE), and Mixup across all modalities.
  • Quantitative Gains: On the Wine Quality dataset, PNL achieved a test accuracy of 59.03%, outperforming the original dataset (55.00%) and standard noise-based augmentation (57.85%).
  • Fine-Grained Control: Qualitative results on CIFAR-10 showed that PNL could preserve intricate details (like the color of an ostrich's beak) while varying the background, a level of control that crude shape-to-color transformations often lose.

Limitations and Future Work

While promising, the framework has a few hurdles to clear:

  • Invariance: The current poset-based approach isn't naturally designed to handle symmetries like rotation or index permutations (important for graph data).
  • Dimensionality Scaling: Although efficient, the convex optimization required for projection can become memory-intensive for extremely high-dimensional data (e.g., raw 16kHz audio).
  • Expert Knowledge: Designing the optimal poset structure currently requires some domain expertise, though the authors suggest that "natural" tensor structures work well as a default.

Future research will likely focus on automating the poset design and extending the framework to handle more complex geometric invariances.

Summary updated

Abstract

We propose a simple yet novel data augmentation method for general data modalities based on energy-based modeling and principles from information geometry. Unlike most existing learning-based data augmentation methods, which rely on learning latent representations with generative models, our proposed framework enables an intuitive construction of a geometrically aware latent space that represents the structure of the data itself, supporting efficient and explicit encoding and decoding procedures. We then present and discuss how to design latent spaces that will subsequently control the augmentation with the proposed algorithm. Empirical results demonstrate that our data augmentation method achieves competitive performance in downstream tasks compared to other baselines, while offering fine-grained controllability that is lacking in the existing literature.
BibTeX
@inproceedings{hu2026pseudo,
  author = {Pingbang Hu and Mahito Sugiyama},
  title = {{Pseudo-Nonlinear Data Augmentation: A Constrained Energy Minimization Viewpoint}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=p9A1oyktVB},
}
Figure