Introduction
Data augmentation is a cornerstone of modern deep learning, particularly when labeled data is scarce. A prominent technique is Mixup, which creates synthetic training examples by linearly interpolating between pairs of inputs and their corresponding labels in Euclidean space. However, many data modalities—such as natural language with its hierarchical syntax or audio signals with hyperbolic interference—possess inherent non-Euclidean structures. Forcing these "tree-like" structures into flat Euclidean space can lead to significant vectorial distortions, resulting in suboptimal synthetic samples.
To address this, we present HypMix, a novel data augmentation framework that performs interpolation within hyperbolic space. By leveraging the Poincaré ball model, HypMix captures the hierarchical and complex latent structures of data more effectively than standard Euclidean methods.
Methodology
The core innovation of HypMix lies in moving the interpolation process from the flat Euclidean plane to the curved geometry of the Poincaré ball.
MixH and HypMix
The fundamental operation, MixH (Hyperbolic Mixup), replaces standard linear addition with Möbius operations. As illustrated in Figure 1, the process involves:
- Mapping Euclidean hidden states ($h_k$) into hyperbolic space using an exponential map ($\exp_0$).
- Performing a weighted Möbius addition and scalar multiplication within the hyperbolic manifold.
- Projecting the resulting mixed representation back to Euclidean space via a logarithmic map ($\log_0$).
This approach is modality-agnostic and can be applied to the latent representations of any neural network architecture, such as BERT for text or ResNet for vision.
Möbius Gyromidpoint Label Estimation (MGLE)
For semi-supervised learning, the authors introduced MGLE. This technique estimates labels for unlabeled data by generating multiple augmentations of a single sample and calculating their hyperbolic weighted midpoint. This produces a stable "soft label" that respects the underlying geometry of the feature space.
Experimental Setup
HypMix was evaluated across three distinct modalities to prove its versatility:
- Speech: ESC-10, US8K, and Urdu SER (emotion recognition).
- Text: AG News, DB Pedia, and Arabic HS (hate speech detection).
- Vision: CIFAR-10 and CIFAR-100.
The inclusion of low-resource datasets like Urdu SER and Arabic HS was critical to testing the model's performance in data-constrained environments.
Results and Discussion
HypMix consistently outperformed standard Euclidean Mixup (EUCMIX) across all modalities and tasks.
Performance in Low-Resource Settings
The advantages of hyperbolic interpolation are most pronounced when training data is extremely limited. For instance, on the Urdu Speech Emotion dataset with only 5 samples per class, HypMix achieved an error rate of 16.5%, significantly lower than the 18.8% achieved by Euclidean Mixup and the 28.7% of the base model.
Robustness and Efficiency
Beyond accuracy, HypMix enhances model robustness. Models trained with this method showed a ~6% improvement in resistance to adversarial attacks (e.g., FGSM). Furthermore, despite the mathematical complexity of hyperbolic operations, HypMix is computationally efficient. On modern GPUs, the operations are highly parallelizable, adding only a negligible constant factor to training time compared to standard Mixup.
Conclusion
HypMix demonstrates that the geometry of the augmentation space is a critical factor in model generalization. By aligning the interpolation manifold with the inherent hierarchical structure of the data, HypMix provides a more physically and semantically plausible set of synthetic examples. Future work will explore its application to large-scale benchmarks like GLUE and its potential in multimodal learning environments.
Summary updated