Abstract Standard deep learning models typically operate in Euclidean space, which often fails to capture the inherent hierarchical structures found in natural language and speech. This paper introduces ADAPT (Adaptive Poincaré Transfer), a framework that enables neural networks to utilize hyperbolic geometry by treating the manifold's curvature as a learnable parameter. By initializing and jointly optimizing the radius of curvature based on dataset-specific hyperbolicity, ADAPT achieves superior performance across 12 languages in both text and speech tasks.
Introduction
Most contemporary NLP models are grounded in "flat" Euclidean space. While computationally convenient, Euclidean geometry is ill-suited for representing hierarchical or tree-like data, leading to significant structural distortion. Hyperbolic space, such as the Poincaré disk, offers a more natural fit for such data because its volume grows exponentially with distance from the origin. However, existing hyperbolic models often rely on a fixed radius of curvature, which ignores the varying "tree-likeness" (hyperbolicity) of different datasets and the dynamic nature of latent embeddings during training.
The ADAPT Framework
The ADAPT framework addresses these limitations by making the curvature of the mathematical space an adaptive component of the model.
The framework follows a three-step process:
- Measuring Hyperbolicity: The "tree-likeness" of a dataset is quantified using Gromov’s $\delta$-hyperbolicity, calculated from the distances between encoded data samples.
- Smart Initialization: The radius ($R$) of the Poincaré ball is initialized based on the measured $\delta$-hyperbolicity, rather than using a generic default.
- Joint Optimization: The radius of curvature is treated as a trainable parameter, updated via backpropagation alongside the network weights. This allows the manifold to dynamically "stretch" or "shrink" to accommodate evolving latent representations.
To demonstrate the framework's versatility, the authors "hyperbolized" two state-of-the-art techniques: Variational Information Bottleneck (ADAPT-VIB) and Saliency-aware Span Mixup (ADAPT-SSMix), utilizing Möbius arithmetic for operations within the curved space.
Experimental Evaluation
The authors evaluated ADAPT across a diverse set of tasks and languages:
- Text Tasks: Natural Language Inference (XNLI) across 12 languages (including Hindi, Turkish, Urdu, Chinese, and Arabic), Named Entity Recognition (CoNLL-2003), and GLUE benchmarks (RTE, MRPC).
- Speech Tasks: Emotion recognition in Urdu, Italian (EmoVO), and Persian (ShEMO).
Results and Discussion
ADAPT consistently outperformed Euclidean and fixed-curvature hyperbolic baselines. For instance, on the Chinese (Zh) XNLI task, ADAPT-VIB reached 51.35% accuracy, significantly higher than the 42.32% of the base model. Speech tasks showed even more dramatic gains, with Urdu Speech Emotion Recognition jumping from 81.25% to 92.50%.
A key insight is the "moving target" of curvature. As shown in Figure 2, the optimal radius of curvature shifts as the model learns. By allowing $R$ to be trainable, the geometry stays synchronized with the model's internal state. Furthermore, speech signals appear to be naturally hyperbolic, explaining the substantial performance boosts in audio tasks. Despite the mathematical complexity of Möbius arithmetic, ADAPT-VIB proved computationally efficient, often reaching benchmark accuracy in fewer epochs than standard VIB.
Conclusion
ADAPT provides a robust set of guidelines for integrating hyperbolic geometry into multimodal and multilingual NLP. While predicting the exact optimal curvature theoretically remains a challenge, the empirical success of trainable manifolds suggests a promising path forward. Future work may extend this framework to Graph Neural Networks and more complex multimodal interactions.
Summary updated