On the Applicability of $p$-adic Numbers in Computational Modules for Neural Networks

Vladimir EvgrafovarXiv 2026 · arXiv

TL;DR

This paper investigates whether $p$-adic arithmetic can serve as a "drop-in" replacement for standard floating-point math in neural network modules to mitigate numerical instability. Through experiments with linear regression, the author discovers a significant "negative result": current methods for converting between real and $p$-adic numbers are too unstable and imprecise to maintain model accuracy, failing to reconstruct even a single test prediction correctly.

What Problem Does This Paper Address?

In deep learning, particularly with recurrent neural networks, researchers often face the exploding gradient problem. When you multiply many Jacobian matrices together during backpropagation, the gradients can grow exponentially, leading to unstable training or numerical overflow.

The author explores a novel mathematical detour: $p$-adic numbers ($\mathbb{Q}_p$). Unlike the real numbers we use every day, $p$-adic numbers use a different definition of "size" (the $p$-adic norm). Crucially, they satisfy the strong triangle inequality, which implies that adding two numbers cannot produce a result larger than the largest input. Theoretically, if a neural network's operations were performed in a $p$-adic field, gradients might be naturally bounded, preventing them from "exploding."

The specific question addressed is: Can we insert a $p$-adic computational module into an existing machine-learning pipeline (Rational $\to$ $p$-adic $\to$ Computation $\to$ Rational) without breaking the model?

Methods

The author tests this "integration scheme" using a simple one-feature linear regression. The process involves:

  1. Forward Conversion: Mapping rational or floating-point inputs and fitted coefficients into the field of $p$-adic numbers ($\mathbb{Q}_p$).
  2. $p$-adic Computation: Performing the linear regression prediction ($y = ax + b$) using $p$-adic arithmetic.
  3. Rational Reconstruction: Using algorithms (like the extended Euclidean algorithm) to convert the $p$-adic result back into a standard rational number that the rest of the ML pipeline can understand.

The study compares two primary software implementations: the Python library pyAdic and the computer algebra system SageMath. It also tests an alternative representation based on digit reversal to see if aligning $p$-adic and real metrics more closely would help.

Datasets

The study uses the standard Diabetes dataset from scikit-learn.

  • Feature: BMI (normalized, feature index 2).
  • Target: Disease progression.
  • Split: The last 20 observations were used as the test set to evaluate the reconstruction accuracy.

Key Insights and Ideas

  • The "Strong Triangle Inequality" Hook: The primary motivation is that in $\mathbb{Q}_p$, the norm $\|u + v\|_p \leq \max(\|u\|_p, \|v\|_p)$. This suggests a world where "addition is safe," providing a potential mathematical shield against numerical instability.
  • The Precision Trap: $p$-adic representations in computers are finite approximations. The author found that rational reconstruction is hyper-sensitive to the chosen precision. Small changes in precision didn't just cause small errors; they could flip the sign or change the magnitude of the result by orders of magnitude.
  • The Carry Propagation Conflict: The "digit reversal" experiment provided a deep insight into why $p$-adic and real arithmetic are hard to mix. In standard addition, "carries" move toward higher powers. In a reversed $p$-adic representation, the carries would need to move in the opposite direction, meaning standard hardware addition cannot be reused.

Key Results

The results were overwhelmingly negative for the "drop-in" module approach:

  • Zero Success Rate: In the main experiment ($p=2$, precision 50), 0 out of 20 test predictions were reconstructed correctly within a tolerance of $10^{-6}$.
  • Reconstruction Failures: In pyAdic, the reconstruction function frequently returned False, meaning it couldn't find any rational number that matched the $p$-adic result.
  • Baseline Mismatch: Even when SageMath successfully completed the "round trip" conversion, the resulting values differed significantly from the baseline scikit-learn predictions.
  • Numerical Instability: A simple test converting the value 0.04716281 to $p$-adic and back resulted in a value with the opposite sign and a different order of magnitude.

Limitations and Future Work

The author concludes that $p$-adic numbers cannot be used as a simple intermediate module in a standard real-valued pipeline. The "interface" between the two number systems is currently too lossy and unstable.

Future Directions: * Native $p$-adic Models: Instead of converting back and forth, researchers should look at models defined entirely over $p$-adic fields from the start (data, weights, and loss functions). * Automated Parameter Selection: Investigating how to automatically choose the prime $p$ and the required precision for specific tasks. * $p$-adic Cellular Neural Networks: Exploring architectures where $p$-adic digits correspond to individual neuron states, which may be better suited for parallel computation and image processing.

Summary updated

Abstract

The applicability of $p$-adic arithmetic as an intermediate computational module in an otherwise unchanged machine-learning pipeline is investigated. A one-feature linear regression on the scikit-learn Diabetes dataset serves as the test model. The fitted coefficients are represented in $\mathbb{Q}_p$, predictions are computed in that field, and rational reconstruction is applied to the outputs. In the pyAdic experiments, rational reconstruction either returns no result or produces values that differ from the baseline predictions; changing computational precision can change both the magnitude and sign of the reconstructed value. In SageMath, forward conversions succeed, but the reconstructed predictions differ from the baseline. An alternative representation based on digit reversal fails to preserve ordinary addition because of carry propagation. Under the tested settings, none of the test predictions is reconstructed correctly. These results show that the tested conversion and reconstruction procedures do not provide a compatible intermediate module even for linear regression. Models defined directly over $p$-adic data and operations offer a direction for further investigation.
BibTeX
@misc{evgrafov2026applicability,
  author = {Vladimir Evgrafov},
  title = {{On the Applicability of $p$-adic Numbers in Computational Modules for Neural Networks}},
  year = {2026},
  eprint = {2611.20467},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20467},
}
Figure