Abstract
Recent research explores the intersection of number theory and machine learning by investigating whether neural networks can predict the Frobenius traces ($a_p$) of elliptic curves. By treating trace sequences as a language, Transformer models demonstrate an ability to discover deep mathematical structures, such as modular congruences, without explicit instruction. This paper summarizes the methodologies, key findings, and the "sign struggle" encountered when predicting these fundamental arithmetic invariants.
Introduction
Elliptic curves, defined by equations of the form $y^2 = x^3 + Ax + B$, are central objects in modern number theory. Their properties are often captured by "Euler factors" or Frobenius traces, denoted as $a_p$, which count the number of points on the curve over a finite field of size $p$. While a finite set of these traces can uniquely identify a curve's isogeny class, calculating subsequent traces typically requires knowledge of the curve's conductor. This study investigates if machine learning models can learn to predict unknown traces from a limited input set, effectively "discovering" the underlying L-function patterns.
Methodology
The researchers utilized two primary neural architectures to tackle the prediction task. The core model is an encoder-only Transformer, specifically a variation of the Int2Int model. This architecture treats sequences of integer traces as tokens, allowing the model to learn contextual relationships between different primes. For specific binary classification tasks, such as determining the sign of $a_2$, standard Feedforward Neural Networks (FNNs) were employed.
The prediction problem was framed as a classification task rather than regression. For instance, predicting $a_{97}$ involved selecting from 39 possible integer classes. To interpret the models' internal representations, Principal Component Analysis (PCA) was applied to the learned embeddings.
Data and Experiments
The study leveraged massive datasets of elliptic curve isogeny classes, including the ECQ6 and ECQ7 sets containing up to 28.2 million classes. A high-quality subset from the LMFDB with conductors between $2^{15}$ and $2^{16}$ was also used for targeted experiments. Researchers had to carefully manage these datasets to remove duplicate entries that could lead to data leakage, particularly when working with modular versions of the data.
Key Insights: Discovering Mathematical Structure
The most significant finding was the models' ability to implicitly learn number-theoretic congruences. PCA visualizations revealed that the models naturally clustered trace embeddings based on parity (even vs. odd) and divisibility by 3. This suggests that the networks "discover" modular arithmetic as a foundational strategy for prediction.
The Sign Struggle and Parity Path
A recurring theme in the results is the "sign struggle." While models were highly proficient at predicting the absolute value and parity of a trace, they frequently faltered in determining whether a trace was positive or negative.
As shown in the figure above, the model's predictions often fall on the anti-diagonal, indicating a sign error. Despite this, the models achieved impressive results in parity prediction, reaching accuracies near 0.94 for certain primes. For the complex task of predicting $a_{97}$ given all prior traces, the Transformer achieved an accuracy of 0.4923—nearly six times better than a random baseline.
Conclusion and Future Work
This work demonstrates that machine learning can capture sophisticated arithmetic patterns in elliptic curves. While the "black box" nature of Transformers remains a challenge, the discovery of modular congruences through embedding analysis provides a window into the model's reasoning. Future research will focus on improving sign prediction and exploring whether these techniques generalize to more complex arithmetic objects like higher-genus curves or general L-functions.
Summary updated