A Survey on the Use of Machine Learning within Computer Algebra

Uzma Shafiq, Matthew England, Nayyar ZaidiarXiv 2026 · DOI · arXiv

This survey provides a comprehensive map of the emerging intersection between Machine Learning (ML) and Computer Algebra (CA). It highlights how data-driven models are being used to either optimize existing, computationally expensive symbolic algorithms through better heuristics or to perform symbolic mathematics directly using generative architectures like Transformers.

TL;DR

The paper explores how Machine Learning can overcome the "curse of dimensionality" in symbolic computation by replacing rigid human-designed heuristics with adaptive, data-driven models. It categorizes a decade of research into two paradigms—ML to guide algorithms and ML to generate solutions—while identifying critical gaps in benchmarking and explainability.

What Problem Does This Paper Address?

Computer Algebra (CA) is the field of "exact" mathematics, dealing with symbolic expressions rather than numerical approximations. While CA algorithms are mathematically rigorous, they are often notoriously slow—some, like Cylindrical Algebraic Decomposition (CAD), have a worst-case complexity that is doubly exponential.

To manage this complexity, these algorithms rely on heuristics: "rules of thumb" for choices like variable ordering or term selection. Traditionally, these were hand-crafted by experts. However, a "good" heuristic for one problem might be disastrous for another. This paper surveys how researchers are using ML to learn these choices from data, aiming to make symbolic computation practical for larger, real-world problems in physics, robotics, and engineering.

Methods

The authors categorize the research into two primary paradigms:

  1. ML to Guide CA: Here, the deterministic CA algorithm remains the "engine," but ML acts as the "driver." Models (ranging from Decision Trees and SVMs to Reinforcement Learning agents) are trained to make internal algorithmic choices. For example, in CAD, an ML model might predict which variable ordering will result in the fewest "cells" (computational units), as seen in the dramatic difference in Figure 2. A comparison of two different variable orderings for the same polynomial system, showing how one results in 13 cells while the other explodes to 89.

  2. ML to Generate CA: This is a more radical approach where generative models, primarily Transformers and LSTMs, are trained to perform symbolic tasks end-to-end. This includes directly integrating expressions or computing a Gröbner basis (a simplified set of polynomials, illustrated in Figure 3) without following a traditional step-by-step algorithm.

A comparison of two different variable orderings for the same polynomial system, showing how one results in 13 cells while the other explodes to 89.

The survey also highlights the use of Explainable AI (XAI) tools like SHAP to "peek inside" these models, helping researchers understand why certain features matter and even deriving new, simple human-readable heuristics from complex neural networks.

Fig. 1. Taxonomy for ML within CA employed in this paper.

Datasets

A recurring theme in the paper is the struggle for high-quality data.

  • SMT-LIB: A standard library for "Satisfiability Modulo Theories" problems, frequently used for CAD research.
  • MATH Dataset: Used for evaluating the mathematical reasoning of Large Language Models (LLMs).
  • Synthetic Data: Many studies rely on randomly generated polynomials. However, the authors warn that models trained on "random" math often fail to generalize to the structured math found in real-world applications.
  • New Benchmarks: The survey notes recent efforts to create larger, labeled synthetic datasets (e.g., 20,000+ problems for CAD) to support deep learning.

Key Insights and Ideas

  • The Bias Trap: Early ML models for CA were often "cheating" by learning biases in the datasets (e.g., one specific variable ordering happened to be better 80% of the time in the training set). Researchers now use data augmentation—permuting variable names—to force models to learn the underlying algebraic structure rather than just labels.
  • Regression vs. Classification: While it's intuitive to treat heuristic selection as a classification task ("Pick Option A or B"), reframing it as a regression task (predicting the actual runtime) often yields better results because it accounts for the "cost of being wrong."
  • XAI as a Discovery Tool: One of the most exciting "aha" moments is using XAI to simplify models. Instead of running a heavy neural network in production, researchers use XAI to identify the three most important algebraic features and then write a tiny, lightning-fast C++ function based on those features.
  • The "Unique Selling Point" Conflict: CA's value lies in 100% accuracy. Generative ML (like Transformers) can solve problems traditional systems can't, but they can "hallucinate" incorrect math. This creates a tension between the power of ML and the rigor of CA.

Key Results

  • CAD Optimization: ML-selected variable orderings consistently outperform traditional human-designed heuristics like "Brown's heuristic."
  • Speedups: Transformer-based models have achieved up to a 2x speedup on CAD benchmarks and a 3.5x speedup in border basis computations.
  • Integration: In the SymPy library, ML-guided integration led to 6x fewer steps to find a solution.
  • Direct Computation: Transformers have shown the ability to predict squared amplitudes in high-energy physics correctly 97–99% of the time, running two orders of magnitude faster than symbolic methods.

Limitations and Future Work

The field is still in its infancy. The authors identify several hurdles:

  • Generalizability: Models trained on synthetic data often perform poorly on real-world engineering problems.
  • Dataset Scarcity: There is no "ImageNet" for most areas of Computer Algebra. The community needs unified, large-scale benchmarks, particularly for Gröbner bases.
  • Trust and Verification: For "ML to generate" approaches, we need better ways to efficiently verify that a model's symbolic output is actually correct.
  • Neuro-Symbolic Integration: The ultimate goal is a tighter loop where ML provides the "intuition" for the search space, and CA provides the "rigor" to guarantee the result.

Summary updated

Abstract

This survey explores the use of Machine Learning (ML) in the field of Computer Algebra (CA), both to optimise existing CA algorithms, and to perform symbolic computation directly. Traditional symbolic methods, while mathematically rigorous, often are computationally expensive thus limiting their use in real-world applications. Recent advances have shown that data-driven techniques can address these limitations by guiding heuristic decisions, selecting optimal algorithms, predicting structural properties of algebraic objects, or even making direct symbolic computations. We take a systematic literature review approach and uncover work in CA applications including cylindrical algebraic decomposition, Gröbner basis computation, symbolic integration, and many more. The survey compares the different ML approaches that have been employed for these tasks, ranging from decision trees to transformers. Issues uncovered by the survey include the lack of benchmark datasets for CA, which hinders the comparison of methods and the generalizability of ML models. The survey identifies the potential for explainable AI tools to help develop trust in decisions, and to drive forward CA research itself.
BibTeX
@misc{shafiq2026survey,
  author = {Uzma Shafiq and Matthew England and Nayyar Zaidi},
  title = {{A Survey on the Use of Machine Learning within Computer Algebra}},
  year = {2026},
  eprint = {2611.23361},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23361},
}
Figure