Neurosymbolic AI as an antithesis to scaling laws

Alvaro Velasquez, Neel P. Bhatt, Ufuk Topcu, Zhangyang Wang, Katia Sycara, Simon Stepputtis, Sandeep Neema, Gautam K. VallabhaPNAS 2025 · DOI

TL;DR

This paper argues that the current AI paradigm, heavily reliant on scaling laws (more data, more compute, bigger models), is unsustainable and creates barriers to entry. They propose neurosymbolic AI, which integrates symbolic reasoning with neural networks, as a more efficient, interpretable, and accessible alternative that can achieve comparable or superior performance with significantly fewer resources.

What Problem Does This Paper Address?

The paper tackles the growing concerns surrounding the current trajectory of AI development, primarily driven by "scaling laws." These laws suggest that increasing model size, data, and computational power leads to improved performance. While this has yielded impressive results, it has also led to:

  • Prohibitive Costs: Training and deploying state-of-the-art models like large language models (LLMs) require immense financial and energy resources, limiting access to a few large tech companies.
  • Computational Sustainability: The energy consumption and carbon footprint of these models are significant and unsustainable in the long run.
  • Monolithic Perspective: An overreliance on a single approach (scaling) stifles innovation and creates a "gatekeeping" effect in AI research.
  • Lack of Interpretability and Trust: Opaque, closed-source models raise concerns about their reliability and trustworthiness.

The authors contend that this "bitter lesson" of relying solely on computation is not the only path forward and that human cognition offers a more efficient model.

Methods

The core of the paper's proposal is neurosymbolic AI, which aims to bridge the gap between data-driven neural networks and symbolic reasoning. This integration can occur in two main ways:

  • Symbols to Neurons: Incorporating human domain knowledge, such as logic rules or physical laws (e.g., differential equations), as inductive biases into neural networks. This can guide learning, improve data efficiency, and make models more robust. Examples include physics-informed neural networks and knowledge graphs.
  • Neurons to Symbols: Extracting symbolic representations from trained neural networks. This "symbolic distillation" can lead to smaller, more interpretable models that retain much of the performance of their larger neural counterparts. Techniques like quantization and grammatical inference are mentioned.

The paper envisions a feedback loop where neural representations are "lifted" to symbolic ones for reasoning, and then projected back to neural space. This allows for efficient traversal of complex objective function landscapes, avoiding local optima and reducing the need for massive datasets and parameters.

Datasets

Not discussed in the paper. The paper focuses on the conceptual framework and potential of neurosymbolic AI rather than presenting empirical results on specific datasets.

Key Insights and Ideas

  • Human Cognition as a Model: The human brain achieves remarkable intelligence with very low power consumption (around 20W), demonstrating that extreme scaling is not the only path to efficiency.
  • The "Sharpness" Problem: Neural networks struggle with "sharp" regions in their objective function landscapes, requiring vast amounts of data and parameters to smooth them out. Neurosymbolic AI offers a way to navigate these sharp regions using symbolic reasoning.
  • Complementarity, Not Competition: Neurosymbolic AI is presented not as a replacement for scaling laws but as a complementary approach that can mitigate their downsides.
  • Democratizing AI: By reducing computational and data requirements, neurosymbolic AI can lower the barrier to entry for researchers and developers outside of large tech corporations.
  • Interpretability and Trust: Integrating symbolic components can make AI models more transparent and understandable, fostering greater trust.
  • Compositional Inductive Bias: Symbolic knowledge provides a compositional inductive bias, allowing agents to generalize to novel situations more effectively, much like how a small set of rules can describe many scenarios.

Key Results

The paper does not present new quantitative results but rather synthesizes existing research and proposes a vision. It references studies that show:

  • Neurosymbolic models can be significantly smaller (e.g., 95% smaller) than purely neural baselines while achieving comparable or superior performance.
  • Neurosymbolic approaches can achieve superior common sense reasoning performance with models that are orders of magnitude smaller than large LLMs.
  • One neurosymbolic system for autonomous driving matched state-of-the-art performance using 0.1% of the training time and 1% of the training data, requiring 96.9% fewer parameters.

Limitations and Future Work

  • Scalability of Symbolic Methods: While neurosymbolic AI aims to reduce scale, the efficiency and scalability of symbolic reasoning itself in complex, high-dimensional problems remain an area for exploration.
  • Discovery vs. Refinement: The paper acknowledges the "bitter lesson" that general-purpose learning can outperform handcrafted knowledge, but argues that integrating human knowledge is not mutually exclusive with discovery. The challenge lies in finding the right balance.
  • Practical Implementation: While the conceptual framework is strong, the practical implementation and widespread adoption of neurosymbolic methods require further research and development.
  • Future Work: The authors advocate for a concerted, interdisciplinary effort to develop neurosymbolic AI, emphasizing the need to balance scaling laws with efficiency and trustworthiness. They suggest exploring how symbolic reasoning can guide neural learning and how symbols can be discovered and refined from data-driven representations.

Summary updated

Abstract

The recent progress in machine learning has shifted the trends in artificial intelligence (AI) toward an overreliance on increasing amounts of data, computing power, and model parameters. These trends have resulted in success, but have also created a monolithic perspective for AI, increased the barriers to entry outside of large tech companies, and raised concerns about computational sustainability. Neurosymbolic AI is a growing area that promotes methodological heterogeneity and aims to push the frontiers of AI through affordable data and computing power.
BibTeX
@inproceedings{velasquez2025neurosymbolic,
  author = {Alvaro Velasquez and Neel P. Bhatt and Ufuk Topcu and Zhangyang Wang and Katia Sycara and Simon Stepputtis and Sandeep Neema and Gautam K. Vallabha},
  title = {{Neurosymbolic AI as an antithesis to scaling laws}},
  booktitle = {PNAS},
  year = {2025},
  doi = {10.1093/pnasnexus/pgaf117},
}
Figure