Barron Spaces for High-Dimensional Function Approximation

This direction is merged into another direction: curse of dimensionality (high-dimensional function approximation). Its last version stays readable and citable here.

Version 1 of 130 papers40 conceptsfrozen 2026-10-04

In classical numerical analysis, approximating a function in high dimensions is notoriously difficult. This is the "curse of dimensionality": to maintain a fixed level of accuracy using traditional methods like splines or polynomials, the number of required parameters typically grows exponentially with the number of input dimensions $d$. If you need 10 points to approximate a 1D function, you might need $10^{100}$ points for a 100-dimensional one.

Yet, neural networks routinely solve problems in thousands of dimensions. A growing body of theoretical work suggests that neural networks succeed because they operate within specific function classes—most notably Barron spaces—where the approximation error depends on the number of neurons rather than the spatial dimension.

The Limits of Sobolev Smoothness

To understand why Barron spaces matter, we must first look at why traditional frameworks fail. In classical analysis, we often use Sobolev spaces, which classify functions based on how many derivatives they have. If a function is "smooth" (has many derivatives), it is easier to approximate.

However, as Hrushikesh N. Mhaskar and Charles A. Micchelli noted in How to Choose an Activation Function, even smooth functions in Sobolev spaces suffer from the curse of dimensionality. The number of parameters needed to achieve an error $\epsilon$ scales as $\epsilon^{-d/k}$, where $k$ is the smoothness. As $d$ grows, this rate becomes unusable.

The Barron Insight: Dimension-Free Rates

The breakthrough came from Andrew R. Barron, who identified a class of functions that neural networks can approximate efficiently. In Uniform Approximation by Neural Networks Activated by First and Second Order Ridge Splines, Barron and Klusowski showed that if a function has a specific Fourier representation—essentially, if its Fourier transform decays sufficiently fast—a two-layer neural network with $m$ neurons can approximate it with an error of $O(m^{-1/2})$.

Crucially, this rate is independent of the dimension $d$. The intuition is that neural networks do not try to fill the entire high-dimensional volume (like a grid). Instead, they use "ridge functions" (functions that only vary along one direction, like $f(wx+b)$) to pick out the most important directions in the data.

Formalizing the Barron Space

While Barron's original work focused on Fourier transforms, researchers like E Weinan and Stephan Wojtowytsch formalized this into a rigorous functional framework. In Representation formulas and pointwise properties for Barron functions, they define the Barron space as the set of functions that can be represented as an integral over ridge functions:

$$f(x) = \int_{\Omega} a(\omega) \sigma(\omega \cdot x) d\pi(\omega)$$

This representation views a neural network not as a collection of discrete weights, but as a continuous distribution of neurons. A function belongs to the Barron space if it has a finite "Barron norm," which measures the complexity of this distribution. If the norm is finite, the function is "simple enough" for a neural network to learn without hitting the curse of dimensionality.

Spectral Barron Spaces and Physics

This theory has profound implications for scientific computing. Many physical systems, such as the Schrödinger equation in quantum mechanics, involve hundreds of dimensions (one for each particle coordinate). Traditional grid-based solvers fail here, but neural networks often succeed.

Jianfeng Lu and Yulong Lu have extended Barron's ideas to these problems using Spectral Barron spaces. In A priori generalization error analysis of two-layer neural networks for solving high dimensional Schrödinger eigenvalue problems, they prove that if the potential function of a system lies in a Spectral Barron space, the ground state solution can be found by a neural network with an error rate that ignores the number of particles.

Similarly, in A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Partial Differential Equations, they show that the Deep Ritz Method—a popular neural PDE solver—is effective precisely because the solutions to many elliptic PDEs naturally reside in these low-complexity Barron spaces.

Beyond Approximation: Generative Models

The utility of Barron spaces extends beyond solving equations to understanding modern generative AI. Score-based generative models (like diffusion models) learn to map noise to data by estimating the "score" (the gradient of the log-density).

In Score-based generative models break the curse of dimensionality in learning a family of sub-Gaussian probability distributions, Frank Cole and Yulong Lu demonstrate that if the log-density of a distribution belongs to a Barron space, the model can learn to generate samples with a complexity that does not explode with the data's dimensionality. This provides a mathematical explanation for why diffusion models can generate high-resolution images (thousands of dimensions) using a manageable amount of training data.

Summary of the Mechanism

The reason Barron-type spaces "break" the curse of dimensionality is a shift in perspective:

  1. From Grids to Integrals: Instead of measuring smoothness by local derivatives (Sobolev), we measure it by the global integrability of the function's representation (Barron).
  2. Directional Sparsity: Neural networks exploit the fact that many high-dimensional functions are "ridge-like"—they vary significantly only in a few directions or along specific manifolds.
  3. Monte Carlo-like Error: The $O(m^{-1/2})$ error rate is reminiscent of Monte Carlo integration. Just as Monte Carlo integration error is independent of dimension, neural network approximation error becomes dimension-independent when the target function is "integrable" in the Barron sense.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_barron_spaces_for_high_dimensional_funct_c5e859a0,
  title = {Barron Spaces for High-Dimensional Function Approximation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/barron-spaces-for-high-dimensional-function-approximation/rdr_c5e859a01e2c429db03503604c6d8fa5},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_barron_spaces_for_high_dimensional_funct_c5e859a0_v1,
  title = {Barron Spaces for High-Dimensional Function Approximation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/barron-spaces-for-high-dimensional-function-approximation/rdr_c5e859a01e2c429db03503604c6d8fa5/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}