In many areas of machine learning, from physics-informed modeling to computer vision, we are faced with a fundamental bottleneck: calculating the integral of a neural network. Whether we are computing the volume rendering integral in a Neural Radiance Field (NeRF) or the cumulative intensity of a temporal point process, we typically rely on numerical quadrature—sampling the network at many points and summing the results. This process is computationally expensive, introduces approximation errors, and often scales poorly with dimensionality.
A growing body of research proposes a paradigm shift: instead of training a network to represent a signal and then integrating it, we can train a network to represent the antiderivative of that signal. By leveraging the Fundamental Theorem of Calculus, integration becomes a simple matter of evaluating the network at the boundaries of the interval.
The Fundamental Theorem of Calculus as an Architecture
The core intuition behind this research direction is to bypass numerical integration by design. In a standard coordinate-based network, we might model a function $f(x)$. To find the integral over $[a, b]$, we must sample $f(x)$ many times. Lindell et al. (AutoInt) and Zhou and Yu (Auto-NPP) propose modeling the antiderivative $F(x)$ directly.
If a neural network $G_\theta(x)$ represents the antiderivative, then the definite integral is simply: $$\int_a^b f(x) dx = G_\theta(b) - G_\theta(a)$$ To ensure that $G_\theta$ actually corresponds to the target signal $f(x)$, the authors use automatic differentiation. During training, the network is optimized such that its derivative $\frac{d}{dx}G_\theta(x)$ matches the observed data or the target intensity. At inference time, the integral is computed in constant time—requiring only two network evaluations regardless of the interval's size.
Exact Likelihoods for Point Processes
This "integral-first" approach is particularly powerful for modeling event data, such as earthquakes or neural spikes. In Spatiotemporal Point Processes (STPPs), calculating the likelihood of an event sequence requires integrating an intensity function over time and space. Traditional methods often buckle under this requirement, relying on slow Monte Carlo approximations.
Khajehnejad et al. and Mukherjee et al. demonstrate that by parameterizing the cumulative intensity function as a neural network, one can bypass intractable integrals entirely. Zhou and Yu extend this to higher dimensions with ProdNet, a dual-network architecture that enables exact triple integrals over space and time. By ensuring mathematical consistency between the intensity (the derivative) and the cumulative total (the network output), these models achieve state-of-the-art accuracy without the numerical "noise" inherent in sampling.
Architecting for Integrability
While some methods use automatic differentiation to link a network to its derivative, others design the network architecture itself to be analytically integrable.
- Fixed-Weight Integration: Subr introduced Q-NET, which leverages the analytical properties of sigmoidal activation functions. By knowing the weights of a shallow proxy network, a second "fixed-weight" network can be constructed to evaluate the exact integral over arbitrary dimensions without further training.
- Stein Operators: Ott et al. propose Bayesian Stein Networks (BSNs). By embedding a Stein operator into the final layer of a network, the authors ensure the model is analytically integrable by construction. This allows the integral of a complex, high-dimensional function to be represented as a single learnable weight, providing both speed and rigorous uncertainty estimates.
- Piecewise Polynomials: In the context of Partial Differential Equations (PDEs), Gao et al. utilize piecewise polynomial basis functions within a graph neural network. This allows for exact Gaussian quadrature, eliminating the "variational crimes" (integration errors) that typically plague physics-informed neural networks.
Beyond Point-Wise Sampling
The shift toward neural antiderivatives also changes how networks perceive space. In volume rendering, standard NeRFs sample points along a ray. However, Zhang et al. (PrecNeRF) argue that point-wise sampling ignores the variation within a segment. By providing the network with interval information and training it to model the mathematical remainder of the integral, they achieve sharper textures and more accurate geometry with fewer samples.
Similarly, Li et al. apply this logic to variance reduction. By training a neural network to approximate the antiderivative of an integrand, they create highly flexible neural control variates. Because the integral of the control variate is known analytically (via the antiderivative network), it can be used to significantly reduce the variance of Monte Carlo estimators in complex simulations like the Walk-on-Sphere algorithm.
Summary of the Intuition
The transition from numerical integration to neural antiderivative representation rests on a simple trade-off: we move the computational burden from inference to training. We spend effort during training to ensure the network's derivative matches our target signal. In exchange, we gain the ability to compute integrals exactly and instantly, transforming a linear-time sampling problem into a constant-time evaluation.
Key Foundational Work
AutoInt: Automatic Integration for Fast Neural Volume Rendering (2020) — contributed 2 concept elements to this direction.
Q-NET: A Network for Low-dimensional Integrals of Neural Proxies (2021) — contributed 2 concept elements to this direction.