Evaluating the squared-exponential covariance function in Gaussian processes with integral observations

Johannes N. Hendriks, Carl Jidling, Adrian Wills, Thomas B. SchönarXiv 2018 · OpenReview

Abstract

Gaussian Processes (GPs) are powerful tools for modeling unknown functions, particularly due to their closure under linear operators. This property allows GPs to handle integral observations, such as those found in X-ray tomography or fiber optic strain measurements. However, calculating the covariance between integral measurements requires solving a double line integral of the kernel. For the widely used Squared-Exponential (SE) kernel, no closed-form solution exists. This paper presents a numerically efficient and robust method that reduces this 2D problem to a 1D numerical integration by analytically solving one dimension using the error function (erf). Our method achieves significantly higher accuracy and speed than traditional 2D numerical integration or bivariate normal approximations, particularly in challenging co-linear scenarios.

Introduction

The core challenge in using GPs with integral observations is the computation of the double line integral of the covariance function. While the SE kernel is a standard choice in GP research, its double integral lacks an analytical solution. Researchers typically rely on 2D numerical integration (like Simpson’s rule), which can be computationally expensive, or approximations that often fail when measurement lines are parallel or co-linear. This paper addresses these limitations by proposing a hybrid analytical-numerical approach.

Methodology

The proposed method leverages partial analytical integration to simplify the problem:

  1. Dimension Reduction: By rewriting the double integral of the SE kernel, the authors identify that the inner integral can be solved analytically using the error function (erf).
  2. 1D Numerical Integration: The reduction transforms the 2D problem into a 1D integral, which can be solved rapidly using standard techniques like Gauss-Kronrod quadrature.
  3. Numerical Stability: To prevent rounding errors at large distances, the implementation avoids splitting exponents into separate products.
  4. Robustness: The framework includes logic to handle edge cases, such as zero-length measurement lines, ensuring stability across diverse geometries.

Experimental Setup

The method was evaluated using eight synthetic test sets, each comprising 10,000 pairs of line integrals in a 6-dimensional space. These sets were designed to test various geometric conditions:

  • Set 1: Standard random lines.
  • Set 2: Almost co-linear (parallel) lines—the primary failure point for previous bivariate normal methods.
  • Set 3: Randomly scaled dimensions.
  • Sets 4 & 7: Cases that reduce to single integrals.
  • Sets 5 & 6: Large integral intervals with varying scaling.
  • Set 8: Cases reduced to point observations (no integral).

Key Results and Discussion

The proposed method was benchmarked against 2D Simpson’s rule, the Bivariate Normal method, and Matlab’s built-in integral2.

Table 1: Mean absolute error comparison between the proposed method and existing numerical techniques across eight test sets.

As shown in Table 1, the proposed method maintains an extremely low error floor across nearly all scenarios. A critical finding is the method's performance in the "almost co-linear" case (Set 2). Previous state-of-the-art methods transformed the problem into a bivariate normal distribution, which requires inverting a matrix that becomes singular when lines are parallel. The proposed 1D reduction avoids this inversion entirely. Consequently, while the Bivariate Normal method produces a mean error of $1.84 \times 10^{-1}$ in Set 2, the proposed method achieves a precision of $6.39 \times 10^{-16}$.

In terms of computational efficiency, the proposed method averaged 1.21 x 10⁻⁵ seconds per integral, outperforming Matlab’s integral2 by more than 100 times.

Figure 1: Histograms showing the distribution of errors for the proposed method across different test scenarios.

The error distributions illustrated in Figure 1 further confirm the robustness of the approach. Unlike Simpson's rule, which requires increasing computational cost (more "bins") to maintain accuracy as complexity grows, the 1D reduction method provides consistent, high-precision results with near-constant overhead.

Conclusion

By combining analytical derivation with efficient 1D numerical integration, this paper provides a robust solution for GPs with integral observations using the SE kernel. While currently specific to the SE kernel, the approach sets a precedent for handling other kernels. Future work may involve extending this derivation to Matérn or periodic kernels and implementing adaptive scaling to further refine accuracy in poorly scaled datasets.

Summary updated

Abstract

This paper deals with the evaluation of double line integrals of the squared exponential covariance function. We propose a new approach in which the double integral is reduced to a single integral using the error function. This single integral is then computed with efficiently implemented numerical techniques. The performance is compared against existing state of the art methods and the results show superior properties in numerical robustness and accuracy per computation time.
BibTeX
@misc{hendriks2018evaluating,
  author = {Johannes N. Hendriks and Carl Jidling and Adrian Wills and Thomas B. Schön},
  title = {{Evaluating the squared-exponential covariance function in Gaussian processes with integral observations}},
  year = {2018},
  eprint = {1812.07319},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1812.07319},
}
Figure