GMLS-Nets: A Machine Learning Framework for Unstructured Data

Nathaniel Trask, Ravi G. Patel, Paul J. Atzberger, Ben J. GrossAAAI 2020 · OpenReview

Introduction

Standard Convolutional Neural Networks (CNNs) have achieved remarkable success in computer vision by exploiting the regular grid structure of image data. However, many scientific and engineering domains generate data on unstructured grids, such as point clouds, irregular meshes, or scattered sensor readings. Adapting CNNs to these "scattered data" formats—a field often termed Geometric Deep Learning—frequently lacks the mathematical rigor and physical interpretability required for scientific applications. Issues such as boundary effects, lack of conservation laws, and poor generalization to different resolutions persist.

GMLS-Nets address these challenges by providing a mathematically rigorous framework to generalize CNNs to unstructured data. By leveraging Generalized Moving Least Squares (GMLS), this architecture learns physically interpretable operators capable of discovering governing equations and breaking traditional stability limits in numerical simulations.

Methodology: The GMLS-Layer

The core innovation of GMLS-Nets is the GMLS-Layer, which replaces the standard convolution operation. Unlike fixed-grid filters, the GMLS-Layer performs a local reconstruction of the underlying field from scattered data points.

Overview of the GMLS-Net architecture, illustrating the GMLS-Layer, scattered data input processing, and application to classification and regression tasks.

The process involves three primary steps:

  1. Local Reconstruction: For a given neighborhood of points, the layer performs a GMLS fit to find an optimal representation (e.g., a polynomial basis) of the local data.
  2. Encoding: The data is compressed into a coefficient vector $a(u)$ that characterizes the local field.
  3. Learnable Operators: The network learns a mapping from these coefficients to the desired output. In linear cases, the weights act as stencils for differential operators. In non-linear cases, a Multi-Layer Perceptron (MLP) processes the coefficients.

This framework can also be integrated into Finite Volume Method (FVM) updates, ensuring that the learned models strictly obey physical conservation laws, such as mass or energy conservation.

Experimental Evaluation and Results

The authors evaluated GMLS-Nets across a diverse range of benchmarks, from synthetic operator regression to complex fluid dynamics.

Operator Regression

A key test of GMLS-Nets is their ability to learn fundamental physical operators. Using synthetic 1D and 2D data generated from random Gaussian fields, the network was trained to predict Laplacians and the non-linear Burger's operator.

GMLS-Net performance on operator regression tasks, showing accurate predictions for 1D/2D Laplacians and the 1D Burgers operator compared to target operators.

As shown in Figure 5, GMLS-Nets accurately reconstruct these operators. Beyond simple accuracy, the learned filters are physically interpretable; for example, in MNIST benchmarks, the network naturally developed horizontal and vertical edge detectors corresponding to $\partial_x$ and $\partial_y$ derivatives.

Fluid Dynamics and Stability

In simulations of flow past a cylinder (RANS), GMLS-Nets predicted drag coefficients with a root-mean-square error (MSE) below 1.5%. Remarkably, the network identified the latent Reynolds number relationship without explicit knowledge of the fluid's viscosity.

Furthermore, GMLS-Nets demonstrated the ability to "break" the Courant-Friedrichs-Lewy (CFL) condition. By learning corrections to the underlying physics, the model maintained stability and accuracy at timesteps ten times larger than those allowed by standard numerical methods.

Physics-from-Noise

The framework also excelled at "automated homogenization." By processing noisy Brownian motion trajectories from molecular dynamics simulations, GMLS-Nets successfully extracted a clean, macroscopic diffusion model, effectively bridging the gap between microscopic noise and continuum-level physics.

Discussion and Conclusion

GMLS-Nets represent a significant step toward "SciML" frameworks that combine the flexibility of deep learning with the rigor of approximation theory. Their design ensures interpretability, as filters correspond to specific derivatives, and allows for native data transfer between different degrees of freedom without interpolation errors.

While highly effective, current limitations include the potential restrictiveness of polynomial bases for complex image features and the computational overhead of local neighbor searches in large-scale 3D datasets. Future work will explore alternative dictionary bases and integration with Physics-Informed Neural Networks (PINNs) to further constrain local operator learning with global physical laws.

Summary updated

BibTeX
@inproceedings{trask2020gmls,
  author = {Nathaniel Trask and Ravi G. Patel and Paul J. Atzberger and Ben J. Gross},
  title = {{GMLS-Nets: A Machine Learning Framework for Unstructured Data}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  url = {https://openreview.net/forum?id=NS1dqnGz0_},
}
Figure