Parameter Efficient Geometric Modeling

48 concepts47 papers

Deep learning typically relies on increasing model capacity—adding more layers and parameters—to learn complex spatial relationships from raw data. However, a growing body of research suggests that this approach is often redundant. By explicitly embedding geometric priors, physical laws, and mathematical constraints directly into neural architectures, researchers are achieving state-of-the-art performance with a fraction of the computational cost. This paradigm shift, often termed parameter-efficient geometric modeling, demonstrates that structural awareness is frequently more powerful than raw scale.

Hard-coding Physical Laws with PDE Solvers

One of the most radical departures from standard convolutional stacks is the use of Partial Differential Equations (PDEs) as layer operations. Instead of learning individual pixel weights, these models learn the physical coefficients of evolution equations that govern how information flows through the network.

Pai et al. and Smets et al. demonstrate that by "hard-coding" laws like convection, dilation, and erosion into PDE-G-CNNs, models can achieve a massive 10x to 200x reduction in parameters. Gijs Bellaard provides a rigorous mathematical foundation for this, showing that continuous evolution equations allow for significant parameter reduction without sacrificing accuracy. Furthermore, Daan Bon shows that these networks can be made even more efficient by using sub-Riemannian kernels to restore accuracy in highly anisotropic spaces, matching traditional network performance with 35 times fewer weights.

Classical Geometry as Differentiable Modules

Another effective strategy involves offloading spatial reasoning to classical geometric operations. Rather than forcing a network to learn how to handle perspective distortion or surface fitting, researchers are integrating these established mathematical principles as fixed, differentiable modules.

  • Perspective Correction: Yu et al. introduced Perspective Crop Layers (PCLs), which mathematically "rotate" a virtual camera to look directly at a subject. This removes location-dependent stretching without adding a single parameter to the model.
  • Surface Fitting: Lenssen et al. combined Graph Neural Networks with classical weighted least-squares plane fitting, achieving state-of-the-art surface normal estimation while being 100x faster and significantly more parameter-efficient than black-box alternatives.
  • Multi-view Stereo: Chen et al. integrated traditional plane-sweeping cost volumes into deep learning pipelines for 3D Gaussian Splatting, reducing parameters by 10x compared to purely learned methods.

Manifold-Aware Representations

Standard neural networks operate in Euclidean space, but many geometric problems—such as human motion or medical imaging—exist on non-Euclidean manifolds. Xuan Son Nguyen bridged this gap by using gyrovector space theory to build Recurrent Neural Networks that operate directly on Symmetric Positive Semi-definite (SPSD) matrices. By respecting the underlying Riemannian geometry of the data, these models achieve superior accuracy with a fraction of the parameters required by Euclidean models.

Similarly, Melnyk et al. proposed the Multilayer Geometric Perceptron (MLGP), which replaces flat decision hyperplanes with hyperspheres. By using conformal geometry to embed points, the architecture inherently understands spatial relationships like distance and curvature, requiring fewer learned weights to remain robust against noise and rotation.

Local Structure and Implicit Functions

For tasks like point cloud understanding and image segmentation, focusing on local geometric regularity allows for extreme compression. Wang et al. and Wang et al. utilize Taylor series-inspired convolutions and dynamic polynomial fitting to capture smooth shapes and fine details. These specialized aggregators allow models with only ~2.4M parameters to outperform massive pre-trained transformers.

In the domain of face segmentation, Sarkar et al. leveraged Local Implicit Image Functions (LIIF) to treat parsing as a continuous coordinate-based task. By assuming a predictable spatial layout for human faces, their model achieves state-of-the-art results with 26x fewer parameters than general-purpose networks.

Aligning Architecture with Domain Geometry

Finally, efficiency is often gained by simply aligning the model's discretization with the physical domain. Linander et al. introduced PEAR, a weather forecasting model that replaces "square" grids with HEALPix, a spherical discretization where every pixel covers the exact same surface area. This geometric alignment allows the model to outperform state-of-the-art baselines with a fraction of the computational overhead.

In 3D segmentation, Robert et al. grouped millions of points into a hierarchical superpoint structure. By integrating handcrafted geometric features like linearity and planarity, they achieved state-of-the-art results with 200x fewer parameters and 70x faster training times.

As Ben-Chen et al. noted in their synthesis of the field, bridging the "representation gap" between digital geometry and deep learning is not just about accuracy—it is the primary path toward creating models that are small enough to run in real-time on mobile devices and surgical robots.

Key Foundational Work

Towards Natural and Accurate Future Motion Prediction of Humans and Animals (2019) — contributed 1 concept elements to this direction.
Unsupervised Learning of Monocular Depth and Ego-Motion using Conditional PatchGANs (2019) — contributed 1 concept elements to this direction.
Depth and height aware semantic RGB-D perception with convolutional neural networks (2015) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_parameter_efficient_geometric_modeling_364a4d24,
  title = {parameter-efficient geometric modeling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/parameter-efficient-geometric-modeling/rdr_364a4d247abd4dc3aaa79c88417a073c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_parameter_efficient_geometric_modeling_364a4d24_v1,
  title = {parameter-efficient geometric modeling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/parameter-efficient-geometric-modeling/rdr_364a4d247abd4dc3aaa79c88417a073c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}

See something wrong? Report a mistake