Decomposing 3D Geometry into Local Neural Implicit Functions

This direction is split into other directions: part-based neural implicit representation. Its last version stays readable and citable here.

Version 1 of 123 papers26 conceptsfrozen 2026-10-04

In the early days of neural implicit representations, the standard approach was to use a single, global Multi-Layer Perceptron (MLP) to represent an entire 3D object. By mapping a 3D coordinate to a value—like a Signed Distance Function (SDF) or occupancy—these models could represent smooth surfaces with infinite resolution. However, global networks suffer from a "spectral bias," tending to learn low-frequency shapes while smoothing over fine details like wrinkles, sharp edges, or complex internal structures.

To overcome this, researchers have shifted toward part-based and local decomposition. Instead of asking one network to memorize a whole human body or a complex face, the geometry is broken down into a collection of smaller, specialized neural functions. This modularity allows for higher precision, better generalization, and intuitive control over articulation.

The Problem with Global Representations

When a single MLP represents a complex, moving object—such as a person walking—it must learn a high-dimensional mapping that accounts for every possible pose. As Pablo Palafox and Nikolaos Sarafianos observe in SPAMs: Structured Implicit Parametric Models, these global models often "lose" parts of the geometry or create "ghosting" artifacts during fast motion because the network struggles to maintain semantic consistency across the entire volume.

Furthermore, global networks are computationally expensive to refine for high-frequency details. As noted in Level-Set Parameters: Novel Representation for 3D Shape Analysis by Hongdong Li and Andreas Geiger, there is an inherent trade-off between a network's ability to understand global semantics and its ability to capture fine-grained geometric precision.

Decomposition into Local Coordinate Systems

The core intuition behind the recent shift is to represent an object as a union of local parts. Each part is governed by its own small neural network operating in its own local coordinate system.

In NASA Neural Articulated Shape Approximation, Boyang Deng and Geoffrey E. Hinton demonstrate this by tying neural occupancy functions to a skeletal hierarchy. Instead of the network learning how a whole arm moves in world space, a "forearm network" only needs to learn the static shape of a forearm in its own local "bone space." When the skeleton moves, the local coordinate system moves with it, and the global shape is simply the union of these transformed local parts.

This approach was extended to hands in LISA: Learning Implicit Shape and Appearance of Hands by Enric Corona and Minh Vo. By using per-bone neural fields, LISA avoids the "melting" artifacts common in mesh-based models, allowing for independent control over pose and shape while maintaining high-resolution skin details.

Capturing Detail through Local Deformation

Decomposition isn't just for rigid parts; it also enables the capture of non-rigid, high-frequency details like facial expressions. In ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural Representations, Mingwu Zheng and Di Huang disentangle identity and expression into separate deformation fields. By blending local functions, they capture complex, nonlinear details—like the specific way skin bunches during a pout—that traditional linear models miss.

Similarly, Implicit Neural Head Synthesis via Controllable Local Deformation Fields by Chuhan Chen and Matthew O'Toole breaks down a global deformation field into smaller fields centered around facial landmarks. Each local field is managed by a dedicated small-scale network. This allows for asymmetric control—such as winking one eye—without affecting the rest of the face, significantly increasing the expressive capacity of the avatar.

Generalization and Patch-Based Priors

Beyond animation, local decomposition allows models to generalize to entirely new objects by learning "geometric primitives." PatchNets, developed by Ayush Tewari and Michael Zollhöfer, represents 3D shapes as collections of learnable "patches." Because these patches learn local geometric structures (like the curve of a chair leg or the flat surface of a table), the model can reconstruct categories of objects it has never seen before by combining these learned local priors.

This concept is further refined in Neural Part Priors (NPPs) by Alexey Bokhovkin. NPPs use coordinate-based networks to model individual object parts, allowing for high-fidelity completion of messy, incomplete RGB-D scans. By optimizing these part-based latent spaces at test-time, the system ensures that the reconstructed object is both geometrically detailed and structurally logical.

Summary of the Intuition

The transition from global to local neural implicit functions mirrors the transition from global Fourier transforms to local wavelets. By decomposing a complex 3D signal into localized, semantically meaningful components:

  1. Complexity is reduced: Each network solves a simpler, local problem.
  2. Details are preserved: Local networks can focus their capacity on high-frequency features like wrinkles or sharp edges.
  3. Articulation is simplified: Movement is handled by transforming local coordinate systems rather than re-learning the entire shape.
  4. Generalization is improved: Learned local "patches" can be reused to describe new, unseen geometries.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_decomposing_3d_geometry_into_local_neura_986fbbbd,
  title = {Decomposing 3D Geometry into Local Neural Implicit Functions},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/decomposing-3d-geometry-into-local-neural-implicit-functions/rdr_986fbbbd09e044b7bc9de73552d79b64},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_decomposing_3d_geometry_into_local_neura_986fbbbd_v1,
  title = {Decomposing 3D Geometry into Local Neural Implicit Functions},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/decomposing-3d-geometry-into-local-neural-implicit-functions/rdr_986fbbbd09e044b7bc9de73552d79b64/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}