Localized Neural Implicit Functions for Complex Geometry

20 concepts16 papers

Neural implicit representations—which model 3D surfaces as the level sets of continuous functions like Signed Distance Functions (SDFs) or occupancy fields—have revolutionized 3D reconstruction. However, a single "global" Multi-Layer Perceptron (MLP) often struggles to represent complex, large-scale scenes or highly detailed articulated objects. The research direction emerging from these papers proposes a modular alternative: decomposing global geometry into a collection of localized neural implicit functions.

The Limits of Global Implicit Functions

A monolithic neural network representing an entire object faces a fundamental trade-off between capacity and smoothness. Because MLPs tend to learn low-frequency components first (a phenomenon often called spectral bias), a global model often "smooths over" fine-grained details like wrinkles, sharp edges, or small extremities. Furthermore, global models struggle to generalize; if a network learns a specific human pose, it cannot easily adapt to a new pose because the entire coordinate space is coupled.

To solve this, researchers have moved toward part-based or patch-based representations. By breaking a shape into smaller, overlapping, or semantically meaningful regions, each sub-network only needs to solve a much simpler geometric problem.

Spatial Decomposition and Scalability

One approach to localization is purely spatial. Instead of one network for a whole room, the scene is divided into a grid or a set of local patches. Rohan Chabra et al. introduced Deep Local Shapes (DeepLS), which decomposes scenes into a voxel grid where each cell contains a latent code decoded by a shared, small-scale MLP. This allows for high-detail reconstruction and scalability that monolithic models cannot match.

Similarly, Ayush Tewari et al. proposed PatchNets, which represents shapes as a collection of learnable patches in a canonical space. Because these patches are local, the model can learn generic geometric primitives that generalize across different object categories, requiring significantly less training data than global alternatives. For pose estimation, Daniel Barath et al. showed that regressing 3D coordinates relative to compact surface fragments, rather than the whole object, simplifies the learning task and improves precision for symmetric or texture-less objects.

Articulation through Piecewise Occupancy

For dynamic objects like humans, localization is often tied to a skeletal structure. Boyang Deng et al. and Mohammad Norouzi et al. developed NASA (Neural Articulated Shape Approximation), which assigns independent neural occupancy functions to specific bones.

This piecewise architecture works because:

  1. Local Rigidity: Each network only learns the geometry of a single part (e.g., an upper arm) in its own bone-relative coordinate system.
  2. Compositionality: The global shape is simply the union of these local fields.
  3. Generalization: When the skeleton moves, the local fields move with it, allowing the model to represent unseen poses without geometric distortion.

Shenhan Qian et al. extended this with UNIF, using a "Partition-from-Motion" strategy to reconstruct clothed humans. By ensuring each sub-network only represents locally rigid geometry, they prevent the surface artifacts that typically plague global models during extreme pose extrapolation. Bangbang Yang et al. further demonstrated that this modularity is essential for capturing fine details in extremities like hands and feet, which are often lost in global latent representations.

Capturing High-Frequency Detail

Localization also enables the capture of non-linear, high-frequency details that are spatially restricted. In facial modeling, Mingwu Zheng et al. showed that adaptively blending local implicit functions allows for the representation of complex expressions that linear models miss. Chuhan Chen pushed this further by decomposing a global deformation field into local deformation fields centered around facial landmarks. This allows for asymmetric control—such as winking one eye—and the capture of localized micro-structures like skin creases.

For dynamic modeling, Boyan Jiang et al. introduced LoRD, which uses thousands of overlapping local parts to decouple global skeletal motion from fine-grained surface deformations like clothing wrinkles. By anchoring local neural fields to spatial keypoints, as seen in the NPHM framework by Simon Giebenhain et al., researchers can reconstruct complete heads, including complex hair structures, from sparse and noisy data.

Structural Interpretability and Composition

Finally, part-based decomposition offers a path toward interpretable and editable 3D models. Feng Liu 0037 et al. utilized branched implicit functions to learn Part Embedding Vectors (PEVs), enabling the model to identify semantic correspondences and even recognize when a part (like a chair arm) is missing.

Taking this to the level of "shape programs," Boyang Deng et al. developed ProGRIP, which learns to represent objects as collections of repeatable, neural implicit parts. This allows the model to discover structural regularities—such as the fact that a chair has four identical legs—without manual supervision. Similarly, Qimin Chen et al. showed with DAE-Net that combining instance-specific transformations with per-part neural deformations allows for fine-grained, unsupervised co-segmentation across diverse shape collections.

In scene-level tasks, Qianyi Wu et al. demonstrated in ObjectSDF++ that treating a scene as a composition of individual object-centric SDFs, rather than a single surface, prevents structural blurring and results in much cleaner object boundaries.

Key Foundational Work

NASA Neural Articulated Shape Approximation (2019) — contributed 1 concept elements to this direction.
PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations (2020) — contributed 1 concept elements to this direction.
Learning Implicit Functions for Topology-Varying Dense 3D Shape Correspondence (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_part_based_neural_implicit_representatio_262216f0,
  title = {part-based neural implicit representation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/part-based-neural-implicit-representation/rdr_262216f065964d85b804067c2cfcc35a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_part_based_neural_implicit_representatio_262216f0_v1,
  title = {part-based neural implicit representation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/part-based-neural-implicit-representation/rdr_262216f065964d85b804067c2cfcc35a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}

See something wrong? Report a mistake