Abstract
This paper presents a novel 3D shape completion framework that leverages adaptive local basis functions to reconstruct full geometry from partial point clouds. Unlike traditional methods that rely on rigid voxel grids or fixed geometric primitives, our approach learns custom-shaped local implicit functions. This "local-to-local" representation enables high-fidelity reconstruction of fine details with significantly reduced computational overhead. Our model achieves state-of-the-art performance on the ShapeNet benchmark, improving Intersection-over-Union (IoU) by 6.2% while requiring an order of magnitude fewer FLOPs than leading voxel-based competitors.
Introduction
3D shape completion—the task of predicting full 3D geometry from a partial scan—is a fundamental challenge in robotics, autonomous driving, and augmented reality. Current methodologies typically face a trade-off between detail and efficiency. Voxel-based methods capture fine details but suffer from exponential memory costs at high resolutions. Conversely, global implicit functions are efficient but often fail to recover thin structures or sharp edges. Local basis function methods, such as Local Deep Implicit Functions (LDIF), offer a middle ground but are often limited by fixed analytical shapes (e.g., Gaussians) that cannot represent complex local geometries.
To address these limitations, we propose a "Goldilocks" solution: a representation that is as detailed as voxels but as efficient as local functions by utilizing learnable, adaptive local basis functions.
Methodology
Our framework is built upon three primary pillars designed to maximize both representational power and computational efficiency:
1. Adaptive Local Basis Functions
Rather than using fixed mathematical primitives, our model learns the basis functions themselves. Each function is a small Deep Implicit Function (DIF) capable of adapting its shape to fit specific local geometric features, such as curved handles or sharp corners. This allows the model to represent complex objects using a sparse set of components (e.g., 128 bases for a chair).
2. RBF-based Domain Learning and Sparsity
We utilize Radial Basis Function (RBF) kernels to define the "influence zone" of each local basis. This ensures that during inference, the model only needs to consider the nearest basis functions for any given spatial point, drastically reducing the computational cost of rendering the final surface.
3. Transformer-based Completion
The completion pipeline operates in two stages. First, a Visible Points Encoder maps the partial input to a set of local bases. Subsequently, a Transformer architecture treats these bases as a sequence, predicting the "missing" bases required to complete the object. This sequence-to-sequence approach enables the model to learn structural correlations, such as the relationship between a visible front chair leg and its hidden counterpart.
Experimental Results
We evaluated our model on the ShapeNet benchmark across 8 common object classes, as well as real-world data from ScanNet and Kinect v2.
Quantitative Performance
Our method significantly outperformed existing state-of-the-art models, including IF-Net and ShapeFormer. As shown in the table below, our approach achieved the highest IoU and F1-scores across the benchmark.
Efficiency and Generalization
Beyond accuracy, our method is notably more efficient. It requires only 1.65 Billion FLOPs, compared to 20.6 Billion for IF-Net and 112.5 Billion for ShapeFormer. Furthermore, the model demonstrated strong generalization capabilities, performing well on object categories not seen during training, which suggests that the local-to-local logic captures fundamental geometric principles rather than simple category-specific memorization.
Conclusion and Future Work
By learning adaptive local bases, we provide a scalable and accurate solution for 3D shape completion. While the model excels at local detail preservation, future work will focus on improving global topology. Integrating high-level graph representations could help enforce global connectivity, ensuring that predicted components (like chair rungs and legs) connect with perfect structural integrity.
Summary updated