TL;DR
Pointfilter is a novel end-to-end deep learning framework designed to clean up noisy 3D point clouds while meticulously preserving sharp geometric features. By using an encoder-decoder architecture to regress displacement vectors, it eliminates the need for the tedious manual parameter tuning required by traditional optimization-based filtering methods.
What Problem Does This Paper Address?
3D scanning—whether from consumer-grade depth sensors or high-end LIDAR—inevitably produces "noisy" point clouds. To use this data for downstream tasks like 3D printing, autonomous driving, or digital archiving, the noise must be filtered out.
The challenge lies in the "smoothing vs. sharpening" trade-off. Traditional algorithms (like LOP or RIMLS) often require users to manually tune complex parameters; get them wrong, and you either leave noise behind or "melt" the sharp edges of your model. Early deep learning attempts (like EC-Net or PCN) struggled with generalization or required manual labeling of sharp edges. The authors of Pointfilter sought a method that is automatic, robust to different noise levels, and capable of keeping edges crisp without human intervention.
Methods
The authors propose a point-wise learning architecture called Pointfilter, which treats denoising as a regression problem.
- Encoder-Decoder Architecture: The network takes a noisy point and its local neighbors as input. The Encoder (based on the PointNet architecture) extracts a latent representation of the local geometry. The Decoder then regresses a displacement vector—essentially a "push" that moves the noisy point back to its rightful place on the underlying clean surface.
- PCA Alignment: To ensure the network is invariant to rotation (so a chair looks like a chair regardless of its orientation), the authors use Principal Component Analysis (PCA) to align each local patch to a canonical coordinate system before feeding it into the network.
- The "Secret Sauce" Loss Function: The core innovation is a bilateral projection loss ($L^b_{proj}$). Unlike a standard $L_2$ loss which just minimizes distance, this loss incorporates normal similarity. During training, it uses ground-truth normals to ensure points are projected onto the correct surface patches, preventing the "blurring" of sharp corners.
- Repulsion Term: To prevent points from bunching up in one spot (which creates "clumpy" point clouds), they include a repulsion term ($L_{rep}$) that encourages a uniform distribution of points.
Datasets
- Training: A curated dataset of 22 3D models (11 CAD-style with sharp edges and 11 organic/non-CAD models). These were augmented with six levels of synthetic Gaussian noise (0% to 2.5% of the bounding box diagonal).
- Evaluation:
- Synthetic: 15 unseen 3D models with varying noise levels.
- Real-World Scans: Data from the KinectV1/V2 sensors and the Paris-rue-Madame database (large-scale urban scans).
- Benchmarks: Compared against traditional methods (RIMLS, GPF, WLOP) and deep learning competitors (EC-Net, PCN, TotalDenoising).
Key Insights and Ideas
- Normals are for Training, Not Inference: One of the biggest "aha" moments is that while the network needs ground-truth normals during training to learn what a sharp edge looks like, it does not need them at inference time. This makes the tool much more practical for raw scan data where reliable normals are unavailable.
- Displacement over Coordinates: Instead of predicting new coordinates directly, the network predicts a displacement. This makes the learning task more stable and easier for the network to converge.
- Local over Global: By focusing on local patches rather than the whole object, the model generalizes remarkably well to complex shapes it has never seen before, including organic faces and architectural details.
Key Results
- Quantitative Superiority: In tests on 15 synthetic models, Pointfilter achieved the lowest errors across three major metrics: Chamfer Distance (CD), Mean Square Error (MSE), and Point-to-Surface (P2F) distance. For instance, its CD was roughly 12% lower than the next best deep learning method (PCN).
- Visual Fidelity: Qualitatively, Pointfilter is significantly better at preserving 90-degree angles and fine ridges (see Figures 6 and 11) where other methods tend to create rounded "fillets" or leave stray outliers.
- Efficiency: While not the absolute fastest (EC-Net holds that title), Pointfilter is roughly 6x faster than PCN on standard models and avoids the "trial-and-error" time sink of manual tuning.
Limitations and Future Work
The authors acknowledge two primary hurdles:
- Extreme Noise: When noise levels are excessively high (e.g., noise that completely obscures the underlying structure), the network struggles to distinguish features from artifacts.
- Large Holes: Because the method is "local" (looking only at a point's immediate neighbors), it cannot "hallucinate" or fill in large missing chunks of a scan.
Future Work: The team suggests incorporating global shape information into the framework. By combining local precision with a "big picture" understanding of the object, the next iteration could potentially fill holes and handle even noisier environments.
Summary updated