Abstract
Point-based representations are fundamental to geometric data structures, yet extracting structured, semantically meaningful information from unstructured point clouds remains a significant challenge. We present PDT (Point Distribution Transformation), a novel framework that leverages diffusion models to transform raw geometric distributions into structured outputs. By utilizing a Diffusion Transformer (DiT) architecture coupled with a Point Voxel CNN (PVCNN), PDT effectively correlates source and target distributions through a denoising process. Our method introduces an exponential noise schedule and Sampling Gradient Adjustment (SGA) to ensure high precision in point placement. Extensive experiments demonstrate that PDT successfully generates surface-aligned keypoints, internal skeletal joints, and continuous feature lines, outperforming existing deterministic and generative baselines in both geometric accuracy and semantic relevance.
Introduction
In 3D geometry processing, the ability to extract high-level "blueprints"—such as skeletal joints for animation or feature lines for design—from raw point clouds is essential. Traditionally, this task has been approached using deterministic feed-forward networks that regress point coordinates directly. However, point clouds are inherently unordered and irregular, and the structures they represent are often multimodal; for instance, there may be multiple valid configurations for a character's skeletal joints.
Deterministic models frequently struggle with this ambiguity, resulting in "averaged" outputs that lack local geometric fidelity. PDT addresses this by framing structural extraction as a distribution-to-distribution mapping problem. By treating the target structure as a probability distribution, PDT can capture complex geometric relationships and produce "artist-like" results that are both precise and semantically coherent.
Methodology
Architecture: PVCNN and DiT
PDT utilizes a hybrid architecture designed to capture both local and global geometric context. A Point Voxel CNN (PVCNN) serves as the feature extractor, processing the input "reference" cloud to provide local geometric guidance. These features are then integrated into a Diffusion Transformer (DiT), which manages the global denoising process. Unlike previous generative models that rely on a single global shape vector, PDT maintains an explicit per-point correspondence between the noisy trajectory and the reference surface, ensuring that the transformation remains grounded in the source geometry.
Precision through Exponential Noise Schedules
A critical observation in the development of PDT was that standard linear noise schedules—common in image-based diffusion—are ill-suited for sparse 3D structures. In image generation, minor noise at the end of the diffusion process results in slight blurring; in 3D keypoint generation, it results in "fuzzy" clusters that fail to align with specific vertices. PDT introduces an exponential noise schedule that rapidly reduces noise levels as the process approaches $t \to 0$. This allows points to converge into tight, mathematically precise clusters necessary for downstream mesh operations.
Sampling Gradient Adjustment (SGA)
To further refine the output during inference, PDT employs Sampling Gradient Adjustment (SGA). This technique applies a correction gradient to the denoising steps, nudging points toward specific geometric targets based on the task. For surface-aligned tasks like remeshing, SGA pushes points toward the input mesh surface; for skeletal tasks, it directs points toward the medial axis (the volumetric center) of the shape.
Experimental Results
Remeshing and Keypoint Extraction
Using the Objaverse dataset, PDT was evaluated on its ability to propose vertex locations for remeshing. In a user study involving 24 participants, PDT’s results were preferred over 74% of the time compared to traditional methods like Quadratic Error Metrics (QEM) or Spectral remeshing. Quantitatively, PDT achieved a lower Chamfer Distance (2.82 vs. 3.11 for QEM) while maintaining a more efficient vertex count, demonstrating its ability to capture semantic importance better than purely geometric algorithms.
Skeletal Rigging
On the ModelsResource RigNetv1 dataset, PDT was tasked with predicting internal skeletal joints for articulated characters. PDT significantly outperformed specialized models like RigNet, achieving a joint Intersection over Union (IoU) of 57.5% (compared to RigNet’s 47.0%) and a precision of 84.8% (compared to 70.4%). The generative nature of the model allowed it to handle diverse character topologies—from humans to multi-legged animals—with high consistency.
Feature Line Extraction
Evaluated on the GarmentCode dataset, PDT demonstrated robustness in extracting continuous feature lines from 3D clothing. Notably, the model showed strong generalization capabilities, successfully processing high-density inputs (up to 8,192 points) even when trained on significantly lower-density data. This suggests that the learned distribution transformation is scale-invariant and robust to sampling noise.
Discussion and Limitations
While PDT represents a significant step forward in point distribution transformation, it is computationally intensive. Training the remeshing model required approximately seven days on eight NVIDIA A6000 GPUs. Additionally, the current framework relies on post-process thresholding to cluster generated points into final keypoints. Future work will explore end-to-end connectivity learning to allow the model to simultaneously predict both point locations and their underlying topological graph (e.g., bone connectivity in a skeleton).
Conclusion
PDT offers a powerful new framework for 3D geometry processing by reframing structural extraction as a diffusion-based distribution transformation. By combining the semantic intuition of deep learning with high-precision geometric constraints, PDT enables the automated generation of structured 3D data that meets the standards of professional artistic workflows.
Code and pre-trained models are available at: https://github.com/shanemankiw/PDT
Summary updated