RDT-3D: Resolution-Decoupled Training for High-Fidelity 3D Geometry Generation

Jiale Zhang, Yang Liu, Chengyang Li, Jin Yuan, Qianxi Jia, Taicheng HuangarXiv 2026 · arXiv

TL;DR

RDT-3D introduces Resolution-Decoupled Training, a novel strategy that fixes a fundamental mismatch between how 3D generation models are trained and how they are used at inference. By anchoring spatial guidance at a coarse resolution while supervising geometry at an ultra-high resolution, the framework enables the generation of high-fidelity 3D assets with significantly sharper edges and finer details than current state-of-the-art methods.

What Problem Does This Paper Address?

Modern 3D generation often follows a "coarse-to-fine" approach: a first stage generates a rough global shape, and a second stage refines the local surface details. Many top-tier models use Rotary Position Embeddings (RoPE) to tell the refinement network exactly where in space it is working.

The authors identified a critical "resolution discrepancy" in this process:

  • During Training: Models are typically trained symmetrically. If the target output is high-resolution (e.g., 1536³), the positional guidance (RoPE) is also provided at that same high resolution.
  • During Inference: The model receives positional guidance from a coarse stage-1 mesh (e.g., 512³), but it is still expected to produce a high-resolution output.

This mismatch creates a severe distribution shift. Because the model only ever saw "perfect" high-res coordinates during training, it struggles to handle the "imprecise" coarse coordinates it gets in the real world, leading to oversmoothed surfaces, rounded edges, and lost thin structures.

Methods

The core innovation is Resolution-Decoupled Training (RDT). Instead of aligning guidance and supervision resolutions, RDT explicitly decouples them:

  1. Coarse Localization Branch: During training, the RoPE coordinate guidance is anchored at a coarse resolution ($R_g = 512$). This forces the network to handle the same level of spatial "uncertainty" it will encounter during actual use.
  2. Fine Supervision Branch: Simultaneously, the geometric target (the ground truth) is kept at an ultra-high resolution ($R_s = 1536$).

By training this way, the model learns a robust mapping: it becomes an expert at extrapolating high-frequency details from coarse spatial cues. The architecture utilizes a RoPE-guided Diffusion Transformer (DiT) and a rectified-flow objective for stable optimization.

Fig. 1: Overview of the proposed RDT-3D framework. Training (Left): RDT-3D decouples spatial guidance from geometric supervision. Coarse point clouds (5123) provide spatial query locations injected into the DiT via ROPE, while fine point clouds (15363) are encoded into high-resolution GT shape latents for diffusion training. Inference (Right): Spatial queries sampled from a coarse mesh guide the trained DiT to generate fine shape latents, which are subsequently decoded into a high-fidelity 3D mesh via Marching Cubes.

Datasets

To support this high-resolution training, the authors introduced Dual3D-300K, a massive dataset of 300,000 watertight 3D point cloud pairs.

  • Alignment: Each object is explicitly aligned across 512 and 1536 resolutions.
  • Curation: Data is sourced from high-quality repositories like ABO, 3D-FUTURE, and Objaverse-XL.
  • Sharpness-Aware Sampling: To ensure the model learns to respect fine features, the dataset uses a mix of uniform surface sampling and curvature-concentrated sampling (focusing more points on edges and thin parts).
Fig. 2: Visual cases in Dual3D-300K. Higher resolution (1536) captures significantly finer geometric details and sharper surfacecompared to the base resolution (512). (Best viewed when zoomed in.)

Key Insights and Ideas

  • Structured Noise vs. Random Noise: The authors found that simply adding random Gaussian noise (jitter) to coordinates during training doesn't solve the problem. The resolution gap is a structured shift, and only explicit resolution decoupling (RDT) effectively bridges it.
  • Extrapolation is Key: The refinement network shouldn't just be a "copy-paste" mechanism for coordinates; it needs to be an inference engine that predicts what a sharp edge should look like based on a blurry spatial hint.
  • Stability through GT: While the model eventually uses stage-1 outputs at inference, the authors train using low-res versions of the ground truth for guidance. This maintains global structural alignment and prevents the training process from becoming unstable due to the "hallucinations" of upstream models.

Key Results

RDT-3D was tested against heavyweights like Hunyuan3D-2.1, TRELLIS, and UltraShape.

  • Quantitative: RDT-3D achieved a Chamfer Distance (CD) of 0.0164 and an F-Score of 0.6886, consistently outperforming all baselines.
  • Qualitative: The visual difference is striking. RDT-3D produces meshes with crisp, sharp boundaries and intricate topologies (like the scales on a dragon or the fine carvings on a jade ornament) where other models produce "melted" or oversmoothed versions.
Qualitative comparison showing RDT-3D producing much sharper details than SOTA baselines.

Ablation studies confirmed that the RDT strategy is the primary driver of these improvements, as shown in the comparison between the baseline and the RDT-trained version.

Ablation study highlighting how RDT recovers sharper edges compared to standard training or simple jitter.

Limitations and Future Work

While the paper demonstrates a significant leap in fidelity, a few areas remain for exploration:

  • Inference Dependency: The quality of the final high-res mesh is still somewhat tethered to the global topology established by the stage-1 coarse model. If the coarse model misses a major limb or part, the refiner may not be able to "invent" it.
  • Computational Cost: Generating at 1536 resolution is computationally intensive. While the Dual3D-300K dataset lowers the barrier for others to enter this space, the inference time for such high-fidelity assets is naturally higher than for lower-res models.

Summary updated

Abstract

High-fidelity 3D geometry generation requires maintaining global structural coherence alongside fine-grained surface details. While recent coarse-to-fine frameworks leverage Rotary Position Embeddings (RoPE) to condition local queries, we reveal a critical training-inference resolution discrepancy: coordinate guidance and geometric supervision are aligned during training, whereas inference decodes high-resolution geometry from coarse spatial coordinates. This mismatch introduces a severe distribution shift that degrades fine details. To overcome this, we propose Resolution-Decoupled Training (RDT), which anchors RoPE guidance at 512 resolution while supervising geometry at 1536, enabling robust detail extrapolation. Additionally, to eliminate high-resolution data bottlenecks, we present Dual3D-300K, a large-scale paired dataset comprising 300K watertight 3D point clouds explicitly aligned across 512 and 1536 resolutions. Extensive experiments demonstrate that RDT substantially improves sharp geometric regions, while Dual3D-300K offers a scalable benchmark for high-resolution 3D generation.
BibTeX
@misc{zhang2026rdt,
  author = {Jiale Zhang and Yang Liu and Chengyang Li and Jin Yuan and Qianxi Jia and Taicheng Huang},
  title = {{RDT-3D: Resolution-Decoupled Training for High-Fidelity 3D Geometry Generation}},
  year = {2026},
  eprint = {2611.06141},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.06141},
}
Figure