Abstract
QUADify is an end-to-end framework that reconstructs high-quality 3D objects from 2D images directly into a quad-dominant mesh format. By making the re-meshing process differentiable, the system produces clean, artist-friendly geometry with pixel-level details and decomposed materials. Unlike traditional methods that produce irregular triangle soups, QUADify integrates quad-extraction into the optimization loop, ensuring that the underlying geometry is optimized specifically for high-quality topology and professional rendering pipelines.
Introduction
Most modern 3D reconstruction techniques, such as NeRFs or photogrammetry, produce either implicit neural volumes or irregular triangle meshes. While these representations can achieve high visual fidelity, they are often unsuitable for professional industry workflows. Artists prefer quad meshes for their predictable behavior during sculpting, animation, and subdivision. Traditionally, converting a raw 3D scan into a clean quad mesh is a manual, non-differentiable post-processing step. QUADify addresses this by making the quad-extraction process part of the differentiable optimization loop, allowing the system to learn a shape that favors clean topology.
Methods
The authors bridge the gap between implicit neural representations and explicit geometric editing through several key innovations:
- Differentiable Re-meshing: The system represents shape using a Signed Distance Field (SDF). It learns "orientation" ($O$) and "position" ($P$) fields across the surface via two small neural networks (o-MLP and p-MLP). These fields guide the placement of quad faces.
- The Stop-Gradient Trick: To handle the non-differentiable nature of iterative smoothing, the authors use a technique similar to Vector Quantization. They skip the gradients of the iterative smoothing process and instead "copy" the gradients from the final quad mesh directly back to the initial predicted fields. This allows the SDF to learn a shape that specifically favors a clean quad layout.
- Subdivision and Displacement: Once a base quad mesh is formed, the system applies Catmull-Clark subdivision, the industry standard for smoothing meshes. To capture fine details like wrinkles or pores, it learns a displacement field that shifts vertices at the pixel level, rather than relying solely on shading tricks like normal maps.
- Inverse Rendering: The entire pipeline is supervised by an image-space loss. The model renders its current mesh, compares it to the original photos, and back-propagates the error to update the geometry, materials (diffuse and specular), and lighting.
Key Results
QUADify was evaluated on the NeRF Synthetic, Myles, and NeRD datasets. It demonstrated superior mesh quality, achieving a 0.00% outlier percentage for face aspect ratios on the Myles dataset, compared to ~12-16% for standard Marching Cubes. Geometrically, it outperformed baselines like Nvdiffrec, achieving a Chamfer Distance of 4.63 compared to Nvdiffrec's 7.8.
The method produces competitive render quality while providing a mesh that is significantly easier to edit and animate than traditional triangle-based reconstructions. By prioritizing quad-dominant topology, it provides a stable canvas for displacement mapping and material decomposition.
Limitations and Future Work
While QUADify represents a significant step forward, it faces challenges in reconstructing deep concave regions where the MLP-based SDF can get stuck in local minima. Additionally, the current implementation initializes the SDF as a sphere, which can struggle with complex topological changes if foreground masks are inconsistent. Future work will focus on moving toward more flexible representations that can handle arbitrary topology and improving the reconstruction of highly complex real-world scenes.
Summary updated