Training Unsigned Distance Fields Against a Learned Mesh-Defect Surrogate

Many real objects, such as shirts, leaves and sheet-metal car panels, are open surfaces with no inside and outside. Neural networks represent them well as unsigned distance fields (UDFs): a network maps every 3D point to its distance from the surface, and the surface is where that distance is zero. To use the shape downstream, someone still has to turn the field into a triangle mesh, and that is where these methods break. Current meshers read the field one small grid cell at a time. Wherever the learned field is slightly noisy or flattened near the surface, they produce holes, cracks and non-manifold edges. A non-manifold edge is one shared by more than two triangles, which simulators and CAD tools cannot use. The field itself is trained without any knowledge of these failures. It is judged by point-distance losses, and mesh quality is checked only afterward.

We propose to close that loop. A small network, the defect surrogate, looks at the same local field values a mesher reads in one grid cell and predicts whether that cell will produce a topological defect when meshed. It is trained on real defect labels, obtained by actually meshing the current field and running a mesh checker. Its predictions are differentiable, so they can be added to the loss that trains the UDF. The field is pushed toward configurations the mesher handles correctly. As in MetricGAN-style training, the surrogate is refreshed on a replay buffer of the field's own recent states, which keeps the field from exploiting the surrogate's blind spots. The claim is that a field trained this way yields meshes with markedly fewer cracks, holes and non-manifold elements under several different extractors, without losing distance accuracy.

Where the gap is

Neural UDFs (Neural Unsigned Distance Fields for Implicit Function Learning) remove the sign, so the field has a V-shaped kink at the surface and its gradient flips direction there. Networks round this kink off, gradients vanish near the surface, and extraction becomes fragile.

One line of work repairs the field itself, and these methods motivate the proposal. DUDF passes distance through a hyperbolic-tangent function to make it smooth. LevelSetUDF regularizes the surface using off-surface level sets. Metric—Phase Fields adds a separate phase network, which turns open sheets into thin signed shells. All three optimize continuous properties of the field and never see the mesh.

A second line repairs extraction, and these methods supply the machinery we reuse. MeshUDF assigns pseudo-signs to grid corners from gradient flips and makes vertex positions differentiable. NSD-UDF classifies each cell's sign configuration from 32 local values: 8 corner distances and 24 gradient components. Iterative UDF meshing passes information between neighboring cells to close gaps. DualMesh-UDF fits planes from off-surface samples. DCUDF meshes an offset shell, projects it onto the surface and peels it with a min-cut.

All of these extractors are post-processes. None sends a signal back into the field, and MeshUDF's gradients move vertices but cannot change topology. A crack appears or disappears as a discrete event, so its gradient is zero almost everywhere.

The learned-surrogate literature supplies the training recipe. Quality-Net and MetricGAN+ train a network to imitate a non-differentiable audio quality metric and use it as a loss. MetricGAN+ adds a replay buffer so that the imitation stays accurate as the generator changes. It reports PESQ 3.15 against 2.86 for the original MetricGAN on VoiceBank-DEMAND. MetricOpt does something similar for vision metrics.

The same literature also constrains us. The PESQetarian shows that a model optimized against a learned metric produces artifacts that score well on the metric but are bad in fact. Our design and evaluation are built around that warning.

In our setting a mesh's defects can be attributed to individual grid cells. The surrogate can therefore be local: it gives a dense per-cell signal rather than one global score. This makes it far easier to learn than MetricGAN's single-scalar discriminator, and also harder to fool.

Core mechanism

Let $f_\theta$ be the UDF network, and let $p_c(\theta)$ be the local patch the extractor reads in grid cell $c$. For MeshUDF and NSD-UDF, the patch is the corner distances and gradients, optionally concatenated with those of the six neighboring cells.

The surrogate $q_\phi(p_c)\in[0,1]$ predicts the probability that cell $c$ contributes a defect under a fixed extractor $E$. A defect here means a non-manifold edge or vertex, a crack edge where adjacent cells disagree across their shared face, or a triangle belonging to a small connected component far from all input points.

The field is trained with

$$ \mathcal{L}(\theta)=\mathcal{L}_{\text{fit}}(\theta)+\lambda\,\mathbb{E}_{c\sim\mathcal{N}_\theta,\;\delta\sim U}\big[q_\phi(p_{c+\delta}(\theta))\big], $$

where $\mathcal{L}_{\text{fit}}$ is the base UDF fitting loss, $\mathcal{N}_\theta$ is the set of cells within a band around the current near-zero region, and $\delta$ is a random sub-cell shift of the grid. The base loss is LevelSetUDF's unsupervised point-cloud loss, or DUDF's supervised loss when ground-truth distances are available.

The random shift matters. Without it, the field could learn to be clean only at one grid alignment. With it, the field has to be meshable under a whole distribution of grids.

The surrogate is trained by binary cross-entropy against real labels. Every $K$ field steps, we run $E$ at a random shift and resolution, run a mesh checker, and assign each defect back to the cells that produced it. We then push the labeled patches into a replay buffer, following MetricGAN+. Early in training, the buffer is seeded with patches from many fitted shapes, so the surrogate starts useful. Afterward, it is fine-tuned on the current shape's own trajectory.

Research program

The first step is to establish that local defect prediction is learnable. We will measure the surrogate's calibration and AUC on held-out shapes, and check whether per-cell predictions summed over a mesh correlate with the true defect count. If a 32- to 224-value patch cannot predict defects, the idea stops here.

The second step is to integrate the surrogate into per-shape fitting and study the trade-off between data fidelity and meshability as $\lambda$ varies.

The third step studies gaming directly. We will track the gap between the surrogate's predicted defect rate and the measured rate over training, with and without the replay buffer. We will also watch for degenerate strategies: erasing thin parts to avoid defects, which would show up as lost F-score near boundaries, or inflating sheets into double walls, which would show up as a mismatched count of boundary loops.

The fourth step tests transfer. We will train the surrogate against one extractor, MeshUDF, and ask whether the resulting field also improves under extractors it never saw: NSD-UDF, iterative meshing, DualMesh-UDF and DCUDF. Transfer would mean the surrogate taught the field something genuine about local unambiguity, not a quirk of one mesher.

Baselines

The naive reference is the same UDF network and fitting loss with no surrogate term, meshed by each extractor. It isolates what the loss changes about the field.

The strongest field-side competitor is LevelSetUDF, trained with its published losses. It is the leading method in the file for unsupervised UDF fitting, and it reports Chamfer-L2 of 0.098 against 0.119 for CAP-UDF on ShapeNet Cars. For the supervised setting we will also use DUDF.

The strongest extraction-side competitors run on the baseline field. On ABC at $128^3$, NSD-UDF reports a median Chamfer distance of 2.52, against 2.97 for MeshUDF and 44.6 for DCUDF. On ShapeNet Cars at $512^3$, iterative meshing reports 8.84 against 56.9 for NSD-UDF. On MGN, the file lists DualMesh-UDF at Chamfer 2.38 and F-score 98.09%, against 4.76 Chamfer for MeshUDF. On DeepFashion3D, DCUDF reports 0% non-manifold vertices, against more than 18,000 for MeshCAP.

Fairness rules apply to all of these. Every method gets the same point clouds, the same network architecture and the same number of optimizer steps. We report wall-clock time, and give baselines extra steps equal to our surrogate overhead. Every method also gets the same hyperparameter search budget, including a search over each baseline's regularizer weights.

The ablations isolate the key ingredient. The first uses a frozen surrogate with no replay. The second replaces the surrogate with the entropy of a frozen NSD-UDF classifier, so no metric labels are involved. The third drops the random grid shift. The fourth trains the surrogate on a different extractor from the one used at evaluation.

Beating the baselines means the following. Under MeshUDF and NSD-UDF, our fields should cut crack edges and non-manifold elements by a large factor, which we set at 2× in advance. Chamfer distance and F-score should stay within noise of the base loss. The count of boundary loops and holes should match ground truth more often. We will also combine our field with the best extractor and compare against that extractor on the baseline field.

Why not just…?

Why not just mesh the ordinary field with DCUDF, which already reports 0% non-manifold vertices? This is the real threat. We take it seriously by evaluating DCUDF on both baseline and surrogate-trained fields. DCUDF guarantees a manifold by construction, but it can only express the topology the field contains. A crack in the field becomes a manifold boundary where none should exist, and a spurious sheet becomes a manifold spurious sheet. DCUDF also needs the user to say whether the shape is open or closed, it is about 5× slower than MeshUDF, and it reports 44.6 median Chamfer on ABC. We therefore measure topology correctness against ground truth: counts of boundary loops and components, and Euler characteristic. Counting only non-manifold elements would miss these errors. If DCUDF on the baseline field matches our fields on those measures at comparable accuracy, the proposal's value reduces to speed, and we will say so.

Why not just use a stronger learned mesher, such as iterative meshing, and leave the field alone? Better extractors read the same ambiguous patches. The file's own numbers show errors growing with grid resolution, for example 56.9 Chamfer for NSD-UDF at $512^3$. That points to the field as the source of noise. The extractor-transfer experiment settles the question directly. If our field gives no gain under iterative meshing, extraction alone is enough.

Why not just add a stronger analytic regularizer, such as DUDF smoothing, Monge-Ampère terms or a larger LevelSetUDF weight? Analytic regularizers trade detail for smoothness. DiffCD's Theorem 1 (DiffCD) shows that SIREN's off-surface penalty acts as an area penalty. We give these baselines a matched tuning budget over regularizer strength and compare the resulting curves of defect count against Chamfer distance. The surrogate must dominate that curve, not just one point on it.

Why not just use the entropy of a pretrained cell classifier as the loss, with no metric labels or replay? It is cheaper and might suffice, so it is an ablation rather than a strawman. The field can make the classifier confident while its predictions become wrong, which is exactly the failure the PESQetarian paper documents. The measured-versus-predicted defect gap will show whether the ground-truth labels and the replay buffer earn their cost.

Decisive experiments

We will run per-shape fitting on MGN and DeepFashion3D for open garments, ShapeNet Cars for interior panels, and ABC for sharp CAD shapes. Point clouds come in dense, sparse and noisy variants. We report Chamfer distance, F-score and normal consistency, plus non-manifold vertices and edges, crack length, the error in the number of boundary loops, and spurious-component area.

Three checks decide the question. First, defect reduction must hold at grid resolutions and shifts never used in training, from $128^3$ to $512^3$. If it holds only at training grids, the field has overfit to the surrogate. Second, gains must carry over to extractors the surrogate never saw. Third, the gap between surrogate-predicted and measured defect rates must stay small throughout training with replay, and grow without it. If all three hold while Chamfer distance is unchanged, the field has learned to be meshable rather than to fool its critic.