Sheets and Junctions: A Compiler-Style Intermediate Representation for Learning on Wild Meshes

Most geometric deep learning assumes the input is a clean surface. In that kind of surface, every edge borders at most two triangles and every point looks locally like a small patch of the plane. Real 3D assets rarely meet this standard. Game characters, CAD exports and scans arrive as "wild" meshes: three or more triangles meet at an edge, pieces float disconnected, faces point the wrong way, and separate parts touch or pass through each other. On such input, the tools that learning pipelines depend on stop working. The most important of these is the surface Laplacian, the operator that tells a network how signals spread across a surface.

We propose to treat this the way compilers treat messy bytecode. A compiler first lifts code into a small, normalized intermediate language and only then analyzes it. We would build a mesh compiler that rewrites any triangle soup into two kinds of records. "Sheets" are maximal patches that are guaranteed to be clean surfaces, so standard operators apply to them exactly. "Junctions" are explicit, typed records of where sheets meet, which are either glued seams or mere physical contacts. A network then runs ordinary surface operators inside each sheet and passes learned messages only across junctions. The original vertices, faces and UV coordinates are never moved or resampled.

Core Claim

The defects that break learning on wild meshes are combinatorial facts about connectivity. They are not geometric noise. They should therefore be resolved by discrete lifting rules, not by wrapping the shape in a smooth volume. Our main bet concerns two situations that look alike in space: two surfaces glued along a seam, and two surfaces that merely press against each other, such as a hand resting on a torso. A discrete representation can tell these apart. Any method that transfers signals through 3D space cannot. If the bet is right, the sheet-and-junction representation should match a cage-based proxy on overall accuracy. It should also clearly beat that proxy in the regions where separate surfaces touch, and it should do so without changing a single input vertex.

Positioning Against Prior Work

The closest work is CageNet. CageNet wraps the wild mesh in a simplified clean "cage" surface, runs a standard backbone such as DiffusionNet on the cage, and maps features back using mean value coordinates. Mean value coordinates are a smooth volumetric interpolation scheme. CageNet's own limitations section motivates this proposal. Because the mapping is defined by position in 3D space, touching surfaces collapse onto the same cage coordinates. Aggressive cage simplification also fuses fine structures, so fingers become a "mitten." Our representation is intrinsic rather than volumetric. Nothing is transferred through space, so neither failure can occur by construction. The open question is whether we lose something CageNet gains, namely smooth propagation across badly broken connectivity. Our weld pass and learned junction messages are meant to recover that, and the experiments test it directly.

Cell Complex Neural Networks supplies part of the method. That work defines message passing over cells of several dimensions, using adjacency and coadjacency between them. It offers no pipeline that turns broken triangle soups into well-formed complexes, and it reports no results on defective meshes. We add the missing lifting step. We also keep continuous surface operators inside each sheet instead of reducing everything to discrete cell-to-cell messages.

The repair line of Visual-Guided Mesh Repair and Visual-Preserving Mesh Repair is a strong constraint on our design. These methods build an exact-arithmetic space partition, run a global graph cut and extract one watertight outer shell, while preserving appearance. Their output is a single outer surface, so interior parts are removed. CageNet notes that such interior parts matter for skinning in artist-made characters. We do not try to produce a watertight result at all.

Two further papers establish that the problem is real. Subdivision-based Mesh Convolution Networks shows that a network forced onto remeshed input can be no better than the remesher. MapTree notes that the standard discrete Laplacian (the cotangent Laplacian) is undefined once an edge borders three or more faces.

The compiler analogy shapes the design in specific ways. Dexpler lifts register bytecode into three-address code and needs a dedicated typing pass for ambiguous instructions, such as an opcode that could mean integer zero or a null reference. Our weld-versus-contact classification plays the same role for geometry. REPATT uses a simplified three-address form to untangle expressions into separate, indexed statements. That is the move we make when we unroll a non-manifold edge into separate sheets. CoDist shows that lossy normalization helps translation but collapses when identifiers are stripped. We take this as a constraint: the representation may discard connectivity quirks, but it must keep every original face, vertex identity, UV coordinate and part label. JBMC inspires the verification step. JBMC uses SAT/SMT solving to check programs it has lowered into a simple control-flow form. We take only the discipline of having a checker after every compiler pass. Our checks are cheap combinatorial ones, not solver queries.

Research Program

The compiler has four passes. Each one is followed by a checker that refuses to continue if an invariant is violated. The invariants are fixed in advance. Every input face appears in exactly one sheet. Every sheet edge borders at most two sheet faces. Each sheet is consistently oriented. Every original vertex is reachable from its copies through junction records. UV and attribute data are carried through bit-for-bit.

The first pass is the weld pass, which handles disconnected soups. It merges near-coincident vertices along boundary edges, using a tolerance relative to the length of local edges. The second pass is unrolling. It splits every edge that borders three or more faces, and every vertex whose neighborhood does not look like a single disk, into separate sheet-local copies. Each split emits a seam junction that records which copies were once the same point. The third pass orients each sheet independently, which avoids the ill-posed problem of finding one global orientation for a non-orientable or open shape. The fourth pass detects contacts. It records pairs of face interiors from different sheets that lie within a distance of each other or intersect, and emits contact junctions. Contact junctions are never glued.

The hardest design decision is the line between welding and contact. A crack in a shell should be welded, while a palm resting on a thigh should not. Our proposed rule uses only combinatorial evidence. Points close to each other on two boundaries are weld candidates. Closeness between face interiors, or between a boundary and an interior, is a contact. We will measure how often this rule fails on held-out assets with known part structure.

The learning layer runs one shared DiffusionNet-style block on each sheet, using that sheet's exact cotangent Laplacian. A junction layer then exchanges features between the copies on either side of each seam. Seam and contact junctions get separately learned gates, and the two layers alternate. The model can therefore learn to spread information freely across seams and cautiously or not at all across contacts. Predictions live directly on the original faces and vertices, so no projection step is needed afterwards.

Baselines

The naive reference point is DiffusionNet run directly on the raw wild mesh, falling back to its point-cloud Laplacian wherever the mesh Laplacian is undefined. This measures how much any structural treatment helps. CageNet reports that DiffusionNet degrades significantly on its souped segmentation benchmark but gives no exact figure, so we will measure this baseline ourselves.

The strongest existing method is CageNet. It is the only method in our file built for exactly this setting, and it reports the best numbers on both relevant benchmarks. It reaches an average L1 skinning-weight error of 0.124 on the Roblox Skinning Dataset, against 0.153 for RigNet and 0.575 for bounded biharmonic weights. It reaches 91.7% segmentation accuracy on the souped human benchmark. We will run CageNet with the same DiffusionNet backbone width and depth as ours, the same training split, and the same hyperparameter search budget. We will also sweep its cage resolution so that it is not held back by over-simplification.

Beating CageNet means two things. First, our overall L1 error and accuracy must lie within its run-to-run variation or better. Second, we must show a clear, statistically significant improvement on the contact-region subset defined below.

The remeshing baseline passes every asset through a volumetric remesher or the repair method of Visual-Preserving Mesh Repair. DiffusionNet runs on the result, and predictions are carried back to the original faces by nearest-surface lookup. This tests whether a better clean surface is all that is needed.

Two ablations isolate the key ingredient. The "split-only" ablation removes the junction layer, so sheets do not communicate. This is the cheap fix of cutting along every non-manifold edge. The "weld-everything" ablation treats contacts as seams, which imitates the spatial merging that we argue makes cages fail. If split-only matches the full model, the learned junctions are unnecessary. If weld-everything matches it, telling contacts apart from seams does not matter.

Why not just…?

Why not just remesh everything into a watertight surface first? Remeshing destroys what downstream users need. In the Visual-Preserving Mesh Repair evaluation on ShapeNet, TetWild reaches only 22.2 PSNR and VolumeMesher 32.8, against 58.9 for the appearance-preserving method. Even that best method produces one outer shell and therefore removes interior parts. The remeshing baseline above will show whether the lost fidelity and interior structure actually hurt skinning error, rather than leaving that as an assumption.

Why not just use CageNet with a finer cage? A finer cage reduces fusion of small features. It cannot fix the contact problem, because two touching surfaces map to the same volumetric coordinates however fine the cage is. Our cage-resolution sweep tests this. If CageNet's contact-region accuracy approaches ours as the cage gets finer, our central claim is wrong and we will say so.

Why not just sample a point cloud and drop connectivity? This is the most serious threat, because DiffusionNet already runs on point clouds and needs no clean surface. However, a point cloud places a palm and the thigh beneath it next to each other, so it has the same problem as the cage. It also loses face-level orientation, which Orienting Point Clouds with Dipole Propagation shows is costly to recover. The naive baseline's point-cloud mode measures this directly. A tie on contact regions would undercut the proposal.

Why not just cut non-manifold edges and run DiffusionNet per piece? This is the split-only ablation. It almost certainly fixes contacts, but it blocks information from crossing legitimate seams. We expect it to fail on souped meshes, where nearly every edge has been broken. Whether learned junctions actually beat it is the cleanest test of what we add. Intrinsic non-manifold Laplacians from the geometry processing literature belong in the same family. No such work is in our research file, but we will add one as a baseline if code is available.

Proposed Experiments

The first experiment is the Roblox Skinning Dataset, 753 artist-made bipeds, scored by average L1 skinning-weight error against CageNet's 0.124. The second is the souped human segmentation benchmark, regenerated by following CageNet's described corruption protocol of broken connectivity and flipped normals, and scored by face accuracy against 91.7%.

The decisive check is a contact stress test. We pose Roblox characters so that hands, arms and legs touch or pass into the torso. We then report error only on faces that lie within a set distance of a different part. We also report a leakage score: how much predicted skinning weight for one bone, or label probability for one part, appears on a touching but unconnected part.

To check the compiler itself, we run it on the 400-model Thingi10K subset and the 1,000-model ShapeNet subset used in the repair papers. There we report how often the passes complete, how often each invariant checker fires, how many weld-versus-contact decisions disagree with ground-truth part labels where those exist, and wall-clock time. As a sanity property, we also confirm that vertices and UVs are preserved exactly, which no remeshing baseline can achieve.

The outcome that would most change our plans has two parts: split-only matches the full model on souped meshes, and CageNet or point clouds match it on contacts. In that case the representation adds nothing, and we would drop it in favor of the simplest baseline that wins.