Generating Manifold Meshes with a Bounded Frontier Stack

Transformers that write 3D meshes one triangle at a time now produce clean, artist-like models. They hit a wall, though, at a few thousand faces, and they still leave holes and broken edges. This proposal borrows one idea from the study of how people parse sentences with limited working memory. A reader does not need to remember every word they have already understood; they only need to remember the phrases that are still open. A mesh generator is in the same position. Triangles that are fully surrounded by other triangles can no longer change, and every new triangle must attach to the "open" boundary of the surface built so far. So we propose a mesh generator that keeps only the currently open boundary loops in memory, arranged as a small stack. Its actions are a transition grammar, meaning a fixed set of legal moves in the style of a shift-reduce parser. Some moves extend a loop, some split a loop when the surface branches, and some join two loops when a handle closes. The model attends only to the loops on the stack and never to finished interior triangles.

Two properties follow if this works. First, validity comes by construction. A surface assembled only from legal grammar moves is a consistently oriented 2-manifold, meaning every edge borders at most two faces and the neighborhood of every vertex looks like a disk. If ending the sequence is allowed only when the stack is empty, the surface is also watertight. Second, memory per step is bounded by the open frontier rather than the full history. That could remove the face ceiling. The deeper, testable claim is the mesh analogue of a finding from linguistics: real 3D objects, traversed in the right order, almost never need more than a handful of open loops at once. If that holds, a small, fixed working memory loses almost nothing.

Core Claim

Left-corner Transitions on Dependency Parsing showed that, under a left-corner transition system, at least 98% of parser configurations across 18 languages need a stack depth of at most 3, and that randomizing word order inflates this cost. It measured only a gold oracle and trained no parser. We make two parallel claims for meshes.

The first is a measurement claim. Define the topological memory cost of a traversal at step $t$ as $d_t$, the number of open boundary loops on the frontier stack after $t$ moves. We claim that under a branch-first ordering, meaning one that closes a branch before returning to its parent, the distribution of $d_t$ over real artist meshes concentrates on small values. We further claim that DFS orderings like TreeMeshGPT's and ring orderings like Silksong's carry a much heavier tail, as does a randomized ordering.

The second is a modeling claim. A generator whose attention is restricted to the stack contents plus the conditioning input matches full-context generators on shape fidelity. At the same time it keeps a flat success rate as face count grows, where full-context models decay.

The conditioning input matters for the second claim. In the point-cloud-conditioned setting used by MeshAnything, the global shape is already supplied by the condition. The finished interior triangles are then mostly redundant history, and this is the regime where evicting them should cost nothing.

Relation to Prior Work

The closest work is the recent line of autoregressive mesh tokenizers. PolyGen generates vertices and then points to them to form faces. MeshGPT and MeshAnything learn a codebook of triangle tokens. All three give no manifold guarantee and are limited to under 800 faces because attention grows quadratically.

Two later papers address pieces of this, and they are the real competitors. TreeMeshGPT grows the mesh by depth-first search over triangle adjacency, using two tokens per face and a fixed winding rule that keeps normals consistent. It reaches 5,500 faces, but it keeps the whole trajectory in attention and reports success-rate decay on long sequences. Mesh Silksong orders vertices in breadth-first rings, handles connections within a ring through a sliding window, and already guarantees manifold, watertight output. Its rings fragment on thin, branched parts, and it has no push/pop structure for branches or handles.

Our proposal differs from both in two ways. The traversal state is an explicit stack of open loops, with split and join moves that change the genus in a controlled way. And memory is bounded by eviction that follows the stack, rather than by a window defined by distance in the sequence.

The eviction idea comes from Learning to Ignore. With 20 memory cells, that paper matched unbounded coreference on LitBank (75.7 vs 75.9 CoNLL F1), and at 5 cells learned eviction beat a least-recently-used rule by about 3 F1. We replace its learned eviction with structural eviction: a triangle leaves memory exactly when it stops touching any open loop.

That paper also records the known failure of bounded memory, where an evicted item reappears later. For meshes this failure has a precise meaning. If a loop that looks closed is actually needed for a later join, the join is impossible. We handle this by never evicting loops that are still on the stack. Only closed regions leave memory.

Two further papers constrain the design. Context-free Recognition with Transformers shows that fixed-depth transformers cannot recognize general nested structure without looping or extra scratch space. It also notes that the fact a model could represent a parser does not mean gradient descent will find it. This is why we enforce the grammar externally by masking illegal moves instead of hoping the model learns it. Memory-Bounded Left-Corner Unsupervised Grammar Induction reports a dip in learning when depth-2 structures appear. That motivates tracking accuracy separately by stack depth, since we expect genus-changing joins to be the hard cases.

The diffeomorphic approach of Hybrid Neural Diffeomorphic Flow and the convex-union approach of BSP-Net guarantee closed surfaces by other means. Both give up flexible genus or clean concave shapes.

Research Program

The first stage builds and measures the grammar. The moves adapt classical boundary-traversal ideas from mesh connectivity coding. One move attaches a triangle with a new vertex to the active gate edge. One attaches a triangle to a neighboring frontier vertex. A split move divides the current loop and pushes one part onto the stack. A join move, used when a handle closes, merges the current loop with a loop lower in the stack and records one unit of genus. A close move pops a finished loop. Every move is checked against the half-edge structure, so illegal moves are masked at decoding time. Allowing the end token only on an empty stack makes the output watertight.

We write a deterministic oracle that converts each manifold training mesh into a move sequence under several orderings: TreeMeshGPT-style DFS, Silksong-style rings, our branch-first order, and a random legal order. For each, we report the distribution of $d_t$ and of frontier length. This stage is cheap, and it can kill the project. If real meshes routinely need deep stacks or very long frontiers, a bounded memory is not justified.

The second stage trains the generator. A decoder-only transformer, conditioned on a point cloud as in MeshAnything, predicts the move and the quantized vertex coordinates. At each step its attention is limited to three things: the conditioning tokens, the tokens of half-edges on the top $k$ stack loops, and one summary token per popped region. Frontier length, not stack depth, is the quantity that could still blow up. So we also cap the attended half-edges per loop to a window around the gate and let pointer lookups reach the rest of the loop. The stage-1 statistics will tell us whether that cap bites. Training uses teacher forcing over the oracle sequences, with the attention mask computed from the oracle stack.

The third stage is analysis by stack depth and genus. We report error rates for extend, split, and join moves at each depth, and check whether generated meshes have the correct Euler characteristic $V - E + F = 2 - 2g$ for the target genus $g$.

Baselines

The naive reference is MeshAnything. It generates tokens with full context, gives no validity guarantee, and is capped at 800 faces. It shows what happens with no structure at all. We run its released setup on the same conditioning point clouds.

The strongest existing competitor on scaling is TreeMeshGPT. It is the only method in the file reaching 5,500 faces, and it reports Chamfer Distance 0.0070 and Normal Consistency 0.798 on Objaverse (vs 0.0102 and 0.167 for MeshAnythingV2, per TreeMeshGPT).

The strongest competitor on validity is Mesh Silksong. It reports Chamfer Distance 0.025 and Normal Consistency 0.792 on gObjaverse (Mesh Silksong) and already guarantees manifoldness.

To make the comparison fair, we retrain both on the same roughly 75k-mesh manifold Objaverse subset TreeMeshGPT uses. Each gets the same parameter count, the same total training compute, and the same quantization. We also sweep each method's main tokenization hyperparameters under an equal budget.

Beating them means three things. First, matching TreeMeshGPT's Chamfer Distance and Normal Consistency within noise below 5,500 faces. Second, a clearly higher success rate than both beyond 5,500 faces. Third, a manifold and watertight rate of 100% by construction, which ties Silksong, together with better genus accuracy on branched and handled objects where Silksong's rings fragment.

Three ablations isolate the key ingredient. The first uses the same grammar with full-context attention; it separates the effect of the grammar from the effect of eviction. The second uses the same grammar and the same memory budget but a sliding window over sequence position instead of stack-based eviction; this tests whether structure-aware forgetting matters. The third uses the same budget with learned eviction in the style of Learning to Ignore, to check whether the structural rule is better than, or merely equal to, a learned one.

Why not just…?

Why not just give TreeMeshGPT a longer or sparser context? This is the most serious threat. Sparse or linear attention might remove the ceiling without any grammar. However, TreeMeshGPT's reported decay occurs within its trained context, which points to a failure to track loop closures rather than a lack of context length. The theory in Context-free Recognition with Transformers suggests that more context alone does not supply the nesting bias needed for closures. We test this directly: TreeMeshGPT with a matched sparse-attention variant is included in the long-mesh evaluation. If it holds a flat success rate, the eviction claim fails and only the validity and genus claims remain.

Why not just use a sliding window or least-recently-used cache? When a handle closes, the loop being joined may have been opened thousands of tokens earlier. A window based on recency drops it, and Silksong's within-ring window has exactly this structure. The coreference evidence also favors smarter eviction at small budgets, where learned eviction beat the least-recently-used rule by about 3 F1 (Learning to Ignore). The second ablation measures this for meshes. We expect the gap to show up specifically in join moves and genus accuracy.

Why not generate with any model and repair the mesh afterward? Repair can close holes, but it cannot recover the intended genus or clean edge flow. It also inherits the kind of heuristic-threshold failure that holds BrepGen to 62.9% validity on DeepCAD. We will apply standard manifold repair to MeshAnything and TreeMeshGPT outputs and report genus accuracy and Chamfer Distance after repair, so that this comparison is measured rather than assumed.

Why not skip meshes and run Marching Cubes over a neural field? It is watertight, but it yields triangle soups. BSP-Net reports 183,096 triangles for IM-NET against compact outputs of a few hundred (BSP-Net), and users preferred structured meshes 83.6% of the time (MeshAnything). It answers a different question from compact, artist-style meshes, so it serves as a reference, not a competitor.

Proposed Experiments

The first decisive check is the stage-1 measurement. For each ordering over the Objaverse training meshes and ShapeNet categories with many branches (chairs, lamps), we report the fraction of steps with $d_t \le 3$, along with the tails of the frontier-length distribution. The hypothesis passes if the branch-first order concentrates tightly and clearly beats DFS, ring, and random orders.

The second check is a success-rate curve on Google Scanned Objects and held-out Objaverse meshes, stratified at 800, 2,000, 5,500, and above 10,000 faces. At each level we report success rate, Chamfer Distance, Normal Consistency, manifold and watertight rate, Euler-characteristic match, and peak memory and time per face. The proposal succeeds if its curve stays flat where TreeMeshGPT's and the sparse-attention variant's decay. It must also stay within noise of their fidelity at low face counts, and the gap to the full-context ablation must be negligible.

The third check breaks down errors by stack depth. It tests whether failures cluster at join moves, as the depth-2 dip in Memory-Bounded Left-Corner Unsupervised Grammar Induction would predict. If so, targeted extra capacity, such as looped layers applied only at join steps, is the natural follow-up.

One limitation stays out of scope. The grammar guarantees that each surface is manifold and oriented, but not that separate parts of the surface avoid intersecting each other in space. We measure self-intersection rates and report them honestly.