TL;DR
MeshAnything V2 introduces Adjacent Mesh Tokenization (AMT), a novel method that cuts 3D mesh token sequence lengths in half by leveraging face adjacency. This efficiency boost allows the model to generate complex, artist-quality meshes with up to 1600 faces—double the capacity of previous models—without increasing computational costs or memory overhead.
What Problem Does This Paper Address?
Generating high-quality 3D meshes that look like they were made by human artists (Artist-Created Meshes or AMs) is a major challenge in computer vision. While autoregressive models—which generate meshes vertex-by-vertex like a language model generates text—have shown promise, they hit a "complexity ceiling."
The bottleneck is tokenization. Traditional methods represent each triangular face using three vertices, and each vertex requires three coordinate tokens. This results in a sequence length nine times the number of faces. For complex objects, these sequences become prohibitively long, exhausting GPU memory and making it difficult for transformers to learn long-range dependencies. Furthermore, these sequences are highly redundant, as most vertices in a mesh are shared between multiple faces.
Methods
The core innovation is Adjacent Mesh Tokenization (AMT). Instead of treating every face as an independent triplet of vertices, AMT takes advantage of the fact that most faces in a well-structured mesh share edges.
- Adjacency Encoding: If a new face shares an edge with the previously encoded face, AMT only needs to provide the one additional vertex required to complete the triangle.
- Special Tokens: When the model cannot find an adjacent face to continue a "strip," it inserts a break token (
&) and restarts from a new face. A swap token ($) is also introduced to allow the model to change which edge of the current face it uses to find the next neighbor. - Face Count Condition: Unlike previous versions, V2 allows users to specify an approximate target face count, providing better control over the level of detail (LOD) in the generated output.
- Architecture: The system uses a point cloud encoder to extract shape features, which are then fed as a prefix into a decoder-only transformer (OPT-350M). The transformer then autoregressively predicts the AMT token sequence.
Datasets
The authors utilized the Objaverse dataset, supplemented with a small portion of ObjaverseXL.
- Cleaning: They performed rigorous deduplication based on vertex coordinates and filtered out meshes where the ground truth SDF generation failed.
- Scale: The final training set contained approximately 230,000 point cloud and mesh pairs.
- Complexity: While previous work capped meshes at 800 faces, this dataset includes meshes with up to 1600 faces to push the model's limits.
Key Insights and Ideas
- Efficiency is Exponential: By halving the sequence length, the authors achieved nearly a fourfold decrease in computational load and memory usage within the transformer's attention blocks (since attention complexity is $O(n^2)$).
- Regularity Matters: The authors found that a shorter sequence isn't always better. The sequence must have a "fixed pattern" or regularity. AMT achieves this by prioritizing adjacent faces, creating a more predictable "flow" for the transformer to learn compared to naive vertex sorting.
- Tokenization as a Bottleneck: The paper argues that for graph-based structures like meshes, the choice of 1D sequence representation is perhaps more critical than the specific transformer architecture used.
Key Results
MeshAnything V2 significantly outperforms its predecessor and other baselines across several metrics:
- Capacity: Successfully generates meshes with up to 1600 faces, doubling the previous limit.
- Compression: Achieves a sequence ratio (S_Ratio) of approximately 0.497, meaning the sequences are half the length of standard baselines.
- Quality: Improved Chamfer Distance (CD) and Normal Consistency (NC) compared to models using naive tokenization.
- Human Preference: In a user study, V2 was preferred 67.8% of the time over the original MeshAnything.
Limitations and Future Work
Despite the significant progress, the authors note that the accuracy of MeshAnything V2 is still not quite sufficient for high-end industrial applications (like AAA games or cinema) without further manual cleanup. The model can also struggle to strictly adhere to the "face count condition" if the requested count is extremely low for a very complex shape, as it prioritizes maintaining the overall topology. Future work will likely focus on improving the stability of the generation and further increasing the face count limit to handle even more intricate geometries.
Summary updated