MeshWeaver: Sparse-Voxel-Guided Surface Weaving for Autoregressive Mesh Generation

Instead of predicting 3D coordinates one by one, a new method called MeshWeaver "weaves" a mesh by predicting entire vertices at once. This approach uses a smart 3D grid to guide the process. It is much faster and creates far more detailed 3D models than previous methods.

The field of 3D computer graphics relies on polygonal meshes. These are collections of vertices (points in space) and faces (the flat surfaces connecting them) that define 3D objects. Humans craft these meshes with extreme care to ensure they are easy to animate. Automating this process has proven difficult. Most current AI models struggle to bridge the gap between raw, messy geometric data and the clean, structured meshes required for production.

The bottleneck of coordinate-level sequences

Existing autoregressive models—systems that predict the next element in a sequence based on previous ones—treat mesh generation as a massive string of numbers. They tokenize meshes by flattening every $(x, y, z)$ coordinate into a long stream of tokens (the smallest units of data a model processes). This approach has two fatal flaws. First, it is incredibly inefficient. A single mesh with $N$ faces results in a sequence length of $9N$ tokens. This creates a massive computational burden. It prevents models from scaling to high-polygon, detailed objects.

Second, these models often lack "geometry awareness." They rely on global shape embeddings—broad, high-level summaries of an object—rather than local surface cues. Because the model does not "see" the immediate neighborhood of a vertex, it frequently suffers from surface drift. This is where the generated geometry wanders away from the intended shape. It also causes the loss of fine-grained details, like the texture on a coin or the keys on a keyboard .

Weaving surfaces with sparse voxels

The authors of MeshWeaver propose a fundamental shift. They move from next-coordinate prediction to next-vertex prediction. Instead of treating $x$, $y$, and $z$ as independent steps, the model treats a vertex as a single, atomic token. To make this work, the researchers use a vertex-level tokenization strategy [, right].

The mesh is divided into local patches. The model "weaves" the surface by predicting the center vertex and its neighbors in a structured sequence.

To represent a complete 3D position in one step, the authors use a multi-level voxel representation. Think of this like a digital map that starts with large, blurry provinces and progressively zooms in on specific street addresses. The 3D space is partitioned into hierarchical voxel grids. The model first predicts a coarse voxel. It then iteratively narrows the prediction to finer subvolumes until it reaches the required resolution.

Central to this process is a hierarchical sparse-voxel encoder .

Rather than working in a vacuum, the autoregressive transformer is guided by a "scaffold" of sparse voxels. These are 3D volumetric pixels that mark exactly where the surface exists. The encoder provides three types of help. It represents vertices using local voxel features. It guides token prediction through cross-attention (a mechanism that lets the model "look at" and pull information from specific parts of the voxel grid). Finally, it acts as a physical constraint. It masks out empty space during inference (the stage where the model generates new data). This ensures the model stays anchored to the actual geometry .

Achieving high-fidelity compression

The results of this architectural shift are significant for 3D asset pipelines. The authors report that MeshWeaver achieves a state-of-the-art mesh compression ratio of 18%. This outperforms previous coordinate-level methods that were capped at roughly 22% [Table 1]. This efficiency allows the model to scale to much more complex geometries. It supports meshes with up to 16,000 faces.

In terms of raw geometric accuracy, the paper finds that MeshWeaver consistently outperforms baseline models on the Toys4K benchmark. The authors report a Chamfer Distance (CD) of 0.116 and a Hausdorff Distance (HD) of 0.087. Both metrics indicate much closer alignment to ground-truth surfaces than prior works [Table 2]. Lower scores in these metrics mean the generated shape is physically closer to the target. Furthermore, the model shows a superior ability to preserve surface orientation. It matches or exceeds the performance of top-tier competitors in Normal Consistency (NC) [Table 2].

Limits of the weaving paradigm

Despite these gains, the paper does not claim to have solved all aspects of mesh generation. The authors note that their compression efficiency could still be improved. For instance, they suggest that using specialized token sets for patch centers might further shorten the sequence length.

Additionally, the hierarchical sparse-voxel encoder introduces architectural complexity. While the "subvolume pruning" strategy helps accelerate training by focusing only on relevant areas, the process involves significant computational overhead. The model must manage multi-level cross-attention layers. Finally, while the model excels at reconstructing geometry from point clouds, the paper does not extensively explore generation from other modalities, such as pure text or single images.

The verdict

MeshWeaver is a highly effective step toward automated 3D modeling. By rethinking the fundamental unit of generation from a coordinate to a vertex, the authors have bypassed the scaling bottlenecks of earlier models. The combination of hierarchical voxel guidance and a "scaffolding" mechanism provides the local precision necessary for high-fidelity assets.

If you are building tools for 3D content creation, this approach is worth watching. It moves the needle from "generating rough shapes" to "weaving usable geometry." While the added complexity of the voxel encoder means it is not a "lightweight" solution, the trade-off in fidelity and compression efficiency makes it a compelling candidate for high-end asset pipelines.

Figures from the paper