3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code

The Gap Between Executable Code and Physical Reality

Researchers are teaching AI to act like 3D designers. They want models to write code for software like Blender. Instead of generating a static 3D mesh (a collection of vertices and faces), the goal is procedural instructions. This creates assets that are deterministic and engine-ready.

The field is split between static mesh repositories and experimental agents. These agents attempt to drive 3D software via Python APIs (Application Programming Interfaces). However, we lack a standardized way to measure agent performance. This paper introduces 3DCodeBench. It evaluates how well Vision-Language Models (VLMs) translate text and images into functional Blender scripts. The authors uncover a massive gap. Models can write code that does not crash. Yet, they still struggle to produce objects that look physically plausible.

The Problem

Current 3D generation falls into two camps. Neural implicit representations produce beautiful but "frozen" meshes. Manual procedural modeling produces perfect assets but requires human experts. Moving toward agentic procedural modeling is the logical middle ground. Here, an AI writes the code. But this faces a steep barrier.

Authoring these assets requires deep expertise. Designers must master domain-specific APIs and geometric reasoning. Existing benchmarks fail to capture this complexity. Some focus on editing pre-built scenes. Others limit themselves to simplistic voxel grids (3D pixel structures). Many evaluations are "single-shot." This means they ask the model to write code once and hope for the best. This ignores professional workflows. Real design is iterative. You write, you run, and you fix mistakes. Even when a model successfully generates a render, the geometry often suffers. It frequently shows disconnected or floating components.

How It Works

The authors propose a dual-pronged approach. They use a massive, curated dataset and a robust evaluation environment.

1. Agentic Data Curation: The researchers built a pipeline to transform complex "procedural factories" from the Infinigen simulator into clean Blender scripts. As shown in, this involves a "Skills Library." This includes a Code Simplifier and a Visual Critic.

An "Experience Library" prevents the agent from repeating known API errors. Human-in-the-loop verification ensures the triplets are high-fidelity. 2. Multimodal Benchmarking: The benchmark covers 212 object categories .

This includes organic creatures and manufactured items. It evaluates models on two tracks: text-to-3D and image-to-3D. 3. Iterative Evaluation: 3DCodeBench treats the VLM as an active agent. It tests the model's ability to engage in multi-turn refinement. The model receives execution-error feedback (tracebacks) to correct its code. 4. Human-Centric Ranking: Automated metrics often miss the "feel" of a 3D shape. Therefore, the authors launched 3DCodeArena. This is a public platform for pairwise human preference voting . It uses Elo ratings (a system for ranking players based on wins and losses) to score models.

Numbers

The most critical takeaway is the impact of test-time scaling. This refers to giving a model more resources during inference. The paper finds that multi-turn refinement is necessary for reliability. Implementing a stateless error-feedback loop lifts aggregate executability from 0.692 to 0.972. This is a $+27.2$ percentage point improvement. For an engineer, this means moving from a system that fails 30% of the time to one that almost always runs.

However, there are diminishing returns on intelligence. Increasing the "thinking budget" (more reasoning tokens) helps lightweight models. For example, Gemini 3.1 Flash Lite sees significant gains. Conversely, frontier models like Claude Opus 4.7 plateau early.

Regarding evaluation, the paper validates automated proxies. They found that SigLIP-2 (a vision-language similarity metric) predicts human preference well. It shows a Pearson correlation of $r = 0.964$ with Elo ratings .

This means you can use SigLIP-2 as a scalable substitute for costly human annotations.

What's Missing

The paper is honest about the current ceiling. Even when code is perfect, physicality is often broken. As seen in, models produce "floating" primitives or disconnected fragments.

The models understand API syntax. They do not yet understand the spatial relationships required for structural integrity.

The paper does not explore multi-asset environments. Currently, the task is "one prompt, one object." Real production pipelines require scene composition. Transitioning from single objects to full scenes is a significant leap. Finally, the authors do not dive deep into optimizing latency. Reducing the time in "write-run-render" loops is vital for interactive tools.

Should You Prototype This

Yes, but specifically for the agentic harness, not single-shot generation.

Do not rely on a single prompt to get a usable result. The data is clear: the win is in the loop. The "stateless retry" method is a low-hanging fruit. It provides a massive boost to reliability for almost any model.

If you are building a custom benchmark, use the 3DCodeBench dataset. The 3DCodeArena platform is also available at 3dcodebench.com. Start by implementing the error-feedback loop using the Blender traceback. This is the most cost-effective way to move from broken code to executable assets. Just do not expect perfect geometric reasoning immediately.

Figures from the paper