GE-Sim 2.0: Closing the Loop with a High-Fidelity Video World Simulator for Robots
They report topping the public WorldArena leaderboard with only 2B parameters. That sounds suspect—let us see why it is not. GE-Sim 2.0 outperforms much larger closed-source models like Sora and Veo. This suggests that domain-specific training on action-grounded robot data is more effective than raw model scale alone. It addresses a massive hurdle in robotics: the inability to scale training without the massive cost of physical hardware.
The Problem
Current robotic evaluation is stuck in a bottleneck. Real-robot benchmarking is slow and difficult to reproduce. Traditional simulators (like MuJoCo or Isaac Gym) attempt to solve this with physics engines. However, they frequently fail to capture messy contact dynamics or deformable objects. They often abstract away the robot's own actuation nuances, such as harmonic-drive compliance (the mechanical flexibility in robot joints).
Generative video models offer a workaround by synthesizing photorealistic footage. But they typically function as "view-only" predictors. They might show a beautiful video of a robot picking up a cup. Yet, they do not provide the internal state—the joint angles or gripper positions—that a modern Vision-Language-Action (VLA) policy needs. Without that proprioceptive feedback (the robot's internal sense of its own position), these models remain passive video players rather than functional simulators.
How It Works
GE-Sim 2.0 moves from simple video generation to a closed-loop system. It integrates three specialized modules that act as the eyes, the nerves, and the brain of the simulator. As shown in, the architecture is organized around two parallel "experts" that share the same underlying conditioning.
- The Vision Expert: This is an action-conditioned Diffusion Transformer (DiT). It takes in multi-view history frames and a planned action trajectory. The authors use a "Pose2Image" rendering technique and a per-pixel camera raymap (a map describing the direction of light for every pixel). This allows the model to distinguish between changes caused by camera motion and changes caused by object motion.
- The Proprioceptive State Expert: Because a policy needs to know where its arms are, this lightweight transformer branch runs in parallel. It consumes the intermediate latent features (compressed mathematical representations) from the vision expert. It then decodes the dual-arm joint angles and gripper states. This provides the "nerves" of the system.
- The World Judge: To automate evaluation, the authors implemented a VLM-based (Vision-Language Model) reward model. As illustrated in, this judge looks at the generated video frames.
It compares them against the original task instruction. It outputs a per-frame success signal. This acts as an automated referee for reinforcement learning.
To make this tractable, the authors use DMD2-based distillation (a method to compress complex models). This compresses the multi-step diffusion process into a few-step inference. They also use a "random-stride" training scheme. This allows the model to skip frames during inference to cover longer horizons.
Numbers
The authors report significant performance gains. Most notably, GE-Sim 2.0 tops the WorldArena leaderboard using only 2B parameters. This efficiency is a major takeaway for engineers managing compute budgets.
Regarding throughput—the metric that determines if you can use this for large-scale RL—the authors achieve a 25-frame rollout in just 2.3 seconds on a single H100 GPU. This speed makes large-scale parallel evaluation feasible.
In terms of reliability, the "state expert" is not optional. In closed-loop evaluations, including proprioceptive state conditioning improved the episode-level accuracy from 0.74 to 0.81 . It also boosted recall (the ability to correctly identify successful episodes) from 0.67 to 0.82. Furthermore, the "World Judge" outperformed the 122B-parameter Qwen3.5 VLM. It achieved 79% accuracy on simulated rollouts compared to Qwen's 60% [Table 3]. Finally, the utility is proven by real-world gains. Augmenting training data with filtered synthetic trajectories led to an average absolute gain of 0.150 in real-robot success rates .
What's Missing
There are clear gaps that a practitioner should consider.
First, the model still struggles with fine-grained contact and small-scale visual signals. The authors admit that detecting liquid levels or the movement of a flame remains an open challenge. If your task relies on micro-scale visual cues, this simulator might provide a false sense of security.
Second, the current implementation is tied to a fixed embodiment (the specific physical design of the robot). The training data covers specific dual-arm setups. The model currently lacks the cross-embodiment generalization required to jump between different robot types.
Third, there is an inherent risk of "compounding error" in autoregressive rollouts. Any generative model will eventually drift. If the simulator's errors do not align with real-world physics, the policy will learn to exploit simulator artifacts.
Should You Prototype This
Yes, but specifically for data augmentation.
If you want a replacement for a high-fidelity physics engine, this is not it. However, if you are struggling to collect diverse demonstration data, GE-Sim 2.0 is a strong candidate for a "synthetic data factory." The ability to generate high-reward trajectories and use them for filtered behavior cloning is highly practical. This approach yielded a 15% success rate boost in .
Code is reportedly available; see the paper for the canonical link at https://ge-sim-v2.github.io/. Start by testing it on a single task. See if the "World Judge" catches the specific errors you expect.