Current World Models Lack a Persistent State Core

The Moon is Still There: Why AI Video Generators Can't Keep Track of Unseen Worlds

Current AI video generators are excellent at making pretty pictures. However, they frequently fail to remember what happened when the camera looks away. While these models can render convincing frames on demand, they lack a persistent internal understanding of the physical world. If a cat jumps onto a bed and the camera pans away, a true world model should show the cat on the bed when the camera returns. Most current models simply reset the scene or forget the event occurred.

This capability is a cornerstone of Artificial General Intelligence (AGI). An agent must understand that objects endure and events conclude regardless of observation. Existing benchmarks typically reward "surface properties" like visual fidelity, smooth motion, and camera controllability. However, these metrics fail to ask a critical question: does the generated world keep evolving once it is unobserved? A new study introduces WRBench, a diagnostic benchmark designed to expose this fundamental blind spot.

The gap between rendering and reasoning

The authors argue that current video generators act more like sophisticated "tracking shots" than true world models. In a tracking shot, the camera follows a subject to keep everything in view. In a real world, the camera can intervene by moving away, creating a period of non-observation. The study finds that when the camera turns away and returns, many models resume the target in the state it was in before the camera left. They fail to advance the event that was set in motion.

This failure is not a matter of poor image quality or shaky camera movement. The researchers report a "preservation–access–re-observed-consistency" gap. This means a model might render a beautiful frame (preservation) and move the camera successfully (access). Yet, it fails to ensure the object is in the correct state when it reappears (re-observed consistency). As shown in, camera motion primarily dictates whether the test can even be performed.

It determines if the object leaves the frame, but it does not improve the model's ability to remember the object's state.

Measuring the invisible with WRBench

To diagnose this, the authors developed WRBench. This hierarchical evaluation framework treats camera motion as an "intervention on observability" (a change in what can be seen). The methodology moves through several stages to pinpoint where a model breaks:

1. Control Execution: The benchmark checks if the model followed the requested camera path (requested-camera precision).
2. Visual Integrity: The system uses a DINOv2 feature proxy (a method using pre-trained visual features to detect structural inconsistencies like cuts or identity drift). This ensures the frames are not corrupted by artifacts.
3. Visible Consistency: While the target is still on screen, the benchmark asks if spatial relations and actions are correct.
4. Re-observation Support: This acts as a gate. It checks if the target returns to the view in a recognizable form after being hidden.
5. Re-observed Consistency: For clips that pass the previous gate, the benchmark asks if the returned target reflects the completed event.

By decomposing the evaluation this way, the authors can distinguish between a model that simply "lost" an object and one that "forgot" what the object was doing.

Scaling up doesn't solve the memory problem

One striking finding is that adding more parameters or better data does not fix state persistence. The authors evaluated 23 models, including various scales of the Wan model family. They report that scaling Wan from 1.3B to 14B parameters actually lowered re-observed state consistency from 0.66 to 0.62.

The research highlights that different physical changes pose different difficulties. The authors categorize events into "spatial relocation" (moving an object from A to B) and "in-place state changes" (changing an object's form, like folding a blanket, without moving it). Data suggests models struggle significantly more with the latter. As illustrated in, in-place changes impose a heavy penalty on performance.

Relocation provides a new coordinate for the model to track. Conversely, an in-place change offers no new spatial anchor. This causes the object to "drift or smear" in the model's internal representation.

Different architectural approaches offer different trade-offs. For instance, the Lingbot-World model achieves high visible consistency. However, it suffers from extremely low camera precision. Meanwhile, models using "geometry caches" (stored 3D maps of the scene) are better at returning the object to the frame. Yet, they still fail to reconstruct the changes that happened while the object was hidden.

Limits of the current diagnostic

While WRBench provides a rigorous way to measure these failures, the paper does not propose a definitive architectural cure. The authors acknowledge that their benchmark identifies the symptoms of a missing "state kernel." They have not yet demonstrated a model that fully solves it.

Practitioners should note that effectiveness depends on the "viewpoint condition" (the specific way a model is instructed to move the camera, such as via text or 3D data). Models receiving explicit 3D data or source videos have a much higher "access" rate. This means they can actually perform the test more often than models receiving only text prompts. Consequently, a high score on a text-to-video model might not be comparable to a video-to-video model. The former might simply be avoiding camera movements that trigger failures.

The verdict: A need for "what-memory"

The verdict of the study is clear. Current video generation is a feat of rendering, not a feat of simulation. To move toward true world models, the authors suggest researchers must stop focusing solely on how the next frame appears. Instead, they must focus on how the world unfolds.

They propose a "long-to-short" training recipe. Models should first learn persistence over long temporal horizons. Then, they should be fine-tuned with explicit supervision for camera execution. Until we move beyond "where-memory" (remembering where things are) and develop true "what-memory" (remembering what happened to them), AI agents will inhabit worlds that vanish the moment we turn our heads.

Resources

Code is reportedly available at: https://github.com/JinPLu/WRBench. The project page is located at: https://jinplu.github.io/WRBench.