OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs

Most multimodal agents—those destined for robotics, augmented reality, and autonomous driving—must navigate a continuous, egocentric (first-person) world. To function, they cannot simply analyze a static snapshot. They must build a coherent understanding of space from a constant, flowing stream of visual data. This requires more than just recognizing an object. It requires remembering where that object was even after it moves out of view. Eventually, the agent must construct a mental map of the entire environment.

Current research in multimodal large language models (MLLMs) has made massive strides. They can describe what is happening in a single frame or a curated set of images. However, a critical gap remains. Most existing benchmarks evaluate models on "offline" video. This means the model can look back at any part of the video at any time to find an answer. This ignores the "streaming" reality of an agent. In a real scenario, information is ephemeral and must be actively maintained in memory. The OVO-S-Bench paper aims to close this gap. It forces models to reason about space using only the video prefix (the portion of video preceding the query) that came before the question. This effectively simulates a real-time, causal observer.

The failure of offline spatial reasoning

The central problem the authors identify is a mismatch in benchmarking. Current spatial benchmarks do not meet the requirements of embodied intelligence. As shown in [Table 1], existing benchmarks fall into two problematic camps. Some focus on static images or multi-image sets. These strip away all temporal (time-based) context. Others target long-form video understanding. These focus on narrative events, such as "what happened next?" rather than structural spatial intelligence.

Even when benchmarks use video, they often allow "offline" access. This creates a loophole. A model does not actually need to understand or remember space. It simply needs to re-attend to any previous frame to find a visual cue. This bypasses the core challenge of spatial intelligence. That challenge is the transition from egocentric perception (what I see now) to allocentric mapping (understanding the world from a viewer-independent perspective). Without a streaming constraint, the difficulty of maintaining a persistent spatial state is never actually tested.

A hierarchy of spatial abstraction

To address this, the authors introduce OVO-S-Bench. It is structured around a four-level taxonomy of increasing cognitive load. Instead of treating "spatial intelligence" as a monolith, they decompose it into a progression of complexity. This is illustrated in :

1. L1: Instantaneous Egocentric Perception. This level tests the ability to understand the immediate scene. Can the model perceive geometry, local relations, and dynamic motion in the current view?
2. L2: Spatiotemporal Context Tracking. Here, the evidence appeared in the past but is no longer visible. The model must exhibit spatial memory. It must recognize if a scene has been revisited or locate an object that is now behind the camera.
3. L3: Generative Spatial Reasoning. This requires active mental operations. The model must simulate "what-if" scenarios through mental rotation. It must also verify consistency across time or plan a route through a perceived layout.
4. L4: Allocentric Spatial Mapping. This is the highest level of abstraction. The model must integrate the entire egocentric stream into a viewer-independent representation. It must answer questions about global topology (the connectivity of spaces), cardinal directions, or trajectory-to-map alignment.

By enforcing a strict streaming protocol, the authors ensure that higher levels cannot be solved via simple visual retrieval.

The allocentric bottleneck and the specialization trap

The results of the evaluation across 38 MLLMs reveal a significant gap. The authors report that Gemini-3.1-Pro, a highly capable proprietary model, achieves an overall accuracy of 59.2%. This trails human experts (who score 86.6% under the same streaming conditions) by 27 points. This gap represents a massive deficit in reliable spatial reasoning.

The most striking finding is that L4 (Allocentric Mapping) is the dominant bottleneck. The paper finds that L4 is the lowest-scoring level for 28 of the 34 systems evaluated. While models show some competence in L1 and L2, their ability to synthesize a global map from a continuous stream collapses. Interestingly, the authors note that this is not a simple matter of scale. As shown in, increasing parameter counts (the model's capacity) across the Qwen, InternVL, and Qwen3 families shows that L4 accuracy tends to plateau early. This suggests that scaling laws alone may not solve the fundamental problem of spatial abstraction.

Furthermore, the authors highlight a "specialization trap." They observe that 13 of 15 models specifically fine-tuned (training a model on a specific task) for streaming or spatial tasks actually performed worse than their vanilla backbone models. This suggests that current fine-tuning recipes may be counterproductive. They often focus on discrete-frame QA or narrative event memory. These methods may actually degrade the model's ability to maintain the long-range, cross-frame dependencies required for true spatial intelligence.

Unintended consequences of reasoning

The paper also investigates the impact of Chain-of-Thought (CoT) reasoning. CoT is a technique where a model generates explicit intermediate reasoning steps. The results are mixed. While CoT helps with L2 tasks by improving mean accuracy by ~3.9 points, it can be detrimental to L1 tasks.

More concerningly, the authors find that CoT can amplify spatial errors when the reasoning is ungrounded in the actual visual stream. As detailed in, failures in thinking-mode models frequently manifest as "non-visual" or "visual-content" errors.

In these cases, the model's internal monologue might confidently describe a spatial relationship that is not present in the frames. It effectively hallucinates a mental map to justify a wrong answer. This suggests that "thinking more" is not a substitute for "seeing better." This occurs when the model's internal world model is poorly calibrated to the visual input.

Verdict: Not yet ready for the real world

If you are developing an agent for a physical environment, the takeaway from OVO-S-Bench is clear. Do not trust current MLLM spatial reasoning capabilities for high-stakes autonomy.

The benchmark demonstrates that we are currently in a regime where models can perceive the "now" (L1) and perhaps remember the "then" (L2). However, they cannot reliably conceptualize the "where" (L4). The fact that specialized architectures often regress against their backbones is a major warning sign. It suggests we lack a principled way to teach models persistent, 3D-aware spatial memory. Until we see models that can bridge the 27-point gap to human performance—particularly at the allocentric level—we should view current "spatial" MLLMs with caution.

Figures from the paper