View Dropout: Making Visual Thinking Load-Bearing in Multimodal Models
Current AI models often "hallucinate" images as part of their reasoning process. They do not actually look at them to find the answer. This creates a gap in cross-view spatial reasoning. This task requires understanding how objects relate across different camera angles. A new paper from Mila and ServiceNow AI Research proposes a fix. It forces the model to actually use its generated visual intermediates.
The Problem
Cross-view spatial reasoning is a weak spot for vision-language models (VLMs). These models struggle to maintain a consistent scene representation across multiple viewpoints. Most attempts involve "thinking with images." This means generating an intermediate visual representation, like a depth map, to act as a reasoning bridge.
However, there is a representational mismatch. Even when models generate these images, they reason purely through language. They verbalize observations into text and discard the fine-grained geometry required for the task. The authors find that in standard supervised fine-tuning (SFT)—the process of training a model on labeled data—the "thinking-image" becomes a decorative byproduct. As shown in, the model takes a shortcut.
It looks at the raw input views to answer the question. This leaves the generated thinking-image unused. Dropping the thinking-image at inference barely changes accuracy .
This means the model hasn't learned to rely on its own "imagination."
How It Works
To make the thinking-image "load-bearing," the researchers introduce View Dropout (VDrop). This is a training-time intervention. It does not require changing the model architecture. It is agnostic to the specific type of image being generated.
The mechanism works as follows: 1. Targeted Masking: During training, the algorithm selects a contiguous rectangular region of one input view and masks it. This creates a hole in the visual evidence. 2. Asymmetric Visibility: This mask is applied to the attention mechanism of the answer tokens. It is not applied to the thinking-image tokens .
The model sees the full input views when generating the thinking-image. However, when predicting the final answer, that specific spatial evidence is hidden. 3. Forced Routing: Since the information is missing from the direct input path, the model must use the thinking-image. It is the only way to recover the missing spatial layout. 4. Annealed Curriculum: The authors use a warmup and linear annealing schedule for the masking probability. This prevents the model from collapsing during training. The model first learns to generate a coherent image. Then, it is gradually forced to rely on it for reasoning.
The authors evaluate which "mental imagery" works best using a Learnability–Informativeness (L–I) tradeoff. They compare three strategies: Panoramic (a unified wide-angle view), Top-down (an overhead layout), and Point-matching (annotating correspondences between views).
Numbers
The results suggest that the bottleneck is the training signal, not data scale. Using only 8K synthetic training samples from Infinigen Indoors, the panoramic configuration with VDrop achieves a 6.7-point out-of-domain (OOD) gain over the vanilla BAGEL model. Specifically, the panoramic strategy reaches a 40.0% OOD average accuracy. This outperforms prior methods like ThinkMorph (37.2%) and BAGEL-Zebra-CoT (26.8%). Those methods used much more data (3× and 23× respectively).
The authors use a "generate-then-blind" probe to prove causality. They mask the model's attention to the thinking-image at inference time. VDrop-trained models suffered significant accuracy drops. They saw up to a 10.1% drop in the "Measurement" category [, Figure 5]. Standard SFT models were largely unaffected. This confirms the thinking-image drives the logic. They also measured the attention share of the thinking-image across decoder layers. VDrop increases the model's reliance on the generated image, especially in the early and middle layers [, Figure 6].
What's Missing
There are several gaps a practitioner should note: * Architecture Specificity: The study uses the BAGEL UMM (a 14B parameter Mixture-of-Transformers). It is unverified if VDrop behaves similarly on other architectures. * Generation Fidelity vs. Reasoning: VDrop makes the thinking-image load-bearing. It does not inherently improve the image quality. If the underlying generator is poor, the model will rely on inaccurate mental images. * Real-World Noise: Training used procedural, synthetic data. Moving from perfect synthetic geometries to noisy, real-world sensor data remains a major challenge.
Should You Prototype This
Yes, if you are building embodied AI or agents that require spatial awareness.
If your roadmap includes robots that must reconcile multiple camera feeds, VDrop is a high-leverage trick. It is architecturally lightweight. You are only modifying the attention mask during SFT. The efficiency seen with 8K samples is critical if your data is expensive. However, do not expect it to fix a broken image generator. VDrop ensures the model uses its eyes; it does not give it better eyes. Code is reportedly available; see the paper for the canonical link.