iVGR: Internalizing Visually Grounded Reasoning for MLLMs with Reinforcement Learning

Current multimodal large language models (MLLMs) often struggle when forced to draw bounding boxes (rectangular coordinates defining an object's location) while they are thinking. This difficulty arises because modern models attempt to solve two disparate problems at once. They must identify exactly where an object is in pixel space and reason about what that object means. Most existing research assumes that forcing the model to output explicit coordinates will improve its perception.

However, a new paper argues that this requirement actually creates a bottleneck. Mandating explicit grounding (linking text to specific pixel coordinates) can introduce task interference. This distracts the model from its primary goal of answering the question. The researchers propose iVGR, a method that uses reinforcement learning (RL, a machine learning paradigm based on rewarding desired actions) to "internalize" these localization skills. Instead of outputting messy coordinates during inference, the model learns to "see" the objects internally within its standard textual reasoning process.

The Problem

The status quo in fine-grained multimodal reasoning typically follows one of two paths: tool-assisted cropping or explicit bounding box generation. In tool-based approaches, the model invokes an external utility to zoom in on a region [Figure 1(a)]. In explicit grounding approaches, the model interleaves <box> tags directly into its Chain-of-Thought (CoT, a technique where the model explains its reasoning step-by-step) [Figure 1(b)].

The authors find that both methods suffer from fundamental friction. When a model is forced to commit to specific coordinates, it often suffers from "task interference." The cognitive load of precise localization detracts from the accuracy of the final answer. The paper provides empirical evidence for this. For models like TreeVGR, textual CoT consistently outperforms grounded CoT across almost all levels of localization quality [Figure 2b]. Even when the model gets the box mostly right, the act of generating those coordinates can degrade the reasoning. Essentially, the mandatory requirement to be a "pointer" makes the model a worse "thinker."

How It Works

The iVGR framework moves away from mandatory coordination. It uses a dual-stream reinforcement learning strategy to transfer skills from a "grounded" teacher to a "textual" student .

1. Dual-Stream Rollouts: During training, the policy MLLM generates two different reasoning paths for every query. The "grounded stream" is prompted to use explicit <box> tags. The "textual stream" is prompted to perform standard natural language reasoning.
2. The Grounded Stream (Teacher): This stream is optimized using a reward function ($R_b$) that combines format adherence, answer accuracy, and a bidirectional IoU (Intersection-over-Union, a metric measuring the overlap between two bounding boxes) matching metric.
3. The Consistency Reward (Alignment): This is the core innovation. To ensure the textual stream actually learns how to see, the authors use an external LLM judge to compare the textual reasoning against a high-quality grounded reference. If the textual description matches the semantic content of the successful grounded trajectory, the model receives a consistency reward ($\alpha$).
4. Rollout Archive: Because the model is learning dynamically, the authors maintain an archive of the best-performing grounded trajectories for each sample. This prevents the textual stream from chasing "noisy" or incorrect visual guides [Figure 4d].

By the end of training, the model has learned to associate specific linguistic descriptions with the spatial features it observed in the grounded stream. It absorbs the localization capability without needing to output coordinates at inference time.

Numbers

The authors report significant gains in fine-grained perception. When applied to Qwen2.5-VL-7B, iVGR achieves an average accuracy of 81.1% on fine-grained VQA (Visual Question Answering), outperforming the base model by 10.0% [Table 2]. On the V* benchmark, the authors measure a +7.9% delta over the baseline.

The method also scales with model size. Moving from a 7B to a 32B parameter model (using the Qwen3-VL backbone) results in an average improvement of 3.3% [Table 2]. The training cost is relatively modest for a reinforcement learning setup. The authors note that training the 7B model took approximately 39 hours on 8 NVIDIA A100 GPUs. This was only a 4-hour increase over a standard single-stream training run. This suggests the dual-stream overhead is manageable in a production training pipeline.

What's Missing

While the results are strong, there are a few gaps for practitioners. First, the method relies heavily on bounding-box annotations for the initial training phase. While the authors suggest using off-the-shelf detectors for pseudo-labeling, the quality of the "internalized" reasoning depends on the initial grounded stream. If the detector is poor, the consistency reward might teach the model to reason about incorrect visual features.

Second, the reliance on an external LLM judge (like Qwen2.5-72B) for the consistency reward introduces a potential "judge hallucination" risk. If the judge incorrectly assigns a high consistency score to a flawed reasoning chain, the error will be baked into the policy. The authors acknowledge this. They note that scaling the judge model is a primary way to mitigate this, though it adds complexity to the training infrastructure.

Finally, the paper does not extensively explore the latency implications of the "Tool-Assisted Test-Time Scaling" workflow. They show that adding crops and union crops can boost accuracy by up to 3.5% on HR4K [Table 4]. However, this comes at the cost of multiple inference passes and image preprocessing. This could be a concern for real-time applications.

Should You Prototype This

Yes, if you are building high-precision visual inspection systems. If your use case involves reading small text on labels or counting tiny components, this is a massive win. The ability to move from "clunky coordinate output" to "sharp internal reasoning" is highly valuable. The fact that the model remains compatible with tool-assisted workflows means you can use the fast textual path for general queries. You can then trigger the expensive "crop + union crop" path only for high-uncertainty cases.

Code is reportedly available at https://visual-ai.github.io/ivgr/. If you have the budget for a few days of A100 time, the performance jump on fine-grained benchmarks justifies the experiment.

Figures from the paper