Mitigating Perceptual Judgment Bias in Multimodal LLM-as-a-Judge via Perceptual Perturbation and Reward Modeling

Why Your AI Judges Are Hallucinating the Truth

When we use large multimodal models (MLLMs) to automate the evaluation of other models, we delegate quality inspection to an AI. Ideally, this inspector looks at the image and flags discrepancies. However, a critical failure mode exists. When visual evidence conflicts with textual cues, MLLM judges reward plausible narratives over perceptually correct answers.

In vision-language reasoning, we rely on the "LLM-as-a-Judge" paradigm to scale evaluation. Human evaluation is too slow as models grow. We use models to grade other models instead. The central problem is ensuring these judges actually "see" the image. They must not just read the response text. This paper addresses this gap by identifying "Perceptual Judgment Bias" and proposing a training framework to force visual grounding.

The Problem

Current judges fail in two distinct ways. The authors formalize this as Perceptual Judgment Bias .

First, "insufficient perceptual capability" occurs when the judge misperceives the image. It rewards a wrong answer because it cannot see the truth [Figure 1a]. Second, "response-anchored judgment reasoning" is more insidious. Here, the judge perceives the image correctly but ignores its own eyes. It favors the linguistic flow of the response instead [Figure 1b]. If a response says "there are nine green particles" and sounds confident, the judge anchors on that text.

The paper quantifies this bias. Baseline judges can identify errors when both perception and reasoning are broken. However, their accuracy drops significantly when the error is isolated to perceptual inconsistency [, right].

This shows that strong performance on compound errors does not imply genuine visual grounding.

How It Works

The authors propose "Perception-Judge." This framework uses reinforcement learning (RL)—a method where models learn through trial and error—to enforce perceptual verification. The method uses two main components.

1. The PPJD Dataset: To teach a model to spot lies, you must show it how they are built. The authors built the Perceptually Perturbed Judgment Dataset (PPJD). They took correct responses ($r_c$) and created two types of "counterfactual" negatives. The first ($r_{rp}$) changes only a visual attribute, like color or count. The reasoning stays intact. The second ($r_{rp+r}$) changes both the visual attribute and the reasoning logic .

This creates a verifiable hierarchy: $r_c \succ r_{rp} \succ r_{rp+r}$.

1. GRPO-based Reward Modeling: The authors use Group Relative Policy Optimization (GRPO). This is an RL algorithm that estimates advantages by comparing multiple responses within a group. It avoids the need for a separate value network (a component used to estimate the value of a state).

2. Verifiable Batch Rewards: The reward function is hierarchical. It first checks for structural validity. It ensures the model uses the required `

Figures from the paper