RoboStressBench: Benchmarking VLM Robustness to Physical Visual Stress in Embodied Scenes

Current Vision-Language Models (VLMs) are impressive in clean, curated benchmarks. However, they tend to fall apart in the messiness of a real room. Most evaluations use idealized images or simple digital noise—like Gaussian blur or pixelation. But a robot in a warehouse does not deal with pixel noise. It deals with specular reflections (mirror-like glares) on a plastic bin, heavy shadows in a corner, or a tool partially hidden behind a box.

Researchers developed RoboStressBench to bridge this gap. Instead of applying arbitrary digital corruptions, they use a framework inspired by the physics of image formation. This allows them to test how specific physical factors—like lighting, material properties, and object geometry—degrade a model's ability to see and reason.

The Problem

Standard robustness benchmarks typically rely on "ImageNet-C-style" corruptions. These involve algorithmic perturbations like shot noise or motion blur. While useful for general computer vision, these do not map well to embodied AI (AI that interacts with the physical world). As illustrates, there is a massive disconnect.

General benchmarks miss real-world stress entirely. Traditional robustness benchmarks focus on digital artifacts that rarely appear in a physical workspace.

Even when models encounter challenging scenes, existing benchmarks rarely annotate why the scene is hard. If a model fails to locate a cup, is it because the cup is transparent (Material), because the lighting is dim (Lighting), or because the cup is being blocked by a plate (Geometry)? Without this distinction, engineers cannot perform a root-cause analysis. Current evaluations provide an aggregate accuracy score that obscures these specific failure modes. This makes it impossible to know if a model needs better training on textures or better spatial reasoning for occlusions.

How It Works

The core of RoboStressBench is a taxonomy grounded in the inverse graphics perspective. The authors abstract image formation as $I = F(M, V, L, G)$. Here, the image ($I$) is a function of Material ($M$), Viewpoint ($V$), Lighting ($L$), and Geometry ($G$). This mathematical foundation allows them to categorize stress into sixteen fine-grained sub-types. Examples include "specular confusion" for reflective surfaces or "cluttered layout" for dense object arrangements .

The dataset curation follows a three-pronged pipeline : 1.

Filtering: Identifying naturally occurring stress cases from existing unconstrained datasets. 2. Synthesis: Creating targeted stress variants from "nominal" (clean) images. They use generative editors like Gemini-3-Pro-Image and Qwen-Image-Edit. They use "edit profiles" to specify exactly what to change while holding other variables constant. 3. Collection: Gathering real-world examples from internet sources and self-captured imagery.

Beyond just benchmarking, the authors propose StressDART. This is a test-time "agentic solver" (a system that uses reasoning steps to solve a task) designed to mitigate these stresses during inference .

The workflow is a three-stage loop. First, a Stress Detector identifies the dominant stressor, such as "Global Underexposure." Second, a Stress Rectifier applies a targeted visual operation, like brightening the image. Finally, a Reasoner processes both the original and the rectified image to produce a final answer.

Numbers

The authors find that current VLMs are nowhere near ready for high-stress embodied deployment. Even the top performer, Qwen3.5-35B-A3B, achieved an overall accuracy of only 58.1% [Table 2]. For context, many state-of-the-art commercial models like Gemini-3.1 and GPT-5.5 scored significantly lower. They obtained accuracies in the mid-40% range.

Crucially, the paper demonstrates that scaling model size is not a silver bullet for physical stress. Larger models generally show higher average performance. However, they still suffer from the same specific weaknesses. For instance, scaling Qwen3.5 from 4B to 27B parameters provides an 8.3% boost. Yet, these larger models still struggle with the same problematic categories [Table 2].

The most significant finding for engineers is the task-dependent nature of these failures . Geometry stress—such as occlusion and cluttered layouts—is the primary killer for localization tasks. This includes target grounding and spatial reasoning. Conversely, material and lighting stresses are more likely to disrupt high-level state understanding and recognition. Finally, the StressDART intervention provides a measurable lift. Using the Qwen3-VL-4B base model, moving from direct inference (43.2%) to using both original and rectified images (49.0%) yielded a nearly 6% absolute increase in accuracy [Table 3].

What's Missing

While RoboStressBench is a significant step toward realistic evaluation, it has notable gaps. First, the taxonomy is not exhaustive. In a real factory or home, stress factors are often deeply entangled. For example, changing a viewpoint ($V$) almost always changes the perceived lighting ($L$) and material reflections ($M$). The paper acknowledges that these axes are not perfectly orthogonal (independent) in practice.

Second, the benchmark is limited to static, image-based tasks. For an engineer building a mobile manipulator, the real challenge is temporal robustness (stability over time). This means maintaining perception as the camera moves through varying lighting and viewpoints. This benchmark does not currently evaluate closed-loop embodied behavior or video-based reasoning.

Lastly, the StressDART solver relies on generative image editing. This introduces a risk of "negative flips." This occurs when the rectification process accidentally alters or destroys task-relevant evidence. This can lead the model to a wrong answer.

Should You Prototype This

Yes, but as a diagnostic tool, not a replacement for standard testing.

If you are developing VLMs for physical hardware, you should integrate RoboStressBench into your evaluation pipeline. The ability to see that a failure is driven by "cluttered layout" (Geometry) rather than "low contrast" (Material) is vital. This allows you to make surgical decisions about your data collection and fine-tuning strategies.

However, do not rely on the StressDART approach as a production-ready "fix" for your perception stack. The 6% accuracy gain is modest. Additionally, the computational overhead of running a detector and a generative rectifier at test time will likely blow your latency budget. Use the benchmark to identify where your models are brittle. Solve those issues through better data and training, not through expensive test-time image editing.

Code is reportedly available; see the RoboStressBench project webpage for the canonical link.

Figures from the paper