When the brightest is not the best: illuminant estimation from the geometry of specular highlights

When looking at shiny objects, people don't just look for the brightest spot to figure out the color of the light hitting them. Instead, they use the shape and position of shiny highlights. They do this even if those highlights land on darker parts of the object.

This ability is central to colour constancy—the phenomenon where we perceive an object's color as stable even when the lighting changes. For decades, a dominant theory in vision science has been the "brightest is white" heuristic. This assumption posits that the visual system identifies the brightest element in a scene. It assumes this element must be a white surface or a mirror-like specular highlight (light that bounces off a surface at a specific angle). The system then uses that element to calibrate the color of the light source.

However, a new study from the University of Oxford challenges this simplicity. The researchers report that human observers can actually override this "brightest is white" rule. They do this by exploiting the geometric structure of specular highlights. This discovery suggests that our brains perform much more sophisticated spatial reasoning than a simple intensity-based search would suggest.

The failure of the brightest-is-white heuristic

The "brightest is white" assumption works well when a scene is simple. It breaks down in complex environments. Consider a glossy object with a high-contrast texture, such as a patterned ceramic vase. If the specular highlight falls on a dark part of the texture, the brightest point might not be the highlight. Instead, it might be a nearby light-colored patch of the material.

As shown in, this creates a "spatially-separated" condition.

In these cases, the brightest pixel is heavily contaminated by the object's diffuse component (the light scattered in many directions by the surface). This makes it a poor proxy for the light source's color. If the visual system strictly followed the "brightest is white" rule, it would pick the wrong color. This would lead to errors in how we perceive the object's true shade. The authors argue that traditional models fail to account for how humans navigate these ambiguities.

Decoding light through specular geometry

To test whether humans use something more sophisticated, the authors designed an experiment using computer-rendered multispectral scenes. They presented observers with 1.5-second animations of a textured sphere. During these animations, the colors shifted in one of two ways. Either the light source changed (an illuminant change) or the sphere's material changed (a reflectance change). The task was "operational colour constancy." This meant observers had to decide which of those two things had happened.

The researchers manipulated the scene to decouple brightness from the highlight center. They used the Ward reflectance model to ensure every point on the sphere was a mixture of diffuse reflection and specular reflection. By rotating high-contrast Perlin noise textures across the sphere, they forced the specular highlight to land on either a bright or a dark region.

The core mechanism being tested is whether humans can locate the center of the specular highlight. This is the point where the light's spectrum is most pure. This must happen even when that point isn't the brightest pixel in the image. To prove this wasn't just a matter of looking at raw color statistics, the authors also included "phase-scrambled" images. These images kept the same global brightness and color distribution. However, they destroyed the spatial arrangement of the highlights. This turned the structured reflection into meaningless noise.

Evidence for a geometry-based strategy

The results strongly favor a geometric strategy over a simple brightness search. The authors report that for matte surfaces (zero specularity), performance was near chance. This means observers could not distinguish between light and material changes. However, as specularity increased, performance improved sharply. Most notably, the authors find that observers performed better in the spatially-separated condition. This occurred when the highlight sat on a dark texture. This is the exact opposite of what the "brightest is white" model predicts.

The researchers quantified this using $d'$, a metric for discrimination sensitivity. As seen in, human sensitivity peaked when specular highlights were present and well-structured.

When the highlight geometry was disrupted via phase scrambling, performance dropped significantly [Figure 4b]. This drop suggests that humans are not just calculating the average color of the scene. Instead, they are actively searching for specific geometric landmarks.

To verify this, the authors simulated two different computational observers. One followed the "brightest element" heuristic. The other focused on the "center of specular highlight." The paper finds that the brightest-element model failed to track human performance. It correlated only weakly ($r = 0.34$) with the actual data. In contrast, the highlight-center models showed a remarkable correlation with human behavior. They reached as high as $r = 0.95$ .

This suggests the human visual system behaves much more like a model that tracks geometric centers than one that tracks peak intensity.

Limits of the simulated environment

While the findings are compelling, the study operates within a highly controlled digital vacuum. The authors acknowledge several limitations. First, the stimuli are isolated. They lack the "rich surround" of a natural environment. This includes shadows, mutual illumination between objects, and complex backgrounds. In a real forest or a living room, these additional cues likely interact with specular highlights to bolster colour constancy.

Second, the study focuses on a single object. The authors note that they have not yet explored how these mechanisms generalize to scenes containing multiple objects. This includes objects of varying shapes and materials. For a practitioner building computer vision systems, this means the "geometry-aware" approach might require more complex integration than a simple single-object detector. Finally, the task is purely discriminative. It measures the ability to tell the difference between light and material changes. It does not measure the ability to accurately name a specific color.

The verdict: Move beyond peak intensity

If you are developing computer vision algorithms for white-balance or automatic color correction, the verdict is clear. Stop chasing the brightest pixel. The evidence from this study suggests that relying on peak intensity is a fragile strategy. It fails precisely when the environment becomes visually complex.

Instead, successful models should attend to the location and structure of specular highlights. By identifying the geometric center of reflections, an algorithm can extract a much cleaner sample of the illuminant. This works even when that reflection is buried in a dark or highly textured region. The transition from "intensity-based" to "geometry-aware" estimation is likely a necessary step. This move brings machine perception closer to the robust, context-sensitive capabilities of the human eye.

Figures from the paper