From Activation to Causality: Discovery of Causal Visual Representations in the Human Brain

Scientists often mistake brain activity caused by background colors or shapes for actual concept recognition. In neuroscience, identifying which brain regions represent specific visual concepts—like faces or tools—is a fundamental challenge. Traditionally, researchers used "activation maximization." This involves finding regions that fire most strongly when a person sees a target category.

The problem is that strong activation doesn't equal representation. A brain region might light up when seeing "dogs" simply because dogs are often photographed on grass. In this case, the region is actually responding to "greenery" rather than the canine itself. This paper introduces BrainCause, an automated framework that moves from mere correlation to causal discovery. By using generative AI to create "counterfactual" images—where a concept like a face is removed but the rest of the image remains identical—the authors can prove whether a brain region truly understands the concept.

The Problem

Current methodologies for mapping the brain rely heavily on category contrasts. Researchers ask whether a voxel (a 3D unit of brain tissue) responds more strongly to a target category than to an average of other categories. While effective for broad classifications, this approach fails to account for semantic or visual "distractors" that co-occur with the target.

As illustrated in, activation-maximization methods identify regions with high responses to a target concept.

However, they cannot distinguish between the concept itself and correlated cues like background, color, or pose. This creates a significant reliability gap. The authors demonstrate that without causal validation, a large portion of previously identified localizations are effectively false positives. If you rely solely on activation, you aren't mapping the concept. You are mapping the easiest way for the brain to react to the stimulus.

How It Works

BrainCause replaces simple activation checks with a targeted causal testing pipeline. The architecture relies on a trifecta of modern AI models to build a custom, controlled dataset for every queried concept .

1. Causal Dataset Generation: The framework uses a Large Language Model (LLM, a type of AI trained on text) to generate prompts for three distinct stimulus types. First, Positive Images of the concept are synthesized using a text-to-image model (FLUX.2). Second, Semantic Negatives are created. These are images of concepts that are related to the target but are not the target itself (e.g., "beach" for the concept "surfing"). Third, Counterfactual Negatives are produced. These take a positive image and use an image editing model to remove the target concept while preserving the rest of the scene.
2. Verification: To prevent errors, a Vision-Language Model (VLM, an AI that understands both images and text) acts as a gatekeeper. It verifies that the target concept is truly absent from the negative images.
3. Scoring and Search: Instead of just measuring raw activation, BrainCause assigns each voxel a "causal score." This score is the average of the semantic-negative score (how much better the voxel responds to the target than to the hardest semantic distractors) and the counterfactual score (how much the response drops when the concept is edited out).

The system then searches for voxels where this causal specificity is maximized. This ensures the identified region is sensitive to the presence of the concept and the absence of its edits.

Numbers

The impact of shifting to causal scoring is best seen in the error rates. The authors report that activation-based discovery methods suffer from a false positive rate (FPR, the rate of incorrect detections) of 73.4%. By implementing causal ranking, BrainCause reduces that FPR to 23.0%. This change significantly cleans up the results. Simultaneously, it increases the true positive rate (TPR, the rate of correct detections) from 26.6% to 38.7% .

In terms of alignment with established neuroscience, the paper finds that BrainCause accurately recovers known functional regions. For instance, the authors measure a 99% voxel alignment accuracy for "Bodies" when looking at the top 100 voxels [Table 2]. Furthermore, the method demonstrates high granularity. shows that BrainCause can resolve spatially distinct representations for closely related concepts.

For example, it distinguishes between "handwritten text" and "logos" in the visual cortex.

Regarding implementation costs, the authors note the workload. The full pipeline takes roughly 2 hours on a single H200 GPU. This includes generating approximately 1,000 images per concept, scoring, and region proposal.

What's Missing

While the framework is powerful, it has clear dependencies on the current state of generative modeling.

• Generative Fidelity: The success of the counterfactuals depends on the ability of models like FLUX.2 to perform "clean" edits. The authors admit in Section 4.3 that failures occur. This happens particularly with broad properties like "sky" or "reflections." In these cases, semantic-negative generation fails to truly isolate the variable. If the AI cannot perfectly remove the concept without changing the background, the causal signal becomes noisy.
• Semantic Leakage: There is an issue where semantic-negative images still contain traces of the target concept. This limits the "hardness" of the negatives. It could lead to an underestimation of a region's true specificity.
• Complexity of Broad Properties: The paper notes difficulty in isolating generic visual properties like lighting contrast. For a practitioner, this means BrainCause is excellent for discrete objects. However, it may struggle with abstract visual textures or global illumination effects.

Should You Prototype This

Yes, if you are working in neurotechnology or BCI (Brain-Computer Interface) development. The ability to automate the design of high-information stimuli is a major leap over manual experiment design. However, do not treat the output as absolute truth immediately. Because the system is gated by the quality of the underlying LLM and Diffusion models, you should treat the "candidate representations" as hypotheses. They should be verified with physical fMRI scans rather than accepted as settled facts. If you have access to H200s or similar high-end compute, the 2-hour turnaround per concept makes this a viable tool for rapid experimental iteration. Code is reportedly available; see the project page for the canonical link.

Figures from the paper