When Are Multimodal Predictions Biologically Supported? A Diagnostic Evaluation Framework

When AI models use multiple types of data—like medical images and genetic sequences—to predict cancer outcomes, they might be right for the wrong reasons. They might pick up on hospital-specific patterns or technical artifacts instead of genuine biology. This is a critical failure mode in oncology. A model that performs well in a lab might fail catastrophically when deployed in a different clinic.

Current state-of-the-art multimodal models aim to integrate these diverse data streams to find "shared biology." This refers to underlying disease signals visible in both imaging and genomics. However, we currently lack a way to verify if a model's high predictive accuracy is actually driven by this shared signal. It might instead rely on spurious correlations—accidental patterns that won't generalize. A new paper introduces DECAT, a diagnostic framework designed to pull back the curtain on these representations. The most striking finding is that common "entangled" architectures, like CLIP, frequently hallucinate shared biology even when none exists.

The Problem

The industry standard for evaluating multimodal models is predictive accuracy, typically measured via AUROC (Area Under the Receiver Operating Characteristic curve). AUROC tells you how well a model discriminates between classes. However, it is blind to the source of that discriminative power. A model can achieve a near-perfect AUROC by latching onto a "batch effect." This is a technical artifact like the specific staining protocol used at a single medical center.

As noted in the paper, these technical signatures often persist despite standard normalization. This creates a dangerous illusion of performance. A model looks robust in a controlled study but collapses under distribution shift (a change in data properties between training and deployment). Existing mitigations, such as preserving sites during cross-validation, require engineers to know which confounders to look for beforehand. If you don't know what the bias is, you cannot test for it.

How It Works

DECAT is a model-agnostic, post-hoc evaluation framework. "Post-hoc" means it does not require retraining the model. It operates entirely on the learned representations (the high-dimensional vectors produced by the model's encoders). The framework uses a rule-based decision tree to classify a modality's predictive behavior into four scenarios: Shared Biology (S1), Spurious Signal (S2), No Signal (S3), or Modality-Specific Biology (S4).

The core mechanism relies on five null-referenced metrics. These compare observed data against permutation-based "null" distributions (randomized versions of the data used to establish a baseline for chance). This ensures statistical significance. The decision process follows a structured hierarchy as seen in :

1. Geometry Check: It first verifies if there is any structural agreement between modalities using $A_{norm}$ (cross-modal shared latent agreement).
2. Signal Gating: It uses a permutation test to determine if the representation carries any task-relevant information. If it doesn't, it is labeled Scenario 3 (No Signal).
3. Localization: For "factorized" models—architectures that explicitly separate shared and private information—DECAT uses $\Delta_{shared}$ to see if the signal resides in the shared or modality-specific component.
4. Stability Testing: Finally, it evaluates cross-cohort stability. It checks if the predictive signal transfers to new cohorts ($P_{transfer}$) and whether the patient ordering remains stable across different cohort compositions ($D_{task}^{quantile}$, a metric using Wasserstein-1 distance to detect instability).

Crucially, the framework is designed to be conservative. If the evidence is insufficient, DECAT returns "indeterminate" rather than making a confident, incorrect claim.

Numbers

The authors' most sobering result concerns the reliability of modern model architectures. They report that "entangled" models (like CLIP or CCA), which merge information from different modalities into a single embedding, suffer from a massive False Shared Claim Rate (FSCR). This is the probability that DECAT incorrectly assigns Scenario 1 when it is not shared biology. On real TCGA (The Cancer Genome Atlas) foundation model embeddings, these models falsely claimed shared biology in 83–84% of cases where it was actually absent .

The paper demonstrates that this error rate is not just a fluke of small datasets. Instead, the FSCR increases with both signal strength and sample size. Larger, more "powerful" models become more confident in their incorrect diagnoses. In contrast, factorized models (like JIVE or DisentangledSSL) maintained a near-zero FSCR across all tested scenarios. They occasionally traded some sensitivity for this safety.

When applied to real-world TCGA data, DECAT proved capable of detecting confounding (Scenario 2) that was invisible to standard AUROC metrics. Specifically, the framework's ability to flag spurious signal (the S2 flag rate) showed high concordance (Spearman $\rho$ = 0.81–0.94) with actual performance collapses observed when stratifying data by cancer type .

What's Missing

While DECAT is a powerful diagnostic, it has clear boundaries. First, it is not applicable to "early-fusion" architectures. These are models where modalities attend to each other during the encoding process. In such cases, the representations are too deeply intertwined for DECAT's post-hoc analysis to untangle.

Second, the framework identifies that a signal is spurious, but not why. It can tell you that a model is relying on a batch effect (Scenario 2). However, it won't tell you if that effect is due to a specific scanner brand or a reagent lot. For a practitioner debugging a production pipeline, DECAT is a smoke detector, not a forensic investigator.

Finally, the paper notes that detecting "proxy" signals is significantly harder than detecting direct confounding. Proxy signals occur when a technical artifact is subtly entangled with biological features. The detection rates for these signals are notably lower. This suggests that the framework's effectiveness is sensitive to the geometric alignment of the bias.

Should You Prototype This

Yes, but only if you are building multimodal biomarkers for clinical deployment.

If you are using entangled models like CLIP to make claims about "shared biological insights," stop. The paper proves these claims are statistically suspect. If you are moving toward production, you should pivot to factorized architectures (like DisentangledSSL) and integrate DECAT into your validation suite. It provides a necessary safety layer that AUROC cannot offer. However, if your pipeline relies on early-fusion Transformers, DECAT won't work for you yet. You will need to wait for a framework that handles intra-encoding interactions.

Figures from the paper