The Dark Regulome: Disentangling Predictability from Regulation in Genomic Foundation Models

High-grade gliomas are not merely passive masses of proliferating cells. They are active participants in the brain's electrical life. These tumors integrate into existing neural circuits by forming functional synapses with neurons. This creates a feed-forward loop where neuronal activity actually accelerates tumor growth .

While the proteins facilitating this hijacking are becoming well-understood, the upstream instructions remain a mystery. These instructions are written in the "dark genome" (the ~98% of non-protein-coding DNA).

Scientists use genomic foundation models to decode this dark regulome. These are large-scale AI architectures trained to understand the "grammar" of DNA. Researchers use a technique called in-silico mutagenesis (ISM). In this process, a digital model "mutates" a DNA sequence to see how it affects predicted gene expression. Researchers then rank these elements by their Regulatory Influence Score (RIS). This score measures how much a model's prediction changes when a specific DNA segment is removed.

However, a fundamental problem exists. These models often struggle to distinguish between biological importance and sequence predictability.

The confounding trap of sequence predictability

The promise of sequence foundation models is a "zero-shot" route to discovery. This means they can identify regulatory elements without specific training on a new disease. But likelihood-based scoring is inherently coupled to local sequence predictability.

If a model encounters a highly repetitive or patterned sequence, removing it will naturally cause the model's mathematical certainty to drop. This occurs with transposable elements (segments of DNA that can move within the genome). The model's certainty drops because the sequence was predictable, not necessarily because it was regulatory. Consequently, current ISM-based studies risk reporting "discoveries" that are actually just artifacts. They may simply be capturing the model's ability to recognize common genomic patterns. Without a way to disentangle these two layers, researchers cannot be sure if they are mapping cancer drivers or merely genomic noise.

Separating grammar from function

To resolve this ambiguity, the authors introduce a residualization-and-permutation diagnostic. This tool strips away the influence of sequence predictability. Their approach functions through three rigorous stages:

1. Residualization: For every candidate element, the researchers use ordinary least squares (a statistical method to find relationships between variables) to account for four "nuisance covariates." These include k-mer entropy (a measure of sequence complexity), GC content (the proportion of guanine and cytosine bases), element length, and the distance to the Transcription Start Site (TSS). By calculating the residuals, they isolate the variance that is truly unique to the element.
2. Permutation Testing: The authors evaluate findings against a "marginal-preserving permutation null." This involves shuffling element rankings within each gene thousands of times. This ensures that any observed overlap between different models is statistically significant. It prevents claiming a discovery based purely on large sample sizes.
3. Cross-Architecture Decomposition: The study uses three different models: Caduceus-Ph (a bidirectional masked language model), HyenaDNA (a causal language model), and Enformer (a supervised model). By comparing these architectures, the authors categorize signals into two distinct layers. They find a "sequence-predictability layer" shared by the language models and a "regulatory-output layer" captured by Enformer.

A sharp horizon and disjoint layers

The application of this diagnostic across 30,448 elements yields a stark decomposition of genomic signal. The two language models, Caduceus-Ph and HyenaDNA, share a massive amount of information. However, this information is largely a byproduct of sequence predictability. They co-rank long, well-predicted transposable elements. A simple six-feature linear baseline can predict which elements these models will rank highly with an AUC of 0.985 .

This high AUC means the baseline is nearly perfect at predicting the models' top choices.

In contrast, Enformer behaves differently. It was trained on experimental expression data rather than just predicting the next DNA "letter." Therefore, it retains a signal that identifies real regulatory elements (cCREs). Most strikingly, the authors report zero overlap at the top-100 level between the language models and Enformer . The language models identify "easy-to-read" DNA. Meanwhile, Enformer identifies "functional" DNA.

Despite this divergence, one robust biological signal survives all controls. There is a sharp 10 kb proximal-regulatory horizon. The influence of an element drops precipitously once it is more than 10 kb away from the TSS .

This transition is extremely steep. It marks the limit of how much a primary-sequence model can "sense" an element without accounting for 3D genome folding. Furthermore, the top-100 elements identified by all three models show a 3.3× enrichment for known brain cis-eQTLs (genetic variants known to affect gene expression in the brain). This provides a validated shortlist of candidates for future laboratory study.

Limits of the digital microscope

The diagnostic is powerful, but it has limitations. First, residualization cannot entirely rule out "pretraining memorization." A model might have seen specific repetitive families millions of times during training. Its reaction to those elements might still be biased by that familiarity.

Second, the study relies on a relatively small panel of 92 genes. While these are relevant to glioma, this scale is insufficient to map all complex interactions. Finally, these models are primarily sequence-based. They are inherently limited in detecting distal regulatory elements. These are elements that rely on 3D chromatin contacts (how DNA loops through space to touch a promoter). The 10 kb horizon likely represents the limits of the model's vision rather than the true extent of the regulatory landscape.

The verdict: calibrate your models

The verdict is clear. If you use likelihood-based language models to find regulatory elements, you are likely looking through a distorted lens. The researchers demonstrated that "convergence" between two language models is often just an agreement on predictability. It is not necessarily an agreement on biological function.

For practitioners, the takeaway is a mandate for rigor. One should not trust raw RIS rankings from models like Caduceus or HyenaDNA without a residualization diagnostic. True regulatory discovery requires moving beyond sequence grammar. Researchers must find signals that correlate with actual expression and biological variation. The authors have provided the calibration needed to turn these foundation models into reliable tools.

Figures from the paper