On the Relationship Between Activation Outliers and Feature Death in Sparse Autoencoders

Why Your Interpretability Models are Wasting 70% of Their Capacity

When training AI to explain itself using Sparse Autoencoders (SAEs), many "features"—the individual directions the model uses to represent concepts—end up doing absolutely nothing. This phenomenon, known as "feature death," effectively wastes the dictionary capacity you paid for in compute and memory. It can even reintroduce the very superposition (the encoding of more concepts than available dimensions) the SAE was supposed to solve.

Previously, engineers treated feature death as a training dynamics problem. They tried to solve it by injecting more gradients or adjusting sparsity penalties. However, a new paper reveals that for many models, death is actually a geometric initialization problem. Identical SAE configurations can produce near-zero dead features on GPT-2 while producing over 70% dead features on AlphaFold3 [Figure 2a]. The reason is not the training loop. It is the activation geometry itself.

The Problem

Current state-of-the-art methods attempt to "revive" dead features using techniques like AuxK (an auxiliary loss) or ghost gradients (injecting synthetic signals). While these help in some regimes, they fail in high-death scenarios.

The issue is that many modern models possess "dimension-level activation outliers." Unlike per-token spikes, these are dimensions where the mean magnitude is massive compared to the per-token variation [Figure 2b]. When an SAE is initialized on these activations, the magnitude of the activation mean ($\mu$) shifts the pre-activations (the values computed before the sparsity nonlinearity) before a single gradient is calculated.

If a feature's weight vector is anti-aligned with this massive mean, its pre-activation is pushed into a permanently negative regime. In ReLU-based architectures, these features are dead-on-arrival. In TopK architectures, they are crushed by the competition. The features aligned with the mean hog all the activation slots .

How It Works

The authors identify a single metric to quantify this danger: $\gamma = |\mu|/|\sigma|$. This is the ratio of the activation mean's magnitude to the per-token standard deviation. This metric categorizes death into two distinct pathways:

1. Dead-by-ReLU: Features with large negative shifts ($w_i \cdot \mu \ll 0$) receive pre-activations that no input can pull above zero.
2. Dead-by-TopK: Even if a feature has a positive pre-activation, it may never rank in the top-$k$ selection. The features aligned with the mean dominate the competition .

Dead features can revive during training. However, the process is bottlenecked by how fast the SAE's bias term can learn to absorb the activation mean. Feature weights scale with the input magnitude, but the bias does not. Consequently, features capture the mean much faster than the bias can [Figure 5b]. At high $\gamma$, the bias learning is so slow that features plateau at 75–90% death even after millions of steps [Figure 5a].

The proposed solution is mean-centering. By initializing the SAE bias to the geometric median (a central point that minimizes distances to all points) of the activations, you subtract the mean offset at the start. This ensures no features are "born dead" from a mean shift.

Numbers

The results suggest mean-centering is a requirement for certain model families. The authors report that $\gamma$ is a highly reliable predictor of initial death rates. It shows Spearman correlations of $\rho = 0.89$ for dead-by-TopK and $\rho = 0.82$ for dead-by-ReLU across 454 model-layer combinations .

The impact on dictionary utility is significant. On ESM3 (a protein model), a mean-centered SAE with only 2,048 features recovered more biological concepts than a baseline SAE with 8,192 features [Table 1]. This means you get better results with 4x less dictionary capacity. Furthermore, mean-centered SAEs produce higher-quality, more monosemantic features (concepts that represent a single, coherent idea) .

What's Missing

The paper is rigorous, but there are a few gaps for practitioners:

• Optimizer-level failures: Mean-centering does not address death caused by "stale" Adam momentum. It also does not fix death caused by extremely high learning rates that push features below zero.
• Heavy-tailed distributions: The analytical derivation assumes Gaussian signal projections. In layers with high kurtosis (heavy tails), the formula may over-predict death. Rare, massive signal fluctuations can occasionally rescue a feature the math says should be dead [Figure S5].
• Low-rank corner cases: Mean-centering does not solve "variance concentration." In extremely low-rank models like Evo1, a few principal components carry nearly all the variance. Features still die because they cannot win the TopK competition [Figure S19]. For these cases, the authors suggest PCA whitening (equalizing variance across directions) is necessary.

Should You Prototype This

Yes. If you are training SAEs on anything other than standard language models, you should implement mean-centering. This includes vision, protein, or genomic models.

The implementation cost is negligible. Instead of initializing the SAE bias to zero, initialize it to the geometric median of your activation batch. This provides a massive boost in dictionary efficiency with zero runtime overhead. Before you commit compute to a large-scale SAE training run, calculate $\gamma$ for your target layers. If $\gamma$ is high, do not rely on auxiliary losses to fix the death rate. Center your activations instead.

Figures from the paper