The Dynamic-Probabilistic Consistency Gap in Chaotic Surrogate Modeling

Why Your AI Simulations Might Be Cheating

When training AI to simulate complex, chaotic systems like weather patterns, models often "cheat" to get better scores. They might achieve impressive short-term forecasting accuracy by ignoring the actual physics. They do this by suppressing the instabilities that define the system. This paper identifies why this happens. It introduces KAFFEE, a training framework designed to ensure a model's uncertainty estimates are physically grounded.

The Problem

In dynamical systems reconstruction (DSR), the goal is to learn a neural surrogate. This surrogate must reproduce the long-run "attractor" geometry (the structural shape of the system's behavior). Most probabilistic approaches use finite-horizon objectives. These involve minimizing Negative Log-Likelihood (NLL) over a short window of future steps.

The authors identify a "Dynamic-Probabilistic Consistency (DPC) gap." Chaotic systems naturally expand uncertainty along unstable directions. Standard Gaussian objectives penalize this growth via a log-determinant term (an "automatic Occam's razor"). To minimize this penalty, the optimizer finds shortcuts. As shown in, these shortcuts manifest in three ways: core collapse, where the model shrinks its learned Jacobians (local derivatives governing expansion) to prevent uncertainty from blowing up; noise masking, where the model inflates its stochastic "shell" to absorb errors; and blind uncertainty, where the model's uncertainty decouples from the local dynamics.

Essentially, the model improves its score by breaking the physics.

How It Works

The proposed solution, KAFFEE (Kalman-Aware Framework For Ergodic Emulation), moves away from open-loop rollouts. It uses a differentiable Extended Kalman Filter (EKF) framework. The core philosophy is to evaluate likelihood on local predictive residuals—called innovations—rather than on long-horizon, unconditioned trajectories.

The architecture operates in two main stages:

1. Dynamically Grounded Prediction: Instead of just rolling out a mean, KAFFEE propagates a full covariance matrix through the learned local Jacobians ($\mathbf{J}$). This ensures uncertainty is tied to the same stretching and contracting forces that govern the deterministic core.
2. Localized Innovation Scoring: Rather than scoring how far a 10-step rollout drifts from the truth, KAFFEE uses the EKF to assimilate observations at every step. It calculates the likelihood of the innovation—the difference between the predicted state and the actual observation—after the uncertainty has been "reset" by the data assimilation step.

By scoring the innovation, the "volume pressure" that causes core collapse is localized. As demonstrated in [Figure 2g], this allows the model to maintain a healthy, expanding uncertainty during unstable episodes.

Numbers

The authors validate KAFFEE using the hyperchaotic Lorenz-96 system and a foundation model called DynaMix. On the Lorenz-96 ablation, KAFFEE is the only method that successfully balances predictive calibration with dynamical fidelity. The authors use $G_{tan}$ (a Spearman correlation metric checking if uncertainty growth matches true local Jacobian expansion) to measure this. While baseline methods like "No-transport" yield a $G_{tan}$ near zero, KAFFEE achieves a significant positive correlation of $0.475 \pm 0.004$ [Table 1]. This indicates that KAFFEE's uncertainty actually tracks the physical expansion of the system.

Regarding the DPC gap in foundation models, the authors show that when adapting DynaMix to 13 different chaotic systems, KAFFEE-DynaMix preserves most of the zero-shot dynamical properties. It achieves competitive predictive scores while avoiding the significant drift from the pretrained attractor seen in open-loop fine-tuning [Table 3].

What's Missing

There are a few practical gaps for a production engineer. First, the paper relies on the EKF's local Gaussian approximation. This may fail in highly non-linear or multimodal regimes. In such cases, the true posterior might not be a simple ellipsoid.

Second, the computational cost is a concern. The EKF involves dense covariance transport. This scales cubically ($O(M^3)$) with the latent dimension $M$. For a small 20D system like Lorenz-96, this is trivial. However, for high-dimensional latent spaces, the overhead of propagating a full $M \times M$ matrix will be significant.

Finally, the paper focuses on "in-context" adaptation of existing models. It does not extensively explore the stability of training a massive, high-dimensional transformer-based surrogate from scratch using this EKF objective.

Should You Prototype This

Yes, if you are building scientific surrogates for chaotic systems. If your model is intended for long-term simulation rather than just short-term point forecasting, standard NLL training is likely sabotaging your physics.

The implementation logic is straightforward. Swap your open-loop loss for an innovation-based loss. Implement a differentiable EKF for covariance transport. The authors have made the DynaMix code available; see the paper for the canonical link to the DurstewitzLab/DynaMix-python repository. If you are working in moderate dimensions ($M < 100$), the cubic cost is a fair price for a model that respects the laws of motion.

Figures from the paper