Indirect Variational Inference: Applications to Earnings Dynamics

Correcting the Bias in Earnings Dynamics

Researchers have found that a popular machine learning method called Variational Inference (VI) often produces biased results when studying complex income patterns. They have developed a new method called Indirect Variational Inference (IVI). This method uses biased results as a starting point to find the true, accurate values. It does this without needing significantly more computing power.

In economics, understanding how individual earnings evolve over time is essential. It helps in designing social insurance and studying wealth inequality. Most modern models treat earnings as a combination of two parts. These include a persistent component (long-term career trends) and a transitory component (temporary shocks like illness). Because these underlying trends are "latent"—meaning they are unobserved variables—economists must use complex mathematical tools to infer them.

The current state of the art often involves calculating a "likelihood." This is a mathematical function describing how likely the observed data is, given a specific model. However, for sophisticated, nonlinear models, this calculation becomes computationally impossible. This happens as the number of time periods or individuals grows. To solve this, many have turned to Variational Inference (VI). This technique replaces difficult integration with a simpler optimization problem. It essentially uses the Evidence Lower Bound (ELBO), a proxy for the likelihood, as its objective.

The catch is that VI is only as good as the "variational family" it uses. This is the mathematical shape it assumes the hidden data follows. If that shape is too restrictive, the model will be systematically biased. This paper identifies how that bias occurs and provides a way to fix it.

The Failure of Simplified Assumptions

The core problem with standard VI is the "family gap." When researchers use a simplified model to approximate a complex reality, they create a gap. This exists between the true log-likelihood and the ELBO. As the authors demonstrate in, this gap is the primary driver of estimation error.

One common simplification is the "mean-field" approximation. This assumes that latent states at different points in time are independent. In the context of earnings, this is akin to assuming your income today has no relationship to your income yesterday. The authors report that this assumption leads to massive errors. In a simulated linear Gaussian model, the mean-field approach produced a persistence parameter ($\rho$) of only 0.76. This is significantly lower than the true value of 0.90 (see [Table 1]).

Furthermore, the authors find that the bias is not merely a byproduct of "amortization" (sharing parameters across observations via neural networks). Instead, as shown in, the "family gap" dominates the "amortization gap." This suggests that simply adding more neural network capacity won't fix the fundamental inaccuracy of a poorly chosen variational family.

De-biasing via Indirect Inference

To resolve this, the authors introduce Indirect Variational Inference (IVI). Rather than seeking a perfect, complex variational family, IVI treats the biased VI estimate as an "auxiliary model." It uses the flawed VI results as a compass to navigate toward the truth.

The mechanism works through a process of calibration:

1. The Auxiliary Step: The researcher first runs standard VI to get a biased estimate of the model parameters ($\theta_{VI}$).
2. The Binding Function: The authors define a "binding function," $b(\theta)$. This maps a true parameter value to the biased value that VI would produce.
3. The Correction: The IVI estimator searches for the parameter value where the binding function most closely matches the actual VI estimate.

The authors propose two ways to implement this. The first is gradient descent, which requires calculating derivatives of the binding function. The second is a fixed-point iteration. This method uses a "damping factor" ($\kappa$) to iteratively nudge the biased estimate toward the true value. As illustrated in, this trajectory moves the estimate from the VI optimum toward the true parameter value in just a few steps.

Because IVI relies on the behavior of the VI estimator rather than the likelihood, it remains computationally efficient.

Reliable Recovery in Complex Environments

The authors tested IVI across increasingly difficult scenarios. These ranged from simple linear models to highly nonlinear, non-Gaussian environments. In the most complex simulations, IVI successfully recovered parameters that standard VI missed entirely.

In a nonlinear model with heavy-tailed transitory shocks, standard VI failed. It severely underestimated the kurtosis (the "thickness" of the tails) of those shocks. However, the authors show that IVI corrected these discrepancies. It delivered estimates remarkably close to the true values (see [Table 2]).

The strength of IVI is evident when scaling the time dimension. In a simulation with 40 time periods ($T=40$), VI began to struggle with bias. However, IVI maintained high accuracy. This suggests IVI is suited for the "long panels" found in modern administrative datasets. Such datasets make traditional likelihood-based methods prohibitively expensive.

Empirical Evidence from the PSID

The authors applied IVI to the Panel Study of Income Dynamics (PSID). This is a long-running U.S. household survey. They estimated a model with nonlinear persistence, individual-specific shock variances, and serial correlation in transitory shocks.

The PSID results reveal several striking features of American earnings. First, the conditional volatility of earnings follows a U-shape. Risk is highest for both the lowest and highest earners. Middle-income earners experience more stability. Second, they found evidence of serial correlation in transitory shocks, with a coefficient of 0.26. Finally, earnings persistence is nonlinear. As shown in [, Panel a], persistence drops significantly for high-income households experiencing negative shocks.

The IVI estimates provided a clearer picture than standard VI. For instance, the curvature of the volatility function was understated by standard VI. IVI captured this curvature accurately (see ).

Assessing the Limits of IVI

Despite its success, IVI has two significant constraints.

First, the success of fixed-point iteration depends on a "contraction mapping" assumption. This means the binding function must be sufficiently smooth. If the mapping is too erratic, the iterative nudges might not converge.

Second, while IVI avoids computing the full likelihood, it requires simulating data. This is needed to estimate the binding function. This adds a layer of "simulation noise" to the process. You exchange the impossible task of integration for the manageable task of repeated simulation.

The Verdict

If you work with complex, latent-variable models and need to scale to large datasets, the answer is yes, use IVI.

The research provides a clear roadmap. Avoid overly simplistic "mean-field" assumptions in your variational posterior. These create systematic biases. Instead, use a flexible Gaussian family and apply the IVI correction. This bridges the gap between machine learning efficiency and econometric rigor. The method is ready for settings where temporal depth and cross-sectional breadth make traditional methods untenable.

Figures from the paper