State-dependent error correlations shape voting thresholds in committees of AI agents

The Myth of the Independent Jury

When organizations deploy a panel of AI agents to verify facts, review code, or screen decisions, they often rely on a simple principle: the wisdom of the crowd. The assumption is that while one model might hallucinate, a committee of ten models will cancel out those errors through a majority vote. This logic relies on the "Condorcet jury theorem." This theorem suggests that if agents make independent mistakes, the collective error rate drops toward zero as you add more members.

However, this assumption rarely holds in practice. Large language models (LLMs) frequently suffer from shared blind spots. They tend to fail on the same difficult prompts or exhibit similar reasoning flaws. Because these errors are linked, simply adding more models to a committee does not guarantee a smarter group. A new study from the Central University of Finance and Economics reveals that these correlated errors create an inescapable "error floor." This means no matter how large your committee grows, a certain level of mistake is mathematically guaranteed.

The failure of the independence assumption

Current multi-agent architectures typically operate under the assumption of independent errors. In these setups, engineers treat each model as a discrete, uncorrelated voter. This approach assumes that the probability of Model A failing on a specific math problem tells you nothing about Model B.

The authors argue that this is a fundamental misunderstanding of how LLMs behave. As shown in, error correlations are substantial.

When the same model is sampled at different temperatures (a technique used to introduce randomness into model outputs), the latent correlation is near-unit. This effectively acts as a single voter rather than a diverse committee. Even across different model families, the authors find significant positive correlations [Figure 2A–C].

When errors co-occur, the protective benefits of traditional voting rules erode. For instance, a "hierarchy" rule (requiring unanimity) is meant to suppress false acceptances. However, the authors report that positive correlation causes the false-acceptance rate to rise as models begin to err together [Figure 1B]. Similarly, the "polyarchy" rule (where a single approval suffices) loses its ability to rescue good proposals as models become more correlated [Figure 1B].

Modeling the shared blind spot

To address this, the authors develop a heterogeneous Gaussian-copula model. This mathematical framework moves away from the "independence" simplification. Instead, it treats error correlation as a measurable design input. The mechanism works by decomposing an agent's judgment into two parts: a diversifiable component (unique to that specific model) and an undiversifiable component (a shared latent factor representing common blind spots).

The modeling process follows these steps:

1. Parameter Estimation: The researchers use "tetrachoric inversion" (a statistical method to estimate underlying correlation from binary data) to calculate how often models agree or disagree on "good" versus "bad" cases.
2. State-Conditional Mapping: Crucially, the model allows correlation to differ depending on the state of the proposal. Models might be highly correlated when reviewing correct code (the "good" state) but even more likely to fail together on buggy code (the "bad" state).
3. Threshold Optimization: Once the correlations are mapped, the model calculates the "expected loss" for every possible $k$-of-$n$ voting rule. This allows the designer to select a threshold that minimizes the specific costs of the task.

As illustrated in [Figure 1A], this approach treats the "common factor" $W$ as the driver of shared errors. This prevents the error rate from ever reaching zero, regardless of committee size.

Evidence from massive model evaluations

The authors validated this model using a massive dataset of 174,384 votes. These were cast by 28 different language models across four critical domains: fact verification, mathematical solution verification, code review, and truthfulness screening.

The results demonstrate that accounting for dependence is far more accurate than assuming independence. The paper reports that the full-matrix dependence model achieved an identity-line $R^2$ of 0.967 when predicting loss on held-out items. This is a significant improvement over the independence model's $R^2$ of 0.840 [Figure 3A–C]. In this context, a higher $R^2$ means the model's predictions align much more closely with actual observed errors.

More importantly for practitioners, the study shows that modeling these correlations leads to better decision-making. In a design-balanced analysis, the authors found that: * Simple majority voting resulted in a scaled loss of 60.25. * Selecting a threshold based on an independence model reduced this to 52.50. * Selecting a threshold that accounts for actual dependence reduced it further to 50.77.

The authors conclude that the overall reduction in loss from moving from a simple majority to their dependence-aware threshold was 15.73% [Table S6]. This demonstrates that knowing how models fail together allows you to pick a much more effective voting threshold.

Limits of the committee approach

While the findings offer a roadmap for better AI oversight, the paper does not explore how to actively reduce these correlations. The study focuses on measuring and adapting to existing error patterns. It does not propose ways to diversify model training to break the correlation.

There are also technical limitations to consider. The authors note that correlation is "unidentified" (cannot be mathematically determined) for models that exhibit nearly constant voting patterns. Essentially, if a model says "approve" to everything, you cannot calculate its specific correlation with others. Furthermore, the method requires estimating complex correlation matrices. These often require "positive-semidefinite projection" (a mathematical regularization step) to ensure the matrices are logically consistent. For an engineer, this means the tool is powerful but requires more rigorous computation than simple majority counting.

The verdict: Optimize, don't just scale

If you are building an AI-driven decision pipeline, the verdict is clear: Stop assuming your models are independent. Simply throwing more models at a problem will yield diminishing returns. You will eventually hit the "Vasicek limit"—an irreducible error floor caused by shared vulnerabilities [Figure 2B].

Instead, you should treat committee design as a formal optimization problem. Rather than defaulting to a majority vote, you should measure the specific error correlations of your chosen roster. You must also account for the relative costs of mistakes. For example, is a false code approval more expensive than a missed bug? The authors' work provides a practical way to do this. Code is reportedly available; see the paper for the canonical link. If you want to minimize risk, you must design for the errors you know your models will share.

Figures from the paper