AI, Take the Wheel: What Drives Delegation and Trust in Human-Computer Cooperative Question Answering?

AI, Take the Wheel: Uncovering the Drivers of Trust and Delegation in Human-AI Teams

Researchers held a competitive trivia tournament where humans worked with AI teammates. They found that while teams often win together, humans sometimes ignore good AI advice (under-reliance) or follow bad AI advice (over-reliance) due to biases and poor explanation quality.

The Problem

As we move toward agentic workflows (automated sequences of AI tasks), the bottleneck is shifting to human-AI coordination. Current research often treats trust as a post-hoc metric. This means asking users if they liked an AI's output after the fact. Such studies use synthetic laboratory tasks that lack real-world pressure. This misses how humans actually delegate authority or adopt suggestions in real-time.

Existing approaches frequently fail to distinguish between two different reliance behaviors. First is proactive delegation (deciding whether to let an AI act autonomously). Second is deliberative adoption (evaluating a suggestion before committing). Because these processes are decoupled, a system might be great at providing advice but terrible at being trusted to act on its own. Prior work has rarely studied these two patterns together in a single environment. This leaves a gap in our understanding of how humans manage the trade-offs between autonomy and oversight.

How It Works

To capture these dynamics, the authors developed a competitive trivia tournament. This forces humans to make strategic choices under pressure. The architecture centers on two distinct question types :

1. Tossup Phase (Proactive Delegation): In this phase, teams decide whether to allow an AI teammate to "buzz in" and answer a question autonomously. Humans can proactively "mute" an AI to prevent it from acting without oversight. This measures the human's belief in the AI's reliability before any output is seen.
2. Bonus Phase (Deliberative Adoption): If a team wins a tossup, they enter a collaborative phase. The workflow follows a rigid three-step loop .

First, humans make an initial guess. Second, they see AI suggestions with confidence scores and textual explanations. Third, they make a final consensus decision.

The researchers utilized 16 diverse AI agents. These ranged from single-model calls to complex multi-step pipelines. Models included GPT-4o, Claude 3.5 Sonnet, and DeepSeek V3. This variety forced humans to build mental models (internal representations of how a system behaves) of different AI personalities through observation.

Numbers

The authors find that human-AI collaboration is generally synergistic. It performs better than humans or AI alone. However, it is plagued by asymmetric errors. The paper reports that under-reliance (missing 3.9% of opportunities to adopt a correct AI answer) is more frequent than over-reliance (being misled by an incorrect AI suggestion 1.7% of the time) [Table 1].

A critical finding involves the psychological drivers of these errors. The study shows that confirmation bias is a massive factor. This bias occurs when humans favor information that reinforces their existing beliefs. When an AI suggestion agrees with a human's initial incorrect answer, the under-reliance rate spikes to 64.5% [Table 5].

Furthermore, the authors highlight a disconnect between what makes an AI correct and what makes a human trust it. While LLM-assessed features like "Question Understanding" (76%) and "Reasoning Coherence" (72%) are strong predictors of actual AI correctness, humans rely on surface-level signals. These include the presence of quotes (70%) or semantic similarity (66%) .

What's Missing

The study is constrained by its domain specificity. The competitive trivia setting provides high engagement. However, the findings may not translate to high-consequence professional environments. Examples include medical diagnosis or legal review. In those fields, the cost of an error is much higher. Temporal pressures also differ significantly.

Additionally, the research is observational. The authors could not implement randomized interventions. For example, they did not purposefully feed a user a high-confidence but incorrect explanation. Therefore, they cannot definitively establish causality (the direct cause-and-effect relationship) between specific features and human trust. We see correlations between "evidence grounding" and trust. However, we do not know if grounding causes the trust. Users might simply gravitate toward well-formatted text. Finally, the study captures a snapshot of interaction. It does not track how trust evolves over weeks or months of continuous use.

Should You Prototype This

Yes, but focus on the interface, not just the model. If you are building agentic workflows where humans supervise autonomous actions, do not settle for a binary "on/off" switch. The paper's findings suggest you should implement granular, context-dependent toggles. Allow users to mute agents on specific topics or difficulty levels [DP 1].

Crucially, if you are designing "explanation" modules, stop optimizing for "sounding smart." The data shows that humans are easily fooled by surface-level similarity and quotes. Instead, prioritize "evidence grounding." This means forcing the model to cite specific clues from the input. This is the only feature that successfully bridges the gap between what predicts correctness and what drives human trust .

Code and datasets are available at: - https://github.com/qanta-org/qb-tournament-runner - https://huggingface.co/datasets/qanta-challenge/qanta25-gamedata

Figures from the paper