Quantifying Faithful Confidence Expression in Large Reasoning Models

When AI models "think" through complex problems, they often sound very sure of themselves even when they are actually uncertain. This study shows that current reasoning models have a hard time making their words match their true internal certainty.

This phenomenon is known as faithful calibration (FC). It is the alignment between a model's intrinsic confidence (its actual computational certainty) and its linguistic decisiveness (how certain it sounds to a human). As Large Reasoning Models (LRMs) become more prevalent, users interpret their long reasoning traces as evidence of competence. However, sounding smart is not the same as being right. Nor is it the same as knowing when you are wrong.

The Problem

The status quo for measuring confidence in LLMs fails when applied to the long, messy reasoning traces of LRMs. Most existing frameworks rely on measuring the consistency of sampled responses to estimate internal uncertainty. This breaks down in reasoning models. Their outputs lack clear step boundaries. They involve inconsistent structures across different runs. They also contain complex conditional dependencies that evolve throughout the trace.

Furthermore, there is a dangerous decoupling between accuracy and faithfulness. As shown in, LRMs stay highly decisive even on difficult tasks where accuracy is low.

This creates a massive reliability gap. A model can provide an authoritative-sounding explanation for a completely incorrect conclusion. The authors argue that reasoning behaviors do not automatically solve this. In fact, moving from standard instruction-tuned models to reasoning-tuned models can actually degrade the alignment between what a model thinks and what it says.

How It Works

To tackle this, the authors introduce a framework to quantify FC. They compare linguistic decisiveness against three distinct sources of internal uncertainty. The core methodology is a step-level comparison. For every reasoning step $s_i$, the model calculates the gap between its expressed certainty $D(s_i)$ and its intrinsic confidence $C(s_i)$.

The framework utilizes three complementary confidence estimators:

1. Recurrent Confidence Chain (RCC): This is a representation-based probe. It uses the model's final-layer hidden states (the internal vectors representing data) and an inter-step relevance filter. This filter propagates confidence through the reasoning chain. It recognizes that uncertainty in an early step should logically impact later steps.
2. DeepConf: This is a token-level log-probability estimator. It looks at the "peakedness" of the next-token distribution. If the model assigns very high probability to a small set of candidates, it is considered more certain.
3. Prefix-conditioned Sampling: This is a black-box approach designed for long traces. Instead of resampling the whole response, the authors condition on the original prompt and all preceding steps. They then sample $k=10$ continuations for a specific step. This measures how stable a local reasoning step is without the structural noise of the entire trace.

To aggregate these results, the authors propose cMFG (a width-weighted conditional mean faithfulness gap). This metric solves a common problem in calibration. Often, a model's confidence is clustered in a narrow range. Standard averaging methods produce biased or unstable results in these cases. cMFG uses equal-mass binning and width-weighting. This ensures the faithfulness score is integrated uniformly across the model's actual operating range.

Numbers

The authors report that current LRMs struggle significantly with this alignment. They achieve cMFG* scores typically between 0.64 and 0.78 across various datasets and models [Table 1]. This score represents how well the model's words track its internal state.

A critical finding for engineers is that different estimators produce divergent assessments. For instance, DeepConf (token-level) yields the highest alignment with linguistic decisiveness. Conversely, Sampling Consistency (black-box) consistently yields the lowest [Figure 2(b)]. This means your choice of metric will fundamentally change your assessment of model reliability.

Regarding the cost of evaluation, the sampling-based estimator is the most expensive. It requires $k=10$ continuations per evaluated step. To keep this tractable for long traces, the authors cap the evaluation at 20 steps per trace. They verify this choice is robust in .

Crucially, the study shows that prompt-based interventions largely fail. Strategies like telling a model to "be mindful of your uncertainty" do not improve faithfulness in reasoning models. While these prompts can sometimes improve task accuracy, they do not reliably repair the relationship between internal confidence and expressed decisiveness .

Areas for Future Research

The paper identifies several directions for further investigation:

• Impact of Post-Training: The study shows that distillation and reasoning training reshape uncertainty expression in complex ways. More research is needed to understand how these processes specifically modulate the gap between internal and expressed confidence.
• Estimator Divergence: Because different estimators (representation, log-prob, and sampling) yield divergent results, the field lacks a single "ground truth" for internal belief. Determining why these signals diverge remains an open challenge.
• Scaling and Architecture: While the study touches on model scale, the relationship between architectural complexity and the ability to express faithful uncertainty warrants deeper exploration.

Should You Prototype This

Depends on your safety requirements.

If you are building a chatbot for creative writing, ignore this. But if you are deploying LRMs in high-stakes environments—medicine, law, or automated scientific research—you should prototype a monitoring layer based on this framework.

Do not rely on the model's own verbalized confidence as a proxy for reliability. Instead, implement a sidecar process. Use the DeepConf (log-prob) or RCC (hidden state) methods to track the delta between what the model says and its internal distributions.

Code is reportedly available at https://github.com/yale-nlp/faithful_lrm. If you have the compute to run a few dozen H100 hours for a validation run, the DeepConf approach is the easiest to integrate into an existing vLLM-based stack.

Figures from the paper