OCC-RAG: Optimal Cognitive Core for Faithful Question Answering

OCC-RAG: Small Language Models Outperform Giants in Faithful Contextual Reasoning

In Retrieval-Augmented Generation (RAG), we usually assume more parameters equal better grounding. We throw massive models at the problem. We hope their vast intelligence helps them navigate retrieved context. However, many practical applications benefit more from robust reasoning than from extensive parametric knowledge (information stored in the model's weights). Researchers have found that task-specialized small language models (SLMs) can outperform general-purpose giants. They do this by focusing on logic rather than memorization.

The Problem

Standard Context Question Answering (Context QA) suffers from a fundamental tension. The model's internal knowledge often fights against the provided context. When a retrieved document contains information that contradicts pretrained knowledge—such as a counterfactual (a claim that contradicts reality) historical claim—large models frequently default to their internal memory. They violate the instruction to use only the provided context.

As shown in, a 1B model might hallucinate an unsupported answer. An 8B model might provide a factually correct answer that fails the prompt's grounding instructions. This creates a reliability crisis for production RAG systems. If the model prioritizes its weights over the context, the retrieval pipeline becomes useless. Most models also struggle with "safe abstention" (declining to answer when evidence is missing). This leads them to hallucinate instead of admitting ignorance.

How It Works

The authors propose the Optimal Cognitive Core (OCC) approach. They treat reasoning as a specialized skill for mid-training (an intermediate training stage between pretraining and fine-tuning). They take existing base models, specifically Qwen3, and perform intensive mid-training. They use a massive synthetic corpus of 3.25 million QA pairs.

The methodology relies on three technical pillars:

1. KG-Conditioned Data Generation: To avoid poor quality synthetic data, the authors use a Knowledge Graph (KG) extraction pipeline. They extract structured factual graphs from Wikipedia. They then sample specific reasoning paths. This guarantees every multi-hop question has a verifiable "bridge" of logic. This ensures the model learns to traverse connections between facts. It prevents reliance on superficial patterns.
2. Structured Reasoning Traces: The authors force the model to generate a highly structured intermediate trace. As seen in, the model executes four distinct phases. First is Query Analysis (identifying entities). Second is Source Analysis (evaluating snippet relevance). Third is Reasoning (combining facts). Finally, it performs a Status check (ANSWERABLE vs. UNANSWERABLE) before the final answer. This scaffold forces the model to commit to a decision boundary regarding evidence sufficiency.
3. Calibrated Abstention: The training set includes "hard" refusal cases. The authors use a DeBERTa model to identify when a reduced context is insufficient. They create examples where the model must trigger the "UNANSWERABLE" status.

The training process uses a mixture of single-hop and multi-hop examples. The authors oversample the harder multi-hop subsets. This ensures the reasoning capability is properly internalized.

Numbers

Specialization beats scale for this specific task. The authors report that OCC-RAG-0.6B and 1.7B models match or exceed general-purpose models 2–6× their size.

On the ConFiQA faithfulness benchmark, the OCC-RAG-1.7B model achieves 81.4% accuracy. This significantly outperforms the Qwen3-1.7B baseline at 64.8%. The authors also measure the "Memorization Ratio" (MR). This measures how often a model ignores context to rely on internal weights. Mid-training reduced this ratio from 12.7 in Qwen3-1.7B down to 5.0 in OCC-RAG-1.7B. For a practitioner, this means the model is much more likely to follow your provided documents.

The training footprint is lean. The authors report that OCC-RAG-0.6B was trained on $9 \times 10^9$ tokens. This took 17 wall-clock hours using 8 NVIDIA H100 GPUs. The 1.7B variant took 28 hours on the same hardware. As shown in, the token budget is heavily weighted toward single-hop examples.

However, the multi-hop components drive the reasoning delta.

What's Missing

Several gaps exist that a production engineer should consider:

• Distribution Shift: The entire training corpus is synthetic. There is no guarantee that real-world user queries will map perfectly to these clean trajectories. Real data is often messy or poorly formatted.
• Inference Overhead: The model generates long, structured reasoning traces. This provides excellent interpretability. However, it increases the number of tokens generated per request. In high-throughput environments, the cost of generating these extra sections might offset the savings from using a smaller model.
• Trace Complexity: The paper does not deeply explore how varying the complexity of the required reasoning trace affects generalization to simpler tasks.

Should You Prototype This

Yes, if your primary bottleneck is RAG faithfulness. Use this if you operate in high-stakes environments. In these settings, "I don't know" is better than a hallucination. The ability of a sub-2B model to rival 8B+ models in refusal accuracy is a major win. It offers lower latency and lower costs.

Do not expect a drop-in replacement for a general-purpose chatbot. These models are highly specialized tools for grounded QA. If you deploy, benchmark the increased token cost of the reasoning traces. Compare this against your current latency requirements. Code and models are reportedly available at github.com/optimal-cognitive-core and huggingface.co/occ-ai.

Figures from the paper