When scaling LLM reasoning via Test-Time Scaling (TTS), we typically launch multiple parallel reasoning branches to explore the solution space. The goal is to find the correct answer by increasing the "width" of our search. However, as these branches run, they often fall into a trap of massive redundancy. Multiple paths independently rediscover the same mathematical constraints or logical shortcuts. This wastes precious compute on work that has already been done elsewhere.
The authors of "Share More, Search Less" argue this is an "information-isolation bottleneck." Current parallel TTS methods keep discoveries trapped within individual trajectories. This paper proposes Collaborative Parallel Thinking (CPT). It is a framework that allows branches to share compact, deduplicated insights through a central pool. This lets them build on each other's work in real-time.
The Problem
The status quo in parallel TTS treats every reasoning branch as a silo. While generating $K$ parallel samples accelerates exploration, intermediate discoveries remain isolated. Branch A's findings stay invisible to Branch B. This creates a significant efficiency leak.
The authors demonstrate this inefficiency through information statistics. In independent parallel sampling, the number of "duplicate" information units increases as search proceeds .
These are facts or logic steps already identified by other branches. This means scaling your rollout budget does not always yield new ground. You may just pay to have multiple models arrive at the same intermediate conclusion. This redundancy slows the accumulation of "complete decision information" (the total set of facts needed to reach a correct answer). It forces the system to work harder for diminishing marginal gains.
How It Works
CPT moves from isolated exploration toward a synchronized, collaborative search. The architecture relies on three coordinated components.
- Collaborative Parallel Search: CPT synchronizes branches at fixed-token intervals (e.g., every 2048 tokens). Each branch maintains its own private reasoning history. This ensures the unique trajectory of a single path is preserved. However, each branch also receives a "shared information block" at the start of each new chunk.
- Shared Information Pooling: Raw reasoning traces are too noisy and long to share directly. CPT uses the policy model to act as an extractor. It distills unstructured traces into compact "information units." These are categorized as insights (useful facts or relations) or pitfalls (warnings about potential errors). These units enter a query-level pool. They undergo semantic de-duplication (removing near-identical entries) using an embedding-similarity threshold ($\tau_{dup}$) .
- Adaptive Broadcast Scheduling: To prevent "premature convergence" (where branches lose diversity by focusing too much on shared info), CPT uses three modes. These are PROBE (independent exploration), BROADCAST (sharing enabled), and FREERUN (synchronization stops). Transitions depend on the relative rate of new information added to the pool. When marginal gain drops, the system triggers broadcasting. When the benefit hits a floor, it stops to save compute.
Numbers
The primary value proposition of CPT is shifting the accuracy-latency Pareto frontier (the optimal trade-off curve between precision and speed). The authors report that CPT establishes a stronger frontier than baselines like DeepConf and LeaP .
This means CPT reaches higher accuracy levels within the same wall-clock latency (actual time elapsed). On the HMMT25 benchmark using a Qwen3-4B model, the authors report a Pass@1 (the probability of a single sample being correct) of 66.79%. This was achieved with an optimal broadcast size of $M=512$ [Table 4].
The cost breakdown is revealing. Parallel sampling remains the dominant driver of latency. Information extraction and de-duplication add moderate overhead [Table 3]. Notably, while CPT is token-efficient, it may increase FLOPs (total floating-point operations) in standard decoders. This happens because broadcasting requires re-prefilling (re-processing the entire prompt) the updated context .
What's Missing
The paper is technically rigorous, but a few points remain unaddressed:
- KV-Cache Inefficiency: Broadcasting via input-context updates requires re-prefilling in standard decoders. This is a major cost. Without cache-aware or attention-level sharing (updating only specific parts of the Key-Value cache), the FLOPs-per-question might be higher than naive sampling.
- Extraction Reliability: The system hinges on the "extractor" model. It must correctly identify useful versus noisy information. If the model extracts hallucinations, it will pollute the shared pool. The paper does not deeply investigate failure modes for a "poisoned" pool.
- Scaling to Massive $K$: Most experiments use $K=64$. It is unclear how semantic de-duplication scales when running hundreds or thousands of parallel branches.
Should You Prototype This
Yes, but with a caveat.
If your primary constraint is wall-clock latency, CPT is a clear win. It optimizes the "intelligence per second" of your inference cluster. This is ideal if you need the fastest possible answer for complex reasoning.
However, if you operate in a highly constrained FLOPs environment, be careful. The re-prefilling overhead might negate your gains. I recommend prototyping this to test "extraction quality" on your specific domain. If your task involves niche logic where the model struggles to distill "insights," the shared pool might become a liability. Code is reportedly available at https://github.com/WangXinglin/CPT.