Test-Time Cognition Makes
Post-Training/RL Data Collection Faster, Better, Cheaper

The bottleneck for frontier model improvement = high-quality data

Frontier model improvement depends on a scarce resource: high-quality data, especially attempts on challenging, near-frontier tasks that end up succeeding. This was true for the pre-training era, when the internet supplied a vast store of human-generated data that enabled ChatGPT and the AI boom that followed. It is still true today, but in a new form: in a process called post-training, models generate their own data by attempting hard tasks and learning from successful attempts.

The ability to produce successful attempts has become a key metric for every company with a stake in frontier model development: neolabs, RL environment companies, data labs collecting and grading trajectories, and frontier labs running reinforcement learning at scale.

High-quality data, especially successful trajectories on hard tasks, therefore defines the central bottleneck determining how quickly models can improve.

Test-Time Cognition (TTC), a cognitive architecture we are pioneering, adds active reasoning, structured search and problem-solving, and dynamic compute allocation to modern LLMs. We find that TTC exhibits characteristics that make it far more efficient source of high-quality data, in three ways:

TTC is model- and domain-agnostic, and we find that its gains hold across a wide range of settings and benchmarks, across a large number of models and trajectories:

The 30,000-foot view: where value accrues in intelligence

Pre-training scales along three coupled inputs: compute, dataset size and model size. Performance improves as each grows only until one of the other two becomes the bottleneck. More compute cannot compensate indefinitely for too little data or an undersized model, just as more data cannot compensate for limited compute.

Of the three, dataset size is the hardest constraint. The internet supplied a vast corpus of human-written text whose quality could largely be taken for granted, but that corpus is finite. Villalobos et al. estimate that the stock of high-quality language data could be exhausted this decade. Compute can be built and models enlarged; new human-written training data cannot be produced at comparable speed.

Post-training has a different bottleneck: searching for successful trajectories. These trajectories are its high-quality data. Where pre-training learned from text written by humans, post-training increasingly generates data from the models themselves: models attempt tasks, their outcomes are scored, and successful trajectories become the learning signal. The scarce resource is not the number of attempts, but successful attempts on tasks the model has not yet mastered.

Producing that data already consumes a huge share of the training budget. Across three frontier models, roughly half of allocated training compute goes to RL post-training, and that share is growing.

Most of that post-training compute is spent producing trajectories, not updating model weights. In Microsoft MAI-Thinking-1's largest RL job, 4,096 GPUs run inference to search for successful trajectories while 768 run the learner: 5.3 GPUs searching for every one learning.

As the bottleneck shifts from collecting internet-scale text to generating successful trajectories, value shifts with it. It accrues to whoever can produce high-quality post-training data reliably—the business at the center of the emerging RL industry.

The scale and breadth of the RL industry

Frontier tasks and beyond: the most impactful tasks

A benchmark is a collection of tasks, and a model does not simply pass or fail a task. Given the same task many times, a model succeeds on some attempts and fails on others. The share of attempts it gets right is its per-task success rate, anywhere from 0% to 100%.

Not every task is equally valuable for improving a model. A successful attempt on an easy task mostly confirms what the model already does well; a successful attempt on a task it rarely solves teaches it something new. The value of a task therefore follows a hierarchy set by its difficulty: beyond frontier > hard > medium > easy. Figure 4 places all 253 tasks in our four benchmarks in this hierarchy, by their success rate under standard inference, pooled across 22 model runs.

Under standard inference, 119 of the 253 tasks (47%) sit at the frontier: 72 medium and 47 hard. Another 59 tasks (23%) are beyond the frontier, and 35 of those are never solved in any attempt. The tasks at the top of this hierarchy are where post-training data is most valuable and, as the rest of this post shows, also where it is most expensive to collect.

Test-Time Cognition improves success rate on frontier tasks

Standard inference is very inefficient on frontier and beyond-frontier tasks. Post-training distills search into a model's weights, so standard inference follows the paths the model has learned to favor. On frontier tasks, and on tasks beyond the frontier, sampling the model more times spends more compute on failures it has already found (Figure 5, left).

The key to improving success rate on these tasks is search. The model can often solve them, but the solution lies off its most likely paths. Finding it means exploring at test time: spreading attempts across distinct, uncorrelated regions of the space of possible trajectories rather than retracing the same few (Figure 5, right).

Test-Time Cognition (TTC) is our initial implementation of search in LLMs. TTC adds active reasoning, structured search and problem-solving, and dynamic compute allocation, while keeping model weights and agent harnesses fixed. Searching efficiently in LLMs, exploring far enough beyond the model's default behavior without wasting compute on unproductive branches, is still an open problem, and TTC is our first step toward solving it. We collect data with TTC across four benchmarks and 25 model comparisons, and we find that TTC substantially improves performance on frontier tasks.

On a task-by-task basis, across all 4 benchmarks, we observe that TTC improves performance for the vast majority of medium and hard tasks, increasing the success rate by 1.61× for hard tasks and by 3.06× for tasks beyond the frontier, from 2.2% to 6.7%.

We observe that across many matched baseline-TTC comparisons, and across all 4 benchmarks, TTC lifts the average success rate on hard/beyond-frontier tasks by 1.44× to 2.31×, taking hard tasks and tasks beyond the frontier together (those the baseline solves less than 35% of the time).

This translates into three gains: Coverage (tasks solved that standard inference never solves), Speed (more successful trajectories collected per hour of agent time), and Cost (a lower cost for each successful trajectory).

Test-Time Cognition expands the frontier of solvable tasks

TTC helps models “unlock” tasks that the baseline never solves, or solves only very rarely, even when given a high attempt budget.

These tasks provide the largest impact on model capabilities, since the first successful trajectory is what will kickstart the learning signal for the model during RL. This materially expands the frontier of frontier LLM capabilities.

We observe that TTC improves coverage across all 4 benchmarks. For most benchmarks, TTC is able to find at least one successful solution for ~8%-12% more tasks in that benchmark.

This also shows that models are actually capable of solving these hard tasks, but standard inference prevents them from reaching these hard-to-find solutions. This leads to concrete problems for frontier data collection, post-training, and RL: it limits the amount of useful training signal that these models receive.

More successful trajectories per hour of agent time

The second gain is speed. Since the speed of collecting successful trajectories grows by 1/(success rate), hard tasks shoulder a disproportionate amount of total data collection time. This is further magnified by models naturally taking longer to solve hard tasks.

These factors result in stark consequences: Across all 253 tasks of the four benchmarks, hard tasks and tasks beyond the frontier account for only 42% of all tasks, but take up 88% of total agent execution time to collect a comparable amount of successful attempts per task: 24% for the hard tasks and 64% for those beyond the frontier.

Any meaningful speedup has to come from the hard/beyond-frontier tasks, and that is where TTC delivers. Per hour of agent time, TTC collects 1.2× to 3.8× as many successful trajectories on hard/beyond-frontier tasks as standard inference on average, and 2.0× to 12× as many in the best comparisons (Figure 9). This means that a week-long data collection campaign can finish in less than 1 day in the best case.

Lower cost per successful trajectory on hard/beyond-frontier tasks

The third gain is cost. A collection budget concentrates on hard/beyond-frontier tasks just as agent time does, since every additional failed attempt costs money as well as time.

Tasks beyond the frontier take 54% of the budget, and hard tasks another 31%. We find that TTC also reduces the cost-to-success significantly: setting a data collection budget of $1M for standard inference on hard/beyond-frontier tasks, TTC lowers the average cost by 39% to 64% on three of the four benchmarks, and by up to 58% to 88% in the best model comparison on every benchmark.

Together with the coverage gain, this means TTC reaches more of the hard/beyond-frontier tasks and collects their trajectories for less.

TTC: The next frontier of model capability

Across 25 models, 4 benchmarks spanning diverse domains from coding to customer research, and more than 10,000 trajectories, Test-Time Cognition (TTC) consistently improves three key factors for frontier data collection, post-training, and reinforcement learning: coverage, speed, and cost for hard/beyond-frontier tasks.

This highlights both the limitations of standard LLM inference, as well as how farther we can still push these models with the right cognitive architecture. We believe that TTC will play a key role in the next stage of post-training and defining the next frontier of model capabilities, ultimately ushering in a new cognition era beyond pre-training and post-training.

Breakdown by model comparison

Each of our 25 model comparisons pairs a baseline model run with a TTC system that uses that model, with a trajectory counting as a success only when it achieves full credit. When a TTC system composes multiple models, we compare it separately against each constituent model run alone, and define the hard/beyond-frontier regime (<35% success) relative to that comparison's baseline so every comparison is weighted equally. A benchmark's best comparison, the best case in Figures 9, 11 and 19, is the one with the largest saving, except that a composed system is eligible only in its comparison against its strongest constituent, the one with the highest baseline average success rate. Figure 12 shows the resulting cost to collect 100,000 successful trajectories on hard/beyond-frontier tasks for every individual comparison on a logarithmic scale.

While comparisons with very few baseline successes on hard/beyond-frontier tasks naturally exhibit more variance, TTC reduces collection costs in the vast majority of pairings, often by well over half.

Finally, we measure coverage directly. In each comparison, we take the tasks the baseline never solved in any of its attempts and count how many TTC solved at least once. Figure 13 shows this share of tasks rescued for every comparison.