Cognition: Scaling Compute
beyond Pre-Training and RL
The Bitter Lesson has two halves: learning and search. The pre-training era relied on learning and scale. The post-training era brings a naive form of search back into the picture. But the tools that got us here won’t get us to the next level of intelligence. We believe that the next frontier will arise from cognition, a research program that treats search as a first-class citizen of intelligent systems.
Two halves of the Bitter Lesson
In 2019, Richard Sutton published The Bitter Lesson, arguing that the most important advances in AI come from general methods that scale with computation, rather than methods that encode human knowledge.
He identified two such methods: learning and search.
For the seven years since, the vast majority of frontier AI compute has been devoted to learning. The internet gave us an enormous source of training data, and scaling laws provided an extremely predictable return per FLOP spent. These favorable conditions gave rise to the pre-training era.
Reasoning models and AI agents ushered in the post-training era, where search and reinforcement learning become important parts of shaping and aligning model behavior, in many cases rivaling the compute spent on pre-training. But the conditions that made pre-training successful are changing, and early evidence suggests that we are approaching the limits of what we can extract from today’s post-training methods.
We believe overcoming these limits requires making search a first-class citizen of intelligent systems. We call the process that does so cognition, and we predict that the next five years will be defined by a new cognition era. Just as scaling learning defined the last era of AI, we believe scaling cognition will define the next, pulling model capabilities beyond their current limits.
This is the other half of the Bitter Lesson that we believe the field has yet to fully embrace.
The pre-training era and the success of learning
The pre-training era was one of the most successful periods of empirical science in AI. The recipe was remarkably simple: gather more data, train larger networks, and spend more compute optimizing next-token prediction.
And it worked.
GPT-3 demonstrated that scaling a single autoregressive model could produce capabilities across a remarkable range of tasks. Kaplan et al. showed that language-model loss followed predictable power-law relationships with model size, dataset size, and compute. Chinchilla refined the recipe by showing how to allocate compute more efficiently between parameters and training tokens.
This success rested on three unusually favorable properties:
First, data was abundant. The internet had accumulated an enormous corpus of human-generated text, code, and other knowledge. We could train on it without first needing our models to discover new information.
Second, supervision was dense. Every training token came with an observed target. The optimizer did not need to solve the underlying problem to obtain a learning signal. Thanks to gradient descent, an incorrect prediction supplied a direction for reducing the loss.
Third, progress was predictable. Given a model architecture, dataset, and compute budget, researchers could often estimate how much the next training run would improve cross-entropy loss.
As a result, entire areas of research and engineering matured around this recipe: distributed training, data curation, numerical precision, optimizers, kernels, parallelism, and scaling laws.
The next phase of AI, if we ever hope that it will exceed human capabilities, does not enjoy these luxuries.
Reasoning models and the birth of post-training
Several developments have nudged us beyond the pre-training era. In September 2024, OpenAI introduced o1, showing that performance could improve along an additional axis: reasoning compute during inference. There is a clear lesson: letting models iteratively work towards or refine an answer through a long reasoning chain is a key lever to improve their intelligence.
Since then, post-training and RL have become central components of frontier-model development. Although figures from leading closed and open model providers vary wildly and can be hard to estimate, in some cases the amount of compute devoted to this process is beginning to rival pre-training itself.
Ai2’s OLMo 3 series is estimated to have spent 15% of all H100 GPU-hours on RL across the training pipeline (pre-training, mid-training, supervised fine-tuning, and reinforcement learning), with RL occupying a disproportionate share of wall-clock accelerator time relative to its arithmetic FLOPs. Ai2’s public recipe breaks out these stages in detail.
On the extreme end, Reflection’s Beam was pre-trained on 23.8 trillion tokens using 6,144 NVIDIA GB300 GPUs in under four weeks. Its subsequent high-compute RL campaign ran on 10,500 GB300 GPUs for four weeks and generated more than 100 million rollouts across coding, reasoning, and agentic environments.
This means that the RL campaign consumed at least 1.7 times as many allocated GPU-hours as pre-training, based on the published hardware counts and durations.
Similarly, xAI’s Grok 4 announcement claims RL training at pre-training scale, with a launch chart illustrating the growing contribution of RL compute relative to earlier model generations.
These examples do not establish a universal percentage split for all frontier model organizations, but the direction is difficult to miss: Increasing amounts of frontier AI infrastructure are being devoted not simply to updating model parameters, but to generating, evaluating, and learning from candidate solutions. The post-training era is born.
But a trillion FLOPs spent on pre-training and a trillion FLOPs spent generating RL trajectories do not produce the same amount of learning signal.
Compute has shifted from learning to search
In pre-training, a model predicts a token, it is compared against the token in the training corpus, and the model is nudged to predict the target if its prediction is incorrect. The learning signal is immediate and plentiful. Within certain known bounds, scaling laws virtually guarantee that a FLOP spent on this process will lead to a better model.
In contrast, RL involves the model producing thousands of intermediate reasoning tokens, or invoking dozens of tools. At the end, the verifier marks the attempt as correct or incorrect.
Here we must realize the fundamental difference between the two regimes: Pre-training starts with the target and learns how to reproduce it. Search starts without the target and must first spend compute discovering it.
Three factors make RL much more inefficient than it seems:
The right answer must be searched for. Before any model updates are possible, compute must first be spent to search for the answer. For hard problems, this search process is prohibitively expensive, and it is difficult to predict if FLOPs spent will result in finding the right answer.
An additional, commonly overlooked consequence is the cost induced by autoregressive generation: since LLMs generate answers token-by-token, not knowing the right reasoning chain ahead of time forces RL pipelines to use GPU hours in this much more expensive regime that pre-training does not suffer from.
Incorrect predictions provide no learning signal. In pre-training, an incorrect prediction supplies the exact feedback needed to align the model to the correct behavior. In RL, the model receives no information about what it should have done instead, or where in the process the mistake was made. Consequently, unlike pre-training, future forward passes (search attempts) do not get progressively better.
We should not mistake these observations for proof that RL cannot scale. Instead, they tell us that the assumptions underlying pre-training’s success no longer apply automatically.
Limits of scaling post-training
These challenges have already manifested in concrete consequences and limits on what post-training can and cannot achieve, even when we invest in it in a scale that rivals pre-training.
Bottlenecks in post-training scaling laws. Although post-training scaling laws are not as mature as pre-training laws, in The Art of Scaling Reinforcement Learning Compute, Khatri et al. study more than 400,000 GPU-hours of RL experiments and fit sigmoidal rather than pre-training-style power-law performance curves. In stark contrast to the power laws observed in pre-training, there is a ceiling to what current RL pipelines can extract from base models.
RL training increases one-shot accuracy at the expense of solution coverage. In Does Reinforcement Learning Really Incentivize Reasoning Capacity Beyond the Base Model?, Yue et al. compare the pass@k of base models and their RL-trained counterparts. At small k, RL-trained models often outperform their base models, but at large k, the advantage often reverses: the base model can solve more distinct problems when allowed enough attempts.
New capabilities require an external source. More recently, knowledge distillation transfers capabilities from a teacher into a student. It can introduce capabilities that the student previously lacked, but those capabilities must first exist in the teacher or its generated data. On-policy distillation has become a popular framework used in many frontier open-model releases, including recent Kimi K-series models.
These pieces of evidence hint at a shared limitation: an optimizer can only use the information available in its training experience.
Does this mean that reasoning, post-training, and RL were the wrong direction for LLMs? No. We instead take these as signs that the other half of the Bitter Lesson has not been fully embraced.
Cognition: Breaking through the limits of post-training and reasoning
Reasoning and post-training proved to be necessary steps for more intelligent LLMs. Together with the scaling of pre-training, models, and training datasets, they form a two-axis backbone of general-purpose AI.
Then how can we break through the limits of the pre- and post-training eras? We think we just need to embrace the other half of the Bitter Lesson: we must bring back search as a first-class citizen of intelligent systems.
Test-Time Cognition (TTC) is our first instantiation of this principle in LLMs, providing a third, independent scaling axis. Unlike the reasoning axis, which always trades off higher performance for additional cost, we show that this new cognition axis can outperform the model's native reasoning while reducing cost:
This allowed TTC to beat the GPT-5.4 performance-cost Pareto frontier on Terminal-Bench 2.0, using models released purely before it.
Cognition: Coordinating the components of search
How is this possible? This result comes from taking the Bitter Lesson seriously: we must bring back search as a first-class citizen of intelligent systems. AlphaGo's Move 37, described as "surely creative" by grandmaster Lee Sedol, was discovered through a deliberative search process that the expert policy found unremarkable.
Search depends on considering all the important or interesting possibilities, even those that the strongest models consider unlikely, and recognizing promising partial solutions and rejecting bad ones. TTC approximates that process in LLMs.
Learning and reasoning are still very useful tools, and we do not claim that we should abandon them. Instead, we view search and learning as two halves of the same coin. We call the process that coordinates between them, and utilizes their complementary strengths to the fullest extent, cognition.
This is not a proposal to abandon neural networks or gradient descent. It is a proposal to build a more capable process around them, one that leverages their strengths to achieve things the models could not previously do. Neural networks can serve as candidate generators, policies, value functions, and world models. Gradient descent can improve each component, while search determines which experiences to acquire and which possibilities deserve further computation.
Cognition is how the two halves of the Bitter Lesson can reinforce one another.
The Cognition Era: The next frontier in model capability
As we hit the limits of the pre-training and post-training eras, we believe the cognition era is the next stage: making search itself more intelligent, so that every unit of compute produces more useful information.
The key is to increase the effective density of rewards. By enumerating better candidates, estimating the value of partial solutions, and extracting information from failures, cognition can turn sparse, delayed feedback into richer signals that guide both search and learning. Instead of relying on a rare successful trajectory to learn something new, an intelligent system should improve from much more of the experience it generates. This is how we believe search can become a reliable engine for discovering capabilities beyond those of the underlying models.
Our prediction is that by 2031, most compute devoted to advancing frontier AI intelligence will be spent on cognition. Not simply on training larger models, nor on generating ever more independent reasoning attempts, but on systems that deliberately explore possibilities, evaluate progress, and accumulate knowledge from experience. The next frontier of intelligence will be driven by how effectively these systems convert compute into discovery.
Further reading: Intelligence Is Search: Cognition Coordinates It
Three components of search
Consider what AlphaGo achieved: Move 37, described as creative by Lee Sedol, was discovered through a deliberative search process rather than simply following an expert policy. Search allowed the system to discover valuable possibilities that its learned policy alone might have overlooked.
Voaige’s mission is to bring the same principles to LLMs, and to build the best research environment for this direction. We believe that making search a first-class research problem, rather than simply scaling today’s RL pipelines, will break us out of the limits of the post-training era, ushering in a new cognition era.
As an operational model of this hypothesis, we break the search problem into three components, show what progress we have made in each, and how much further there is to go.
Candidate enumeration
Rather than treating an LLM as a single end-to-end problem solver, we believe an intelligent system needs a dedicated component for enumerating important or interesting possibilities.
This includes candidates that even a strong individual model considers unlikely. The goal of enumeration is not to achieve a higher one-shot accuracy or pass rate, but to ensure that useful possibilities are not overlooked. These are distinct objectives and intelligent systems should be designed and optimized with this insight in mind.
Our first test-time cognition framework, DCA, demonstrates this principle in modern LLMs. In our coding agent evaluations, a relatively weak model, DeepSeek-v3.2, can nevertheless suggest possibilities that other models might overlook, producing a measurable improvement in the overall system’s performance.
This is what allowed DCA to beat the GPT-5.4 Pareto frontier on Terminal-Bench using models released purely before it.
Candidate selection and value functions
Enumerating candidates is only half the problem. Once we have them, we need to decide which are worth pursuing, often without knowing how a trajectory will end. An agent may spend thousands of actions exploring a problem, but executing every possible continuation to completion is prohibitively expensive. We need value functions that recognize promising partial solutions, decide when to branch or abandon an attempt, and allocate search compute where it is most useful.
Our DCA experiments show this directly: holding the candidate enumeration process fixed, changing the architecture or training data of the value function makes a huge impact on end-to-end performance.
By assigning these responsibilities to different models, DCA achieves improvements that neither component could produce alone. A good value function can save more search compute than a faster generator. See our DCA results on agentic coding tasks.
Learning from failure
The third problem may be the most important: What do we do with unsuccessful attempts? In today’s post-training pipelines, an agent can spend substantial compute generating a long trajectory, only to receive a failure signal at the end. Yet a failed trajectory may contain useful intermediate discoveries, partially correct solutions, or mistakes that can be identified and avoided. Discarding the entire attempt wastes potentially valuable information.
We believe a major opportunity lies in turning this discarded experience into useful information for future search. Rather than spending ever more compute hoping to find a rare successful trajectory, we should learn to identify what went right and wrong, preserve useful intermediate discoveries, and improve subsequent attempts. Our ongoing work on agent memory and episodic retrieval and robotics planning reflects this direction: building systems that retain and reuse what they discover, even when an individual attempt fails.