Voaige

Higher Intelligence at lower cost.Same model, same agent.

Our research delivers gains while keeping the model weights and agent harness fixed.

  • Customer service SOTA Performance
    With TTC55.3%
  • Financial research +13 pp Performance gain
    0.4×the cost
  • Coding +13 pp Performance gain
    With TTC66.5%
  • IT operations +10 pp Performance gain
    With TTC63.8%

Benchmark results

Explore the results

4 distinct use cases, 7 agents evaluated, over 307B tokens.

Customer Service

Test Time Cognition achieves SOTA performance.

55.3% SOTAPerformance
Customer service: Test Time Cognition achieves 55.3% performance. Voaige baselines: Grok, Qwen, GPT-5.5. External reference: Opus 5 from Sierra. Logarithmic cost axis. Arrows point from baseline configurations to TTC. Qwen 3.8 Max (xHigh reasoning): 52.41% performance, $0.7594; Grok 4.6 (High reasoning): 45.83% performance, $0.3949; Test Time Cognition — Qwen 3.8 Max (xHigh reasoning) + Grok 4.6 (High reasoning): 55.33% performance, $1.8053; GPT-5.5 (xHigh reasoning): 39.18% performance, $1.8166; Opus 5 (Max reasoning): 48.71% performance, $13.3215 Performance (%) ↑ 40 50 60 $0.25 $1 $5 $20 Qwen 3.8 Max (xHigh reasoning) Performance: 52.41% Mean cost per task: $0.7594 Qwen 3.8 Grok 4.6 (High reasoning) Performance: 45.83% Mean cost per task: $0.3949 Grok 4.6 Test Time Cognition — Qwen 3.8 Max (xHigh reasoning) + Grok 4.6 (High reasoning) Performance: 55.33% Mean cost per task: $1.8053 TTC GPT-5.5 (xHigh reasoning) Performance: 39.18% Mean cost per task: $1.8166 GPT-5.5 Opus 5 (Max reasoning) Performance: 48.71% Mean cost per task: $13.3215 Opus 5 Mean cost per task (USD, log scale)
  • Baseline · Opus 5 from Sierra
  • TTC: Qwen + Grok

TTC = Test Time Cognition

τ-knowledge

Coding

Test Time Cognition achieves higher performance on software engineering tasks.

+12.5 ppPerformance
Coding: DeepSeek V4 Flash and Sonnet 5.5 with Test Time Cognition on DeepSWE. Arrows point from baseline configurations to TTC. DeepSeek V4 Flash: 41.00% performance, $0.3516; Sonnet 5.5 (Medium reasoning): 53.98% performance, $0.6788; Test Time Cognition — DeepSeek V4 Flash + Sonnet 5.5 (Medium reasoning): 66.52% performance, $1.3007; Test Time Cognition — Sonnet 5.5 (Medium reasoning): 65.34% performance, $1.0509 Performance (%) ↑ 30 40 50 60 70 $0 $0.5 $1 $1.5 DeepSeek V4 Flash Agent: mini-SWE-agent Performance: 41.00% Mean cost per task: $0.3516 DeepSeek V4 Flash Sonnet 5.5 (Medium reasoning) Agent: mini-SWE-agent Performance: 53.98% Mean cost per task: $0.6788 Sonnet 5.5 Test Time Cognition — DeepSeek V4 Flash + Sonnet 5.5 (Medium reasoning) Agent: mini-SWE-agent Performance: 66.52% Mean cost per task: $1.3007 TTC Test Time Cognition — Sonnet 5.5 (Medium reasoning) Agent: mini-SWE-agent Performance: 65.34% Mean cost per task: $1.0509 TTC Mean cost per task (USD)
  • Baseline: DeepSeek V4 Flash · Sonnet 5.5
  • Same models + TTC

TTC = Test Time Cognition

DeepSWE · mini-SWE-agent

Financial Research

Test Time Cognition achieves higher performance at lower cost.

+13.0 ppPerformance
58.6%Avg. cost reduction
Financial research: Kimi and DeepSeek with Test Time Cognition. Logarithmic cost axis. Arrows point from baseline configurations to TTC. Kimi K3 (Max reasoning): 64.20% performance, $0.9930; Test Time Cognition — Kimi K3 (Max reasoning): 72.84% performance, $0.3169; DeepSeek V4 Flash Thinking: 53.01% performance, $0.0197; Test Time Cognition — DeepSeek V4 Flash Thinking: 70.37% performance, $0.0100 Performance (%) ↑ 50 60 70 80 $0.01 $0.1 $1 Kimi K3 (Max reasoning) Performance: 64.20% Mean cost per task: $0.9930 Kimi K3 Test Time Cognition — Kimi K3 (Max reasoning) Performance: 72.84% Mean cost per task: $0.3169 TTC DeepSeek V4 Flash Thinking Performance: 53.01% Mean cost per task: $0.0197 DeepSeek V4 Flash Test Time Cognition — DeepSeek V4 Flash Thinking Performance: 70.37% Mean cost per task: $0.0100 TTC Mean cost per task (USD, log scale)
  • Baseline: Kimi K3 · DeepSeek V4 Flash
  • Same models + TTC

TTC = Test Time Cognition

Finance Agent V2

IT Operations

Test Time Cognition achieves higher performance on IT operations tasks.

+10.0 ppPerformance
IT operations: Sonnet 5.5 and GPT-6 Luna, and DeepSeek V4 Flash, with Test Time Cognition. Logarithmic cost axis. Arrows point from baseline configurations to TTC. Sonnet 5.5 (High reasoning): 51.67% performance, $0.3065; GPT-6 Luna (Max reasoning): 53.75% performance, $0.1645; Test Time Cognition — Sonnet 5.5 (High reasoning) + GPT-6 Luna (Max reasoning): 63.75% performance, $1.9164; DeepSeek V4 Flash: 44.77% performance, $0.1233; Test Time Cognition — DeepSeek V4 Flash: 51.17% performance, $0.0572 Performance (%) ↑ 40 50 60 70 $0.05 $0.1 $0.5 $1 $2 Sonnet 5.5 (High reasoning) Agent: Stirrup 0.2.0 Performance: 51.67% Mean cost per task: $0.3065 Sonnet 5.5 GPT-6 Luna (Max reasoning) Agent: Stirrup 0.2.0 Performance: 53.75% Mean cost per task: $0.1645 GPT-6 Luna Test Time Cognition — Sonnet 5.5 (High reasoning) + GPT-6 Luna (Max reasoning) Agent: Stirrup 0.2.0 Performance: 63.75% Mean cost per task: $1.9164 TTC DeepSeek V4 Flash Agent: Stirrup 0.2.0 Performance: 44.77% Mean cost per task: $0.1233 DeepSeek V4 Flash Test Time Cognition — DeepSeek V4 Flash Agent: Stirrup 0.2.0 Performance: 51.17% Mean cost per task: $0.0572 TTC Mean cost per task (USD, log scale)
  • Baseline: Sonnet 5.5 · GPT-6 Luna · DeepSeek V4 Flash
  • Same models + TTC

TTC = Test Time Cognition

ITBench-AA

Key insights

Insights from our evaluation program

Insight 01

Test Time Cognition is a new scaling axis

For years, AI has scaled along two axes: more training data and bigger models.

Test Time Cognition adds a third scaling axis: the compute an agent spends at test time. On Terminal-Bench 2.0, more compute translates directly into higher performance: Test Time Cognition outperforms GPT‑5 at every reasoning effort, while costing less than GPT‑5 at medium or high effort.

  • Models on their own
  • Test Time Cognition (TTC)

Terminal-Bench 2.0 (89 tasks), mini-SWE-agent 2.2.1. TTC: GPT‑5 + MiniMax M2.5.

Insight 02

Test Time Cognition solves problems in 40% fewer steps

Smarter decisions at each step result in a faster path to the solution. Aggregated over 40 configurations and nearly 10,000 trajectories on Terminal-Bench 2.0, agents with Test Time Cognition take 40% fewer steps per attempt.

−40%average steps per attempt
  • Baseline
  • Test Time Cognition
  • Easiest to hardest →︎

Terminal-Bench 2.0 (89 problems), mini-SWE-agent 2.2.4.

Insight 03

Test Time Cognition gains most on medium-hard problems

Test Time Cognition delivers its largest gains on medium-hard problems. Aggregated over the same 40 configurations and nearly 10,000 trajectories, it lifts performance on medium problems by 15.5 points.

+15.5ppon medium problems
  • Baseline
  • Test Time Cognition
  • Easiest to hardest →︎

Terminal-Bench 2.0 (89 problems), mini-SWE-agent 2.2.4.

Benchmark details

Deep dive into the results

Each benchmark in full: every configuration we ran, what it cost, and how it performed problem by problem.

Appendix

How we evaluate

We evaluate autonomous agents with room to work through a task, using the same evaluation protocol to compare baseline performance with Test Time Cognition.

Room to complete the task

We use generous time and step limits so agents can pursue longer solutions. The goal is to measure task-solving performance with enough room for the agent to finish, rather than have tight execution limits determine the outcome.

Repeated trials

Our standard protocol uses three samples per problem and configuration. Repeated attempts help account for variation between runs.

Framework and sandboxes

We use the Harbor framework to run and evaluate tasks, with execution environments hosted in Daytona sandboxes.