Voaige

Higher Intelligence at lower cost.Same model, same agent.

Our research delivers gains while keeping the model weights and agent harness fixed.

  • Customer service SOTA Performance
    With TTC55.3%
  • Financial research +13 pp Performance gain
    0.4×the cost
  • Coding +5 pp Performance gain
    0.6×the cost
  • IT operations +10 pp Performance gain
    With TTC63.8%

Benchmark results

Explore the results

4 distinct use cases, 7 agents evaluated, over 307B tokens.

Customer Service

Test Time Cognition achieves SOTA performance.

55.3% SOTAPerformance
Customer service: Test Time Cognition achieves 55.3% performance. Voaige baselines: Grok, Qwen, GPT-5.5. External reference: Opus 5 from Sierra. Logarithmic cost axis. Arrows point from baseline configurations to TTC. Qwen 3.8 Max (xHigh reasoning): 52.41% performance, $0.7594; Grok 4.6 (High reasoning): 45.83% performance, $0.3949; Test Time Cognition — Qwen 3.8 Max (xHigh reasoning) + Grok 4.6 (High reasoning): 55.33% performance, $1.8053; GPT-5.5 (xHigh reasoning): 39.18% performance, $1.8166; Opus 5 (Max reasoning): 48.71% performance, $13.3215 Performance (%) ↑ 40 50 60 $0.25 $1 $5 $20 Qwen 3.8 Max (xHigh reasoning) Performance: 52.41% Mean cost per task: $0.7594 Qwen 3.8 Grok 4.6 (High reasoning) Performance: 45.83% Mean cost per task: $0.3949 Grok 4.6 Test Time Cognition — Qwen 3.8 Max (xHigh reasoning) + Grok 4.6 (High reasoning) Performance: 55.33% Mean cost per task: $1.8053 TTC GPT-5.5 (xHigh reasoning) Performance: 39.18% Mean cost per task: $1.8166 GPT-5.5 Opus 5 (Max reasoning) Performance: 48.71% Mean cost per task: $13.3215 Opus 5 Mean cost per task (USD, log scale)
  • Baseline · Opus 5 from Sierra
  • TTC: Qwen + Grok

TTC = Test Time Cognition

τ-knowledge

Coding

Test Time Cognition achieves higher performance at lower cost.

+5.2 ppPerformance
38.5%Lower cost
Coding: GPT-5, MiniMax, and Test Time Cognition with verified mean costs across 89 tasks. Lines connect measured reasoning levels. GPT-5 (minimal reasoning): 26.97% performance, $0.1163; GPT-5 (low reasoning): 37.45% performance, $0.2815; GPT-5 (high reasoning): 51.69% performance, $0.7722; MiniMax M2.5 Thinking: 46.82% performance, $0.0881; Test Time Cognition — GPT-5 (low reasoning) + MiniMax M2.5 (Thinking): 54.31% performance, $0.3424; Test Time Cognition — GPT-5 (medium reasoning) + MiniMax M2.5 (Thinking): 56.93% performance, $0.4747 Performance (%) ↑ 20 30 40 50 60 70 $0 $0.25 $0.5 $0.75 $1 GPT-5 (minimal reasoning) Agent: mini-SWE-agent 2.2.1 Performance: 26.97% Mean cost per task: $0.1163 Minimal GPT-5 (low reasoning) Agent: mini-SWE-agent 2.2.1 Performance: 37.45% Mean cost per task: $0.2815 Low GPT-5 (high reasoning) Agent: mini-SWE-agent 2.2.1 Performance: 51.69% Mean cost per task: $0.7722 GPT-5 High MiniMax M2.5 Thinking Agent: mini-SWE-agent 2.2.1 Performance: 46.82% Mean cost per task: $0.0881 MiniMax Test Time Cognition — GPT-5 (low reasoning) + MiniMax M2.5 (Thinking) Agent: mini-SWE-agent 2.2.1 Performance: 54.31% Mean cost per task: $0.3424 TTC Test Time Cognition — GPT-5 (medium reasoning) + MiniMax M2.5 (Thinking) Agent: mini-SWE-agent 2.2.1 Performance: 56.93% Mean cost per task: $0.4747 TTC Mean cost per task (USD)
  • Baseline: GPT-5 · MiniMax
  • TTC: GPT-5 + MiniMax

TTC = Test Time Cognition

Terminal-Bench 2.0 · mini-SWE-agent 2.2.1

Financial Research

Test Time Cognition achieves higher performance at lower cost.

+13.0 ppPerformance
58.6%Avg. cost reduction
Financial research: Kimi and DeepSeek with Test Time Cognition. Logarithmic cost axis. Arrows point from baseline configurations to TTC. Kimi K3 (Max reasoning): 64.20% performance, $0.9930; Test Time Cognition — Kimi K3 (Max reasoning): 72.84% performance, $0.3169; DeepSeek V4 Flash Thinking: 53.01% performance, $0.0197; Test Time Cognition — DeepSeek V4 Flash Thinking: 70.37% performance, $0.0100 Performance (%) ↑ 50 60 70 80 $0.01 $0.1 $1 Kimi K3 (Max reasoning) Performance: 64.20% Mean cost per task: $0.9930 Kimi K3 Test Time Cognition — Kimi K3 (Max reasoning) Performance: 72.84% Mean cost per task: $0.3169 TTC DeepSeek V4 Flash Thinking Performance: 53.01% Mean cost per task: $0.0197 DeepSeek V4 Flash Test Time Cognition — DeepSeek V4 Flash Thinking Performance: 70.37% Mean cost per task: $0.0100 TTC Mean cost per task (USD, log scale)
  • Baseline: Kimi K3 · DeepSeek V4 Flash
  • Same models + TTC

TTC = Test Time Cognition

Finance Agent V2

IT Operations

Test Time Cognition achieves higher performance on IT operations tasks.

+10.0 ppPerformance
IT operations: Sonnet 5.5 and GPT-6 Luna, and DeepSeek V4 Flash, with Test Time Cognition. Logarithmic cost axis. Arrows point from baseline configurations to TTC. Sonnet 5.5 (High reasoning): 51.67% performance, $0.3065; GPT-6 Luna (Max reasoning): 53.75% performance, $0.1645; Test Time Cognition — Sonnet 5.5 (High reasoning) + GPT-6 Luna (Max reasoning): 63.75% performance, $1.9164; DeepSeek V4 Flash: 44.77% performance, $0.1233; Test Time Cognition — DeepSeek V4 Flash: 51.17% performance, $0.0572 Performance (%) ↑ 40 50 60 70 $0.05 $0.1 $0.5 $1 $2 Sonnet 5.5 (High reasoning) Agent: Stirrup 0.2.0 Performance: 51.67% Mean cost per task: $0.3065 Sonnet 5.5 GPT-6 Luna (Max reasoning) Agent: Stirrup 0.2.0 Performance: 53.75% Mean cost per task: $0.1645 GPT-6 Luna Test Time Cognition — Sonnet 5.5 (High reasoning) + GPT-6 Luna (Max reasoning) Agent: Stirrup 0.2.0 Performance: 63.75% Mean cost per task: $1.9164 TTC DeepSeek V4 Flash Agent: Stirrup 0.2.0 Performance: 44.77% Mean cost per task: $0.1233 DeepSeek V4 Flash Test Time Cognition — DeepSeek V4 Flash Agent: Stirrup 0.2.0 Performance: 51.17% Mean cost per task: $0.0572 TTC Mean cost per task (USD, log scale)
  • Baseline: Sonnet 5.5 · GPT-6 Luna · DeepSeek V4 Flash
  • Same models + TTC

TTC = Test Time Cognition

ITBench-AA

Key insights

Insights from our evaluation program

Insight 01

Test Time Cognition is a new scaling axis

For years, AI has scaled along two axes: more training data and bigger models.

Test Time Cognition adds a third scaling axis: the compute an agent spends at test time. On Terminal-Bench 2.0, more compute translates directly into higher performance: Test Time Cognition outperforms GPT‑5 at every reasoning effort, while costing less than GPT‑5 at medium or high effort.

  • Models on their own
  • Test Time Cognition (TTC)

Terminal-Bench 2.0 (89 tasks), mini-SWE-agent 2.2.1. TTC: GPT‑5 + MiniMax M2.5.

Insight 02

Test Time Cognition solves problems in 40% fewer steps

Smarter decisions at each step result in a faster path to the solution. Aggregated over 40 configurations and nearly 10,000 trajectories on Terminal-Bench 2.0, agents with Test Time Cognition take 40% fewer steps per attempt.

−40%average steps per attempt
  • Baseline
  • Test Time Cognition
  • Easiest to hardest →︎

Terminal-Bench 2.0 (89 problems), mini-SWE-agent 2.2.4.

Insight 03

Test Time Cognition gains most on medium-hard problems

Test Time Cognition delivers its largest gains on medium-hard problems. Aggregated over the same 40 configurations and nearly 10,000 trajectories, it lifts performance on medium problems by 15.5 points.

+15.5ppon medium problems
  • Baseline
  • Test Time Cognition
  • Easiest to hardest →︎

Terminal-Bench 2.0 (89 problems), mini-SWE-agent 2.2.4.

Benchmark details

Deep dive into the results

Each benchmark in full: every configuration we ran, what it cost, and how it performed problem by problem.

Appendix

How we evaluate

We evaluate autonomous agents with room to work through a task, using the same evaluation protocol to compare baseline performance with Test Time Cognition.

Room to complete the task

We use generous time and step limits so agents can pursue longer solutions. The goal is to measure task-solving performance with enough room for the agent to finish, rather than have tight execution limits determine the outcome.

Repeated trials

Our standard protocol uses three samples per problem and configuration. Repeated attempts help account for variation between runs.

Framework and sandboxes

We use the Harbor framework to run and evaluate tasks, with execution environments hosted in Daytona sandboxes.