Same model, same success rate, twice the money. What are you actually paying Claude Code for?

Start with the number that looks like a typo.

GPT-5.6 Sol on Terminal-Bench 2.0 tasks costs $1.355 per attempt in Claude Code and solves 71.1%. The same model in Pi, a minimal open-source harness, costs $0.421 and solves 83.3%. One model, the same tasks. Three times cheaper and twelve points better.

That comes from HarnessTax, published on 16 September by Berkeley's Sky Lab and the Arena team. The authors include Ion Stoica and Matei Zaharia, co-founders of Databricks and creators of Apache Spark, and Wei-Lin Chiang from LMArena. Their question is hard to argue with: could a different harness help the same model solve more tasks, or cost less?

A harness is the program sitting between you and the model: it decides which tools to declare, what goes into the system prompt, and when to stop. Claude Code, Codex CLI and Cursor are harnesses. Until now nobody measured them apart from the model, so "which is better" always collapsed into an argument about the model inside.

What was measured

Seven models across three harnesses: Claude Code 2.1.224, Codex CLI 0.146.0 and Pi 0.85.1, a minimal open harness with four tools (read, write, edit, bash). Thirty tasks each from SWE-bench Lite and Terminal-Bench 2.0, three runs per task, costs at a fixed direct-API price list. For cleanliness the task containers had no network access, so no harness could win by going online.

An aside on where our numbers come from. Opening the study page the ordinary way did not work: the dashboard renders on the client and the server hands over an empty loader. We wasted half an hour pulling figures from second-hand write-ups, where they were already drifting apart. Everything below is rebuilt from the dashboard's own data, and the publication date had to come from the commit history, because the text does not carry one.

Success stands still, cost moves

The flagship shows it most cleanly. Claude Fable 5 on SWE-bench Lite solves 97.8% of attempts in Claude Code and 96.7% in Pi. That gap sits inside the error bars: the confidence interval runs from 93.3% to 100%. The money differs by exactly two, $1.33 against $0.67.

And the agent makes the same number of moves, 15.3 turns against 15.4. Each one simply costs more.

The pattern repeats across all seven models. Here are the four most telling rows:

The Luna row reads like a typo: fifteen cents against three at an identical 55.6%. The Haiku row is more interesting, because there Claude Code is not only pricier but worse, 52.2% against 60.0% in Pi.

Averaged across models, Claude Code costs about twice what Pi does and 1.6 times what Codex does. The authors call that gap a harness tax, and they are precise about when you pay it: when you take the default harness without comparing it to anything.

Where the tax comes from

It starts before the agent does anything at all. The very first request already differs.

Twenty-three declared tools against four, and their schemas take twenty-seven times more room. The opening context is nearly fourteen times heavier, and that weight rides along with every request in the session.

From there the arithmetic is the one we walked through when we counted the weekly limits: long context is resent in full at every step. Then we were looking at the consequence, a limit hit sooner than expected. Here the cause is itemised.

And the surprising part: Pi, with its four tools, reaches the Pareto frontier on both benchmarks. Claude Code's other nineteen tools are not useless, they simply do not pay for themselves on "fix a bug in a repository that already has tests".

A model can do better outside its own harness

This is the strangest finding. Across six Anthropic and OpenAI models on two benchmarks, an alternative harness produced the best result in nine of twelve comparisons.

Sonnet 4.6 solves more in Codex than in Claude Code. Haiku 4.5 does better in Pi. GPT-5.6 Sol in Pi beats itself in its native Codex. Optimising a model for its own environment, which providers do describe doing, does not guarantee the best pairing.

Where this does not apply to you

We read it to the end, and we think the limitations matter more than the findings.

The tasks are short. The cap is 100 turns, the average around fifteen. This is a fix inside a known repository with tests already in place. A harness is not for that. It earns its keep in hour four of a session, when context is full and half the work sits with subagents.

The pricing is direct API. On a subscription, per-call cost is not your problem at all. The weekly limit is, and that is different arithmetic.

None of the reasons people install a harness were measured. No subagents, no hooks, no skills, no MCP, no memory between sessions. Twenty-three tools look like a tax right up until you need one of them.

The authors are also straight about it: the benchmarks are public, and the models may have seen these tasks in training.

What to do about it

Work out your own cost before you change tools. On a subscription, never hitting the limit, you do not pay the harness tax at all; it stays Anthropic's problem.

A short, repeatable task does not need a heavy harness. Bulk edits to a pattern, a linter pass, generating similar tests: twenty-three tools are hauling air here. A long task with an unknown ending wants the opposite, and there the tools and the memory do pay off.

The opening context is the part you control. The twenty-three tools arrive out of the box, but every MCP server you connect writes its own schemas into that same first call. Keep five servers on while using one, and you are paying on top of what Berkeley measured. After this study we started with an audit of our connected servers, and it was the fastest way to see our own bill.

The real news here is not the numbers but that the harness is finally being measured apart from the model. Hold the model fixed and the choice of harness changes the bill by a factor of two, sometimes the result by twelve points. The authors close on a thought we agree with: users should not be making this call by hand, and the right harness adapts as the task unfolds. Until one does, the decision is yours, and at least it now has numbers attached.

Prices and versions as of 17 September 2026: Claude Code 2.1.224, Codex CLI 0.146.0, Pi 0.85.1, direct-API price list dated 1 September 2026. Models and rates change often, so check current figures before budgeting against them.

Sources3expand

1. Melissa Z. Pan, Shuo Yang, Negar Arabzadeh, Wei-Lin Chiang, Ion Stoica, Matei Zaharia, 'HarnessTax: How Much Does the Harness Matter for Coding Agents?', UC Berkeley Sky Lab and Arena, 16 September 2026, harnesstax.github.io.
2. AgentBRANE dashboard data: data/charts/system-frontier-swe, system-frontier-tb, system-agent-context-swe, repository github.com/HarnessTax/HarnessTax.github.io, retrieved 17 September 2026.
3. Codearia Academy, 'Anthropic raised Claude Code's weekly limits by 25% and cut them by 17% the same day', 15 September 2026 (on how context length burns through the limit).