The cheaper model per token can be the more expensive model per task, and Databricks just proved it

Nathaniel Whittemore drew attention to a finding that should sit uncomfortably with anyone who selects AI models on per-token price alone. In Databricks’ coding-agent benchmark run against a multi-million-line production codebase, Sonnet 5 carried a lower per-token cost than Opus 4.8 by roughly 1.7x. Yet when the bill was calculated per completed task, Sonnet 5 came out more expensive: approximately $2.09 against Opus 4.8’s $1.94.

The reason is not a pricing anomaly. Sonnet 5 required more iterations and burned more reasoning tokens to reach the same result that Opus 4.8 reached more directly. The cheaper model, in other words, did more work to finish the job, and that additional work erased the per-token savings and then some.

Databricks published the benchmark on July 8, 2026, on its engineering blog. The methodology is explicit: the team tested coding agents against a real internal codebase of serious scale, not a synthetic benchmark designed to flatter any particular model. The cost figures Whittemore cited, $1.94 and $2.09 per task, appear in that published record and have since been reported by The Register, among other outlets covering the results.

However, Sonnet cost around $2 per task or 2 2.09 per task versus 1.94 for Opus. So, because Sonnet needed more iterations and more reasoning had to spend way more tokens to get to the same results, overall Opus, which is significantly on paper more expensive model, it was cheaper to operate Nathaniel Whittemore

The finding matters beyond this single comparison. Enterprise teams running coding agents at volume tend to think about model cost in terms of what appears on the API pricing page. That figure is the one that shapes budget conversations, vendor negotiations, and the intuition about which model is the “expensive” choice. What Databricks’ benchmark surfaces is that per-token price and per-task cost can move in opposite directions, and the direction that matters for operational spend is the one that accounts for how many tokens a model actually consumes to finish something.

A less capable model attempting a hard task is not merely slower. It is iterative in a way that compounds token usage. Each additional reasoning pass, each corrective loop, adds to the total. When a task is simple enough that Sonnet 5 resolves it in one or two passes, the per-token advantage likely holds. But on a multi-million-line codebase where tasks are genuinely complex, the benchmark suggests the more powerful model’s directness is worth paying for, and then some.

This does not mean Opus 4.8 is the right choice in every agentic deployment. Task difficulty is the key variable, and organizations running lighter workloads may find the per-token math still favors a less powerful model at their actual distribution of tasks. What the Databricks result forecloses is the simpler argument: that the model listed at a lower per-token price is reliably the cheaper model to operate. On complex coding tasks run at scale, the evidence now says otherwise.

Whittemore’s framing captures the inversion concisely. The model that is “significantly on paper more expensive” turned out to be cheaper to operate in practice. That gap between the price on the page and the cost in production is the measurement most teams are not yet making systematically. Databricks made it, published it, and the numbers do not require interpretation to sting.