When does the cheaper model actually cost less?
Claude Opus 5.5 against Claude Sonnet 5 on 20 coding tasks of rising difficulty
Per token, Sonnet costs half as much as Opus. So it’s easy to assume Sonnet is the economical choice and Opus is for when quality matters. We tested that on our benchmark harness, Eurystheus. It sets a model a ladder of 20 coding tasks in five difficulty tiers:
- everyday fixes and parsers;
- algorithms and refactors;
- performance work and interpreters;
- data structures, virtual machines, numerics and geometry;
- mathematical problems.
Each model ran each task three times through Claude Code, with no skills, no web access and a fresh container every run. Hidden tests graded the results afterwards.
The gap grows with difficulty
| Tier | Fix length, lines (mean; range) | Sonnet tokens per task | Opus tokens per task | Sonnet cost ÷ Opus cost | Runs passing every test (Sonnet / Opus) |
|---|---|---|---|---|---|
| 1. Fixes and parsers | 29; 3–46 | 140k | 57k | 0.86× | 10/12 · 12/12 |
| 2. Algorithms and refactors | 78; 59–116 | 338k | 88k | 1.56× | 12/12 · 12/12 |
| 3. Performance and interpreters | 175; 20–329 | 563k | 193k | 1.31× | 11/11 · 12/12 |
| 4. Data structures, VMs, numerics | 199; 79–464 | 1.68 M | 216k | 2.25× | 7/12 · 12/12 |
| 5. Mathematical | 131; 120–140 | 479k | 78k | 2.88× | 12/12 · 12/12 |
Costs are at Anthropic’s API list prices. Each figure is the tier’s average of each task’s median run. Fix length is how many lines the benchmark’s own reference solution adds: a model-independent measure of how much code the task needs.
Tokens. Opus used fewer tokens on every one of the 20 tasks. From tier 1 to tier 4, its tokens per task grew 3.8×; Sonnet’s grew 12×.
Time. Sonnet’s run time also fell further behind as the tasks got harder: 1.6× Opus’s in tier 1 and 6.5× in tier 5.
Cost. Sonnet’s lower per-token price only wins out on the easiest tier, where it is about 14% cheaper per task.
Correctness. Up to tier 3, both models are close to perfect. At tier 4 they part ways. Opus passed every test in all 60 of its runs across the benchmark. On the tier-4 tasks, Sonnet fully passed 7 of 12 runs.
Short fixes favour Sonnet; beyond that, it’s the kind of task
Sort the 20 tasks by fix length and a clear line appears at about 60 lines:
| Fix length | Tasks | Sonnet cost ÷ Opus cost |
|---|---|---|
| 60 lines or fewer | 6 | 0.68×–1.02×: Sonnet cheaper or level on every one |
| Over 60 lines | 14 | Sonnet more than 1.1× as expensive on 10 of the 14 |
Past that line, though, more code doesn’t mean a bigger gap. The three longest fixes are close to level: a regex engine (220 lines, 1.06×), a mini SQL engine (329 lines, 0.99×) and a bytecode VM (464 lines, 1.71×). The worst ratios sit on medium-length fixes: 2.5–3.3× on the four maths problems (120–140 lines) and 5.5× on a stiff ODE solver (79 lines).
What those expensive tasks share is that most hinge on finding a particular algorithm or a piece of mathematics. Sonnet cost more than 2.4× as much as Opus on eight of them: a Myers diff, an async job scheduler, a sequence CRDT, the ODE solver and all four maths problems. It spent its tokens iterating towards an approach; on the ODE solver it took around 40 turns, against Opus’s 3–23.
As a predictor of Sonnet’s relative cost, fix length (rank correlation 0.52) beats how many tokens Opus needed (0.27). The difficulty tier is better still (0.67). With 20 tasks, 0.52 is only borderline meaningful; the 60-line threshold is the firmer result, because it holds on every task.
So when should you use Sonnet?
Rarely, based on this data.
- On a Claude subscription: use Opus. It used the same share of the usage allowance per run as Sonnet on the easier tasks, and a third as much on the harder ones. It was also faster on all 20 tasks.
- Through the API: Sonnet saves a little on short fixes of about 60 lines or fewer: around 20%, or 2 cents a task, while running slower on all of them. It loses heavily when a task turns out hard, as on the ODE solver ($1.69 against $0.31 per run). You rarely know in advance which tasks will be hard, so a mixed workload favours Opus: across all 20 tasks, it came out 45% cheaper.
Sonnet still makes sense for a high volume of short, clearly specified fixes, where tests catch mistakes cheaply and speed doesn’t matter.
Caveats
- Small sample: these are 2–3 runs per task per model, so the figures will shift a little as more runs come in.
- No skills: this is a baseline without any skill. A skill that guides Sonnet’s approach could narrow the gap, and measuring that is what Eurystheus is built for.
- Sonnet 5 only: this tests Sonnet 5, not newer releases.
The full methodology, per-task data and limitations are in the baseline report.