Not scored against the other setups: no group compares it.
Continue · claude-haiku-4-5-20251001 · as published
anthropic
as publishedIts score and figures
Every figure the leaderboard's table prints for this setup.
| rank in group | – |
|---|---|
| Score | not measured: it is scored on no benchmark |
| Pass rate | not measured: it is scored on no benchmark |
| Efficiency | not measured: it is scored on no benchmark |
| cost | not measured: its cost is priced on none of the benchmarks it is scored on |
| tokens | not measured: its tokens is priced on none of the benchmarks it is scored on |
| time | not measured: its time is priced on none of the benchmarks it is scored on |
| Coverage | 0/6 |
| Runs | 69 |
- First run
- 2026-10-03
- Last run
- 2026-10-03
- did not finish (scored as a fail)
- 0
- Tasks with no grade
- 3
On each benchmark
Each benchmark it ran: its status, pass rate, spend and spend per pass. The studies behind them are its evidence, below.
IFEval
below the evidence floor
IFEval: below the evidence floor
- cost
- 41.9% of this setup's cost here went to runs that did not pass or were not graded; efficiency does not price it (3 such runs' cost were not recorded)
- tokens
- 45.3% of this setup's tokens here went to runs that did not pass or were not graded; efficiency does not price it (3 such runs' tokens were not recorded)
- time
- 48.8% of this setup's time here went to runs that did not pass or were not graded; efficiency does not price it
Efficiency and spend per pass can order setups differently: efficiency compares the typical (median) passing run, task by task, while spend per pass averages every attempt, so a few very expensive runs, passing or not, count in full; and spend per pass also counts the attempts that failed or did not finish, and divides by the pass rate. A setup is called the most frugal only where both put it first and both separate it from every other setup.
Evidence
Every published study its runs come from.
- ifeval-13-harnesses-2026-10 (69 runs)