Skip to content
kalanu
Theme
IFEval only · 12 setups · 1 of 6 compared benchmarks

Claude Code · claude-haiku-4-5-20251001 · as published

anthropic

as published

Its score and figures

Every figure the leaderboard's table prints for this setup.

Its score and figures
rank in group1–12
Score76.9 pts72.6–80.6 pts
Pass rate94.2%88.4–98.6%94.2% · 69 of 69 graded
Efficiency25.220.7–28.4
cost1.1×
tokens2.6×
time1.0×
Coverage1/6
Runs69

70.7 + 2.7 + 0.7 + 2.9 ≈ 76.9 (outcome + cost + tokens + time)

  • pass rate on each benchmark's own grader
  • cost
  • tokens
  • time
  • not measured
First run
2026-10-03
Last run
2026-10-03
did not finish (scored as a fail)
0
Tasks with no grade
0

On each benchmark

Each benchmark it ran: its status, pass rate, spend and spend per pass. The studies behind them are its evidence, below.

IFEval

76.9 pts

IFEval: 94.2% · 69 of 69 graded · scored

cost
1.1× the typical spend per passing run7.0% of this setup's cost here went to runs that did not pass or were not graded; efficiency does not price it$0.01 per pass($0.01–$0.02)27% went to the costliest 10% of attempts (7 of 69)
tokens
2.6× the typical spend per passing run6.8% of this setup's tokens here went to runs that did not pass or were not graded; efficiency does not price it40.1k per pass(36.9k–44.2k)17% went to the costliest 10% of attempts (7 of 69)of which 90% cache reads and 8% fresh input
time
1.0× the typical spend per passing run7.4% of this setup's time here went to runs that did not pass or were not graded; efficiency does not price it22.9s per pass(19.9s–27.6s)22% went to the costliest 10% of attempts (7 of 69)

Efficiency and spend per pass can order setups differently: efficiency compares the typical (median) passing run, task by task, while spend per pass averages every attempt, so a few very expensive runs, passing or not, count in full; and spend per pass also counts the attempts that failed or did not finish, and divides by the pass rate. A setup is called the most frugal only where both put it first and both separate it from every other setup.

Evidence

Every published study its runs come from.