The kalanu leaderboard
score = 75 × pass rate above chance + 25 × efficiency
Early results. Setups have been compared on 6 of 6 verified benchmarks with results; an overall ranking needs at least 2. Read why →
Every setup on each model
Grouped for reading. Each setup keeps its own group and rank; a model has no score of its own here. Models follow the catalog's order, never a ranking; each model's setups follow the leaderboard's.
claude-haiku-4-5-20251001
- DeepSeek Harnessharness default, open-book on knowledge questionsscored on AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more69.1 ptsrank in group 4–10pass-rate tier 4–10
- Continueharness default, open-book on knowledge questionsscored on AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more68.8 ptsrank in group 4–10pass-rate tier 4–10
- eveharness default, open-book on knowledge questionsscored on AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more68.6 ptsrank in group 4–10pass-rate tier 4–10
- Hermesharness default, open-book on knowledge questionsscored on AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more65.4 ptsrank in group 4–10pass-rate tier 4–10
- Claude Agent SDKharness default, open-book on knowledge questionsscored on AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more63.8 ptsrank in group 4–10pass-rate tier 4–10
- opencodeharness default, open-book on knowledge questionsscored on AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more61.7 ptsrank in group 4–10pass-rate tier 4–10
- Claude Codeharness default, open-book on knowledge questionsscored on AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more61.3 ptsrank in group 4–10pass-rate tier 4–10
- OpenClawharness default, open-book on knowledge questionsscored on AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more45.6 ptsrank in group 11pass-rate tier 11
- DeepAgentsharness default, open-book on knowledge questionsscored on AIME 2025, GPQA Diamond and HumanEval59.8 ptsrank in group –pass-rate tier 1
- Aiderharness default, open-book on knowledge questionsscored on GPQA Diamond, HumanEval and WindTunnel63.0 ptsrank in group –pass-rate tier 1
- Claude Agent SDKas publishedscored on IFEval84.9 ptsrank in group 1–12pass-rate tier 1–12
- OpenClawas publishedscored on IFEval78.9 ptsrank in group 1–12pass-rate tier 1–12
- Claude Codeas publishedscored on IFEval76.9 ptsrank in group 1–12pass-rate tier 1–12
- DeepAgentsas publishedscored on IFEval73.8 ptsrank in group 1–12pass-rate tier 1–12
- DeepSeek Harnessas publishedscored on IFEval73.6 ptsrank in group 1–12pass-rate tier 1–12
- Aideras publishedscored on IFEval70.2 ptsrank in group 1–12pass-rate tier 1–12
- eveas publishedscored on IFEval69.9 ptsrank in group 1–12pass-rate tier 1–12
- Hermesas publishedscored on IFEval69.8 ptsrank in group 1–12pass-rate tier 1–12
- opencodeas publishedscored on IFEval62.7 ptsrank in group 1–12pass-rate tier 1–12
- Claude Codewith my setupWebMCPscored on WindTunnel81.1 ptsrank in group 1–4pass-rate tier 1–4
20 of 22 shown The rest, in the table →
claude-sonnet-5-5
- Claude Codeharness default, open-book on knowledge questionsscored on AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more78.6 ptsrank in group 1–3pass-rate tier 1–2
- opencodeharness default, open-book on knowledge questionsscored on benchme — intent tasks, GPQA Diamond, HumanEval and WindTunnel79.6 ptsrank in group 1–2pass-rate tier 1
- Claude Agent SDKharness default, open-book on knowledge questionsscored on benchme — intent tasks, GPQA Diamond, HumanEval and WindTunnel79.4 ptsrank in group 1–2pass-rate tier 2
- Claude Codeas publishedscored on IFEval78.7 ptsrank in group 1–12pass-rate tier 1–12
gpt-5.4-mini
- Codexharness default, open-book on knowledge questionsscored on AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more78.1 ptsrank in group 1–3pass-rate tier 3
- Codexas publishedscored on IFEval88.5 ptsrank in group 1–12pass-rate tier 1–12
- Codexwith my setupWebMCPscored on WindTunnel80.8 ptsrank in group 1–4pass-rate tier 1–4
- Codexwith my setupPlaywrightscored on WindTunnel79.0 ptsrank in group 1–4pass-rate tier 1–4
gpt-6.1-sol