The kalanu leaderboard
score = 75 × pass rate above chance + 25 × efficiency
Early results. Setups have been compared on 6 of 6 verified benchmarks with results; an overall ranking needs at least 2. Read why →
Every setup of each harness
Grouped for reading. Each setup keeps its own group and rank; a harness has no score of its own here. Harnesses follow the catalog's order, never a ranking; each harness's setups follow the leaderboard's.
Claude Agent SDK
- claude-haiku-4-5-20251001harness default, open-book on knowledge questionsscored on AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more63.8 ptsrank in group 4–10pass-rate tier 4–10
- claude-sonnet-5-5harness default, open-book on knowledge questionsscored on benchme — intent tasks, GPQA Diamond, HumanEval and WindTunnel79.4 ptsrank in group 1–2pass-rate tier 2
- claude-haiku-4-5-20251001as publishedscored on IFEval84.9 ptsrank in group 1–12pass-rate tier 1–12
Claude Code
- claude-sonnet-5-5harness default, open-book on knowledge questionsscored on AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more78.6 ptsrank in group 1–3pass-rate tier 1–2
- claude-haiku-4-5-20251001harness default, open-book on knowledge questionsscored on AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more61.3 ptsrank in group 4–10pass-rate tier 4–10
- claude-sonnet-5-5as publishedscored on IFEval78.7 ptsrank in group 1–12pass-rate tier 1–12
- claude-haiku-4-5-20251001as publishedscored on IFEval76.9 ptsrank in group 1–12pass-rate tier 1–12
- claude-haiku-4-5-20251001with my setupWebMCPscored on WindTunnel81.1 ptsrank in group 1–4pass-rate tier 1–4
- claude-haiku-4-5-20251001with my setupPlaywrightscored on WindTunnel79.3 ptsrank in group 1–4pass-rate tier 1–4
Codex
- gpt-6.1-solharness default, open-book on knowledge questionsscored on AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more79.4 ptsrank in group 1–3pass-rate tier 1–2
- gpt-5.4-miniharness default, open-book on knowledge questionsscored on AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more78.1 ptsrank in group 1–3pass-rate tier 3
- gpt-5.4-minias publishedscored on IFEval88.5 ptsrank in group 1–12pass-rate tier 1–12
- gpt-6.1-solas publishedscored on IFEval79.2 ptsrank in group 1–12pass-rate tier 1–12
- gpt-5.4-miniwith my setupWebMCPscored on WindTunnel80.8 ptsrank in group 1–4pass-rate tier 1–4
- gpt-5.4-miniwith my setupPlaywrightscored on WindTunnel79.0 ptsrank in group 1–4pass-rate tier 1–4
Aider
Continue
opencode
- claude-haiku-4-5-20251001harness default, open-book on knowledge questionsscored on AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more61.7 ptsrank in group 4–10pass-rate tier 4–10
- claude-sonnet-5-5harness default, open-book on knowledge questionsscored on benchme — intent tasks, GPQA Diamond, HumanEval and WindTunnel79.6 ptsrank in group 1–2pass-rate tier 1
- claude-haiku-4-5-20251001as publishedscored on IFEval62.7 ptsrank in group 1–12pass-rate tier 1–12
DeepAgents
Hermes
DeepSeek Harness
OpenClaw
eve