The kalanu leaderboard
score = 75 × pass rate above chance + 25 × efficiency
Early results. Setups have been compared on 6 of 6 verified benchmarks with results; an overall ranking needs at least 2. Read why →
Every setup, every benchmark
Each cell is the setup's score on that benchmark. Its shade is the cell's stored band within its column: cells the evidence cannot separate share one shade. A cell that was not scored says why, and is never drawn as a zero.
| Setup | AIME 2025 | benchme — intent tasks | GPQA DiamondAbove chance | HumanEval | IFEval | WindTunnel |
|---|---|---|---|---|---|---|
| AIME 2025, benchme — intent tasks, GPQA Diamond and 2 more · 11 setups · 5 of 6 compared benchmarks | ||||||
| Codex gpt-6.1-solharness default, open-book on knowledge questions | 94.8 pts, rank 1–12, tier 1 of 1 | 73.2 pts, rank 1–12, tier 1 of 2 | 77.1 pts, rank 1–15, tier 1 of 1 | 76.6 pts, rank 1–15, tier 1 of 1 | not run | 75.0 pts, rank 1–17, tier 1 of 2 |
| Claude Code claude-sonnet-5-5harness default, open-book on knowledge questions | 91.0 pts, rank 1–12, tier 1 of 1 | 74.0 pts, rank 1–12, tier 1 of 2 | 73.5 pts, rank 1–15, tier 1 of 1 | 76.5 pts, rank 1–15, tier 1 of 1 | not run | 78.2 pts, rank 1–17, tier 1 of 2 |
| Codex gpt-5.4-miniharness default, open-book on knowledge questions | 85.8 pts, rank 1–12, tier 1 of 1 | 88.1 pts, rank 1–12, tier 1 of 2 | 64.9 pts, rank 1–15, tier 1 of 1 | 81.6 pts, rank 1–15, tier 1 of 1 | not run | 70.2 pts, rank 1–17, tier 1 of 2 |
| DeepSeek Harness claude-haiku-4-5-20251001harness default, open-book on knowledge questions | 70.1 pts, rank 1–12, tier 1 of 1 | 88.3 pts, rank 1–12, tier 1 of 2 | 46.4 pts, rank 1–15, tier 1 of 1 | 81.0 pts, rank 1–15, tier 1 of 1 | not run | 59.6 pts, rank 1–17, tier 1 of 2 |
| Continue claude-haiku-4-5-20251001harness default, open-book on knowledge questions | 72.8 pts, rank 1–12, tier 1 of 1 | 83.3 pts, rank 1–12, tier 1 of 2 | 52.4 pts, rank 1–15, tier 1 of 1 | 85.0 pts, rank 1–15, tier 1 of 1 | not run | 50.7 pts, rank 1–17, tier 1 of 2 |
| eve claude-haiku-4-5-20251001harness default, open-book on knowledge questions | 70.2 pts, rank 1–12, tier 1 of 1 | 88.0 pts, rank 1–12, tier 1 of 2 | 46.1 pts, rank 1–15, tier 1 of 1 | 84.6 pts, rank 1–15, tier 1 of 1 | not run | 54.0 pts, rank 1–17, tier 1 of 2 |
| Hermes claude-haiku-4-5-20251001harness default, open-book on knowledge questions | 55.0 pts, rank 1–12, tier 1 of 1 | 82.0 pts, rank 1–12, tier 1 of 2 | 51.2 pts, rank 1–15, tier 1 of 1 | 78.4 pts, rank 1–15, tier 1 of 1 | not run | 60.5 pts, rank 1–17, tier 1 of 2 |
| Claude Agent SDK claude-haiku-4-5-20251001harness default, open-book on knowledge questions | 65.0 pts, rank 1–12, tier 1 of 1 | 77.6 pts, rank 1–12, tier 1 of 2 | 45.6 pts, rank 1–15, tier 1 of 1 | 76.6 pts, rank 1–15, tier 1 of 1 | not run | 54.0 pts, rank 1–17, tier 1 of 2 |
| opencode claude-haiku-4-5-20251001harness default, open-book on knowledge questions | 66.6 pts, rank 1–12, tier 1 of 1 | 68.7 pts, rank 1–12, tier 1 of 2 | 43.3 pts, rank 1–15, tier 1 of 1 | 81.8 pts, rank 1–15, tier 1 of 1 | not run | 47.9 pts, rank 1–17, tier 1 of 2 |
| Claude Code claude-haiku-4-5-20251001harness default, open-book on knowledge questions | 68.3 pts, rank 1–12, tier 1 of 1 | 69.5 pts, rank 1–12, tier 1 of 2 | 49.2 pts, rank 1–15, tier 1 of 1 | 75.4 pts, rank 1–15, tier 1 of 1 | not run | 44.2 pts, rank 1–17, tier 1 of 2 |
| OpenClaw claude-haiku-4-5-20251001harness default, open-book on knowledge questions | 59.3 pts, rank 1–12, tier 1 of 1 | 17.1 pts, rank 13, tier 2 of 2 | 47.2 pts, rank 1–15, tier 1 of 1 | 76.6 pts, rank 1–15, tier 1 of 1 | not run | 27.7 pts, rank 18, tier 2 of 2 |
| benchme — intent tasks, GPQA Diamond, HumanEval and WindTunnel only · 2 setups · 4 of 6 compared benchmarks | ||||||
| opencode claude-sonnet-5-5harness default, open-book on knowledge questions | not run | 80.9 pts, rank 1–12, tier 1 of 2 | 78.1 pts, rank 1–15, tier 1 of 1 | 80.4 pts, rank 1–15, tier 1 of 1 | not run | 79.0 pts, rank 1–17, tier 1 of 2 |
| Claude Agent SDK claude-sonnet-5-5harness default, open-book on knowledge questions | not run | 75.1 pts, rank 1–12, tier 1 of 2 | 77.5 pts, rank 1–15, tier 1 of 1 | 80.3 pts, rank 1–15, tier 1 of 1 | not run | 84.7 pts, rank 1–17, tier 1 of 2 |
| AIME 2025, GPQA Diamond and HumanEval only · 1 setup · 3 of 6 compared benchmarks | ||||||
| DeepAgents claude-haiku-4-5-20251001harness default, open-book on knowledge questions | 44.0 pts, rank 1–12, tier 1 of 1 | too few passes to price | 49.1 pts, rank 1–15, tier 1 of 1 | 86.4 pts, rank 1–15, tier 1 of 1 | not run | too few passes to price |
3 of 7 shown The rest, in the table →
More groups, in the table