Skip to content
kalanu
Theme
What the leaderboard is built on

Every benchmark

Every benchmark the leaderboard may use, and what it discloses about each. A benchmark joins the comparison once 2 setups are scored on it.

  • AIME 2025Knowledge & reasoningin the leaderboard: 30 comparison tasks, 12 setups scored; graded by the benchmark's own answer key or checks
  • benchme — intent tasksWebin the leaderboard: 163 comparison tasks, 13 setups scored; graded by the benchmark's own answer key or checksuploaded by the publisher of this indexlicence not yet verified
  • GPQA DiamondKnowledge & reasoningin the leaderboard: 198 comparison tasks, 15 setups scored; graded by the benchmark's own answer key or checks
  • HumanEvalCodein the leaderboard: 164 comparison tasks, 15 setups scored; graded by the benchmark's own answer key or checks
  • IFEvalKnowledge & reasoningin the leaderboard: 69 comparison tasks, 12 setups scored; graded by the benchmark's own checks
  • WindTunnelWebin the leaderboard: 50 comparison tasks, 18 setups scored; graded by the benchmark's own answer key or checksuploaded by the publisher of this indexlicence not yet verified
  • ARC-AGI (public eval)Knowledge & reasoningno run admitted to the leaderboard yet
  • FACTS Grounding (public set)Knowledge & reasoningno run admitted to the leaderboard yet
  • GSM8KKnowledge & reasoningno run admitted to the leaderboard yet
  • HarmBenchSafetyno run admitted to the leaderboard yet
  • Humanity's Last Exam (text)Knowledge & reasoningno run admitted to the leaderboard yet
  • LiveCodeBenchCodeno run admitted to the leaderboard yet
  • MATHKnowledge & reasoningno run admitted to the leaderboard yet
  • MATH-500Knowledge & reasoningno run admitted to the leaderboard yet
  • MBPPCodeno run admitted to the leaderboard yet
  • MMLUKnowledge & reasoningno run admitted to the leaderboard yet
  • MMLU-ProKnowledge & reasoningno run admitted to the leaderboard yet
  • SimpleQAKnowledge & reasoningno run admitted to the leaderboard yet
  • SWE-bench VerifiedAgenticno run admitted to the leaderboard yetlicence not yet verified
  • Terminal-Bench 2.xAgenticno run admitted to the leaderboard yet
  • TruthfulQAKnowledge & reasoningno run admitted to the leaderboard yet