Codex · gpt-5.4-mini · Playwright
openai
with my setupPlaywrightIts score and figures
Every figure the leaderboard's table prints for this setup.
| rank in group | 1–4 |
|---|---|
| Score | 79.0 pts75.8–81.4 pts |
| Pass rate | 98.0%94.0–100.0%98.0% · 50 of 50 graded |
| Efficiency | 22.018.4–26.2 |
| cost | 0.57× |
| tokens | 0.52× |
| time | 1.0× |
| Coverage | 1/6 |
| Runs | 50 |
73.5 + 2.0 + 0.5 + 3.0 = 79.0 (outcome + cost + tokens + time)
- pass rate on each benchmark's own grader
- cost
- tokens
- time
- not measured
- First run
- 2026-09-28
- Last run
- 2026-09-28
- did not finish (scored as a fail)
- 0
- Tasks with no grade
- 0
On each benchmark
Each benchmark it ran: its status, pass rate, spend and spend per pass. The studies behind them are its evidence, below.
WindTunnel
79.0 pts
WindTunnel: 98.0% · 50 of 50 graded · scored
- cost
- 0.57× the typical spend per passing run1.6% of this setup's cost here went to runs that did not pass or were not graded; efficiency does not price it$0.07 per pass($0.04–$0.10)51% went to the costliest 10% of attempts (5 of 50)
- tokens
- 0.52× the typical spend per passing run1.3% of this setup's tokens here went to runs that did not pass or were not graded; efficiency does not price it247.9k per pass(134.6k–377.1k)56% went to the costliest 10% of attempts (5 of 50)of which 89% cache reads and 10% fresh input
- time
- 1.0× the typical spend per passing run1.8% of this setup's time here went to runs that did not pass or were not graded; efficiency does not price it55.5s per pass(45.4s–1m 08s)28% went to the costliest 10% of attempts (5 of 50)
Efficiency and spend per pass can order setups differently: efficiency compares the typical (median) passing run, task by task, while spend per pass averages every attempt, so a few very expensive runs, passing or not, count in full; and spend per pass also counts the attempts that failed or did not finish, and divides by the pass rate. A setup is called the most frugal only where both put it first and both separate it from every other setup.
Evidence
Every published study its runs come from.