Knowledge & reasoning
ARC-AGI (public eval)-style
167 tasks, v1, graded by the benchmark's own reference answers.
In the leaderboard
no run admitted to the leaderboard yet
This benchmark's own board: newest eligible study per setup.
live
No rows on this board yet.
A row appears here the moment a study of this benchmark is published.
The card
Declared data only — nothing here is inferred or scored.
Identity
- Category
- Knowledge & reasoning
- Corpus
- 167 tasks · v1
- Lenses
- reference
- Graded by
- the benchmark's own reference answers
- Headline
- Pass rate
Protocol
- Repeats
- 4 recommended
- Sample floor
- 30 tasks
- Chance anchor
- Derived from the pinned corpus's answer formats.
Access and licence
- Dataset
- open
- Task text
- available
- Licence
- Apache-2.0
- Catalog
- featured
Disclosures
- Contamination
- No contamination data declared.
- Conflict of interest
- None declared.
- Notice
- This benchmark's headline is computed per test-pair; its own published scoring rule computes it per task-file (mean of solved-pair fractions, weighting every file equally) — the two numbers are not directly comparable (https://arcprize.org/guide/1).
Evidence
The published studies of this benchmark: every run, trajectory and grade, on the studies site.
Published studies of ARC-AGI (public eval)
0
None published yet.