Knowledge & reasoning
MMLU-Pro-style
12,032 tasks, v1, graded by the benchmark's own reference answers.
In the leaderboard
no run admitted to the leaderboard yet
This benchmark's own board: newest eligible study per setup.
live
No rows on this board yet.
A row appears here the moment a study of this benchmark is published.
The card
Declared data only — nothing here is inferred or scored.
Identity
- Category
- Knowledge & reasoning
- Corpus
- 12,032 tasks · v1
- Lenses
- reference
- Graded by
- the benchmark's own reference answers
- Headline
- Pass rate
Protocol
- Repeats
- 1 recommended
- Sample floor
- 100 tasks
- Chance anchor
- Derived from the pinned corpus's answer formats.
Access and licence
- Dataset
- open
- Task text
- available
- Licence
- MIT
- Catalog
- featured
Disclosures
- Contamination
- No contamination data declared.
- Conflict of interest
- None declared.
Evidence
The published studies of this benchmark: every run, trajectory and grade, on the studies site.
Published studies of MMLU-Pro
0
None published yet.