Reading leaderboards

'#1 on the leaderboard!' every launch claims. What do these rankings actually measure — and why can the top model still be wrong for you?

How models get ranked

A benchmark is a fixed test — a big set of questions with known answers — and a model's score is how many it gets right. An arena ranks differently: people vote on which of two anonymous answers is better, and the votes build a leaderboard. Both turn 'which model is good' into a single number you can sort by.

What a rank misses

The catch: a rank sums up tasks that may not be yours. A model that tops a coding benchmark can still write clumsy emails; an arena favourite might fumble your niche. Benchmarks also leak into training data over time, quietly inflating scores. Use leaderboards to narrow the field, then test the top few on your own real work — that's the only benchmark that counts for you.

The short version

• —Benchmarks score models on fixed tests; arenas rank them by human votes.
• —A rank captures some tasks — not necessarily yours.
• —Use leaderboards to shortlist, then test on your own work.