LMArena

How LMArena works

A user prompt is sent to two randomly selected anonymised models. The user votes, and the result updates a Bradley-Terry style rating from which the Arena Score is derived. Aggregated over tens of millions of votes, the ranking is statistically stable. Because the prompts come from whatever real users happen to ask, the coverage is broad but uncontrolled, and there is no ground-truth answer anywhere in the process.

History and current status

The arena launched in 2023 and became the most-watched public ranking in AI, in large part because it resisted the contamination that was breaking static benchmarks: you cannot memorise a preference vote. It rebranded to Arena in early 2026. The same team now also runs Agent Arena, which scores agents from real usage using causal tracing rather than votes, in an explicit attempt to avoid the style-gaming problem.

What the score does not tell you

Preference is confounded with presentation. Response length, formatting, confident tone and markdown structure all raise vote share without raising correctness, so a model can climb by writing more attractively. The 2025 paper The Leaderboard Illusion argues further that private pre-release testing and uneven model deprecation bias ratings toward large labs with the resources to exploit both. Neither critique means the ranking is worthless; both mean it should not be read as a capability score.

This is the general case, not a quirk of one benchmark. For the four failure modes that decide whether any benchmark number means anything, read are AI benchmarks reliable, and for this row in particular, how benchmark contamination inflates scores.

What LMArena scores actually mean

An Arena Score is an Elo-style rating from blind pairwise votes, so it measures which response people prefer, not which one is correct. That distinction governs everything about reading it. Style is a large confound: length, formatting, headers and confident tone all raise win rates independently of substance, which is why models tuned for presentation outrank models tuned for accuracy. Ratings are also only comparable within a snapshot, since the pool of competing models changes constantly. A 20-point gap between two models near the top is usually inside the confidence interval, and the leaderboard publishes those intervals precisely so that small gaps are not over-read.

Who reports LMArena, and how to read it

Labs cite Arena rank at launch, especially when their objective benchmark results are unremarkable. Read it as one axis of three: a preference rating, a composite index, and a hard unsaturated reasoning score. Where a launch leads with Arena rank alone, that is worth noticing. Rank differences inside the published confidence intervals should not be reported as a lead at all.

When to weight LMArena in a model choice

Use the Arena to answer one question well: which model will feel better to a general audience in open-ended chat. That is a real product question and no other benchmark here answers it. Do not use it to judge correctness, coding ability, tool use or factual reliability, all of which it explicitly does not measure. Weigh the critique in the 2025 paper The Leaderboard Illusion, which argues that private pre-release testing and uneven model deprecation bias ratings toward large labs, when interpreting rank differences. For capability, cross-check against LiveBench, which uses objective ground truth and no judges or votes.