Your model ranks #1 on Chatbot Arena. You shipped the PR, open-sourced the weights, published a polished paper, and claimed SOTA.
But what if I told you that changing less than 1% of the battle data — for example, deleting a single vote from certain users about certain models — could knock your model from first to third? And that another model could be pushed to #1 with the same 1% perturbation?
Would you consider that a serious problem?
Oyarhoseini, Lin, and Karimi recently published a paper on arXiv (2605.15761) doing exactly this. They built a unified perturbation framework to analyze how stable leaderboards — especially human-preference pairwise comparison leaderboards like Chatbot Arena — really are.
The answer: not very.
🎲 How Leaderboards Are Built
Chatbot Arena does something clever: instead of asking people to rate models on an absolute scale — where human judgments of absolute scores are notoriously noisy — it has people do A/B tests: show two models' outputs, and ask which is better.
These pairwise preferences are fed into a statistical model called Bradley-Terry. Bradley-Terry dates back to 1952 and was originally used to analyze sports rankings — if Team A beats Team B, and Team B beats Team C, how much stronger is A than C? It converts each match outcome into a "strength" parameter per player (or model), fitted via maximum likelihood estimation.
> When Bradley and Terry published the model in 1952, they were thinking about tennis tournaments. Seventy years later, the same model ranks the intelligence of AI models. This is no coincidence — ranking problems arising from pairwise comparisons share the same mathematical structure in sports and AI evaluation.
Chatbot Arena has accumulated hundreds of thousands of such human preference votes and computes a model leaderboard via Bradley-Terry. Who is #1, who is #10. The entire field watches this table.
But what Oyarhoseini et al. found is that the ranking is abnormally sensitive to changes in the input data.
🪚 Three Surgical Tools
They designed three classes of perturbations — delete, add, flip — to systematically test leaderboard robustness.
"Delete" removes a single match result from the battle data. Not many — one. "Add" inserts a synthetic match result, simulating one user's preference choice. "Flip" reverses a match's outcome — A's win becomes B's win.
They then measured the impact of these operations on three targets: whether the #1-ranked model changes, the ranking consistency of the whole leaderboard (measured by Kendall's tau), and changes in confidence intervals.
The results are unsettling. Sub-1% data perturbation can change who holds first place. This means current leaderboards may not be "objective orderings" but fragile constructs highly sensitive to data selection.
More critically, they prove that the same influence score can be used for two things: auditing fragility ("is this leaderboard easy to manipulate?") and actual manipulation ("how do I achieve the maximal ranking change with minimal data edits?"). The same mathematical tool can be a transparency instrument or an opaque manipulation tool — depending on who wields it.
🎯 Not a Theoretical Concern
They didn't just test in a toy setting. They ran experiments on Chatbot Arena and six other pairwise comparison datasets. The results were consistent across all of them: no robustness.
They also proposed a normalized dataset-level robustness score as a metric for comparing leaderboard stability. The idea: if you maintain a leaderboard, you can use this score to tell the community how stable your leaderboard is — instead of just declaring "we have 100,000 battle records here."
Under their measurements, Chatbot Arena's robustness score is not encouraging.
🤷 What I Don't Know
Several things remain unclear to me.
First, "sub-1% perturbation" is small in absolute terms, but I don't know how "deployable" these perturbations are in practice. If a leaderboard maintainer can detect anomalous patterns — e.g., the same account repeatedly preferring the same loser — attackers would theoretically need cleverer strategies to evade detection. The paper's framework assumes the attacker knows the target model and influence scores, but in reality this information may not be readily available. I'm unsure how effective the framework remains in a limited-information setting.
Second, the paper uses Bradley-Terry as the ranking engine, which is indeed what Chatbot Arena uses. But one of Bradley-Terry's assumptions is "homogeneity" — every match carries equal weight. In reality, some matches are far more informative than others — a battle between two top models tells you more about rankings than one between a top model and a bottom-tier one. Weighted matches or Bayesian variants (like TrueSkill) might be more robust. The paper does not compare against other ranking models.
Third, I'm uncertain how severe "poor robustness" is in practice. If a leaderboard's #1 spot flips under a 0.5% perturbation, that may indicate the leaderboard itself lacks the information to support #1-level granularity — the capability gap between first and second place may be smaller than the measurement noise. In that case, the problem may lie not with attackers but with the leaderboard's resolution. The paper does not discuss this angle.
🔍 Fundamentally, This Is a Measurement Problem
Feynman's first principle: don't fool yourself. But if the tool you use to "not fool yourself" is itself easily manipulated, you need to examine the tool first.
Leaderboards are not bad in themselves. What's bad is pretending they are sturdier than they are. This paper provides a toolbox: before releasing your new model, check whether your ranking can withstand sub-1% data perturbations. If it can, rest easy. If it can't — you and your readers should both know it.
---
References
1. Oyarhoseini, H., Lin, J., & Karimi, A.-H. (2026). *A Unified Perturbation Framework for Analyzing Leaderboard Stability and Manipulation*. arXiv:2605.15761 [cs.LG]. https://arxiv.org/abs/2605.15761
2. Bradley, R. A., & Terry, M. E. (1952). *Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons*. Biometrika, 39(3-4), 324-345.
3. Chiang, W.-L., et al. (2024). *Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference*. arXiv:2403.04132.
4. Herbrich, R., Minka, T., & Graepel, T. (2007). *TrueSkill: A Bayesian Skill Rating System*. Advances in Neural Information Processing Systems, 20.
5. Szymborski, B. (2024). *Benchmarking Benchmark Leakage: LLM Benchmark Manipulation and Protection*. arXiv:2404.12411.