BenchMaxx Ranks 332 AI Models by Their Names — One Minute, Zero Tokens

Modern model evaluation suffers from four chronic problems: data contamination, benchmark saturation, inference cost, and poor reproducibility. A new repository claims to solve all four at once. The method, frankly, is embarrassing: it never runs a single model it evaluates.

The repository is called BenchMaxx, appearing on GitHub two days ago, MIT-licensed, zero dependencies — even the README badge reads "data contamination: impossible." It pulls all 332 callable text models on OpenRouter and ranks them along two dimensions: the version number in the name, and the length of the name. The subtitle states it proudly: "measuring what a model is called, not what it can do."

I followed its reproduction steps and pulled the OpenRouter API myself: currently 461 model entries across 63 vendors, consistent with the repository's October 10 snapshot of 332 models (after filtering out aliases, routers, and non-text models). In other words, this leaderboard is backed by real data — it's just that the data is names.

The Top Spot Belongs to a Model That Uses a Date as a Version Number

First place in the Generation Index track goes to a model called Pareto 26.10 Preview, with a version number of 26.1 — derived from year plus month. Second place, GPT-6.1's 6.1, trails it by 4.3x. I confirmed on OpenRouter that this model is genuinely listed, with a vendor named unbiased. BenchMaxx's methodology docs have a section dedicated to it, titled "Goodhart's Law": when a metric becomes a target, it ceases to be a good metric — rank by version number, and vendors will name themselves 26.10.

First place in the Nomenclature Length track is Gemini 3.1 Pro Preview Custom Tools, at 35 characters — a name that genuinely requires breathing room. Out of curiosity, I counted the longest model name in the current API — a Google image model name with 56 characters — but it doesn't output text, so it doesn't qualify for the leaderboard.

The Serious Parts Are Funnier Than the Joke

The funniest thing about this project is that its "key findings" are all earnestly computed industry facts.

Version number inflation is real: the industry's average version number rises by 0.45 generations per year. The two fastest climbers are StepFun at +2.20 per year and SpaceXAI at +2.16 — covering in two years the naming ground others take four or five to traverse. The opposite of inflation also exists: Meta went from Llama 4 to Muse Spark — no more Llama in the name — with generation numbers dropping from 4 back to 1.x, turning velocity outright negative (−1.13 per year). I pulled Meta's live OpenRouter roster to confirm: Muse Spark 1.1, 1.2, 1.3, plus Muse Wonder and Muse Glimmer — the name Llama is indeed gone. A rename equals a generation reset: perhaps the most elegant form of "negative optimization."

There's even a metric called naming efficiency: o3 packs three generations into two characters, an efficiency of 1.50, the highest in the field. The methodology even includes a "decimal interpretation principle" to handle the normalization of Grok 4.20 — read as a decimal, 4.20 is 4.2, so it ranks below its own predecessor Grok 4.3. As for the 12% of models with no version number in their names at all, the repository categorizes them as "conscientious objectors": Cohere and Perplexity have zero GI coverage, not participating with a single digit.

The Target of the Satire Isn't the Models

One FAQ entry is worth quoting: "Can vendors game this benchmark? Yes — just rename. Unlike other benchmarks, we state this openly."

This line is the project's real edge. Ranking by name obviously measures nothing about intelligence — but when version numbers have themselves become marketing assets (6.1 sounds louder than 6, and 26.10 can intimidate everyone), then "ranking by version number" becomes an honest mirror: it measures the productivity of the naming department, and the naming department has been working very hard these past two years. The repository has a protocol version number, CI for compliance tests, snapshot archives, and a threats-to-validity analysis — its formal rigor exceeds that of many leaderboards that actually run scores but can't articulate what they measure. The highest form of satire is sincerity.

Attribution: a personal repository by shuiyue-cmyk, created October 10, currently 0 stars, with a bilingual README. It doesn't aim to go viral — it aims to be accurate. The full evaluation runs in one minute without burning a single token — in 2026, this is probably the only benchmark that will never be scolded for "failing to run."