Image: Zhang, Da, Lee, and colleagues, GSM1k paper, 2024. Official figure showing accuracy gaps between GSM8k and a fresh math test.
Every major model launch now arrives with a chart of new high scores. This site covers those claims week after week, from SemiAnalysis's GDPx test of agents, to OpenAI's LifeVal work on everyday chores, to the latest Artificial Analysis Intelligence Index. The numbers are easy to rank and easy to share. They are also easier to overread than a launch post usually admits.
A benchmark is a fixed set of tasks with a scoring rule, so labs and buyers can compare models under the same conditions. The usual promise is that the test items are held out, meaning the model was not trained on the questions it is scored on. When a test is well-known and sitting on the public web, that promise is weaker than it looks.
The first way scores get inflated is contamination. Popular benchmarks are published as downloadable files and then copied onto blogs, GitHub repos, and forum posts. Those copies can be scraped into a training corpus even if no one at the lab set out to cheat. A 2024 survey by Xu, Guan, Greene, and Kechadi at University College Dublin calls this benchmark data contamination, and describes it as evaluation data that has entered training in a way that skews the score. Once a test is public, a later pre-training run can absorb it without anyone noticing.
Researchers have also tried to detect that leak after the fact. In a 2024 paper titled Time Travel in LLMs, Golchin and Surdeanu at the University of Arizona describe a guided prompt. They give a model the name of a dataset, whether the item is from the train or test split, and the start of a reference example, then ask the model to finish it. A close match is treated as a sign that the item was already in the model's data. Their best method agreed with human experts between 92% and 100% of the time when they checked seven datasets, and they reported that GPT-4 appeared to be contaminated with AG News, WNLI, and XSum.
A cleaner test of the same problem is to write a fresh exam that looks like the old one. In 2024, Zhang, Da, Lee, and colleagues at Scale AI released GSM1k, a set of 1,000 new grade-school math problems designed to match the widely used GSM8k benchmark on human solve rates, solution steps, and the size of the answers. GSM8k has been public for years, so any model trained on a large web crawl may have seen its questions, or close copies of them.
When the team scored models on both sets, some families dropped by as much as 8% on the new problems. The paper found systematic overfitting in some families, including Mistral and Phi, and in some Llama releases. Those groups were stronger on GSM8k than on GSM1k for almost every size and version the authors checked. Gemini, GPT, and Claude models showed little or no drop, and Mistral Large was the only Mistral model with no signs of overfitting. The authors also reported a Spearman rank correlation of 0.36, a figure they write as r2 of 0.36, between how readily a model generated GSM8k examples and the size of its drop.
That is not a story in which every official score is fake. The same paper says many models, especially those at the frontier, showed little overfitting, and that models in general still solved novel math problems that could not have been in their training data. Phi-2 dropped by 6% and still solved more than half of GSM1k. The useful lesson is narrower. A well-known public test can flatter a model that has seen too much of it, and a gap between the public score and a fresh, matched test is a warning sign worth reading.
A different kind of distortion shows up on live leaderboards. In a May 2025 paper called The Leaderboard Illusion, Singh, Nan, Wang, and colleagues studied Chatbot Arena, the human preference board run by the LMArena team. The authors argue that undisclosed private testing, selective publication of the best score, and uneven access to the prompts that users submit can make the public ranking look more decisive than it is.
They reported that as many as 27 private Meta model variants were tested in a single month in the lead-up to Llama 4. Preferred providers, they wrote, included Meta, Google, OpenAI, and Amazon. The paper's abstract estimates that Google and OpenAI received 19.2% and 20.4% of all Arena data, while a combined 83 open-weight models received an estimated 29.7%. In the body, the authors also write that proprietary providers received an estimated 61.4% of the data, and that 41 fully open-source models received an estimated 8.9%. The snapshot they scraped ran from January to March 2025 and coincided with the Llama 4 launch, so other providers may be undercounted.
The Arena team published a public response on May 9, 2025, which remains, at the time of this writing, the reply posted on its official blog. The team thanked the authors. It said future policy would state more clearly that all providers may test multiple pre-release variants, subject to system limits, and would mark retired models more clearly. If more than 10 models were tested in parallel before release, it said it would treat a score as provisional until 2,000 fresh votes arrive after the model is public. It also disputed several claims in the paper. It said a policy on pre-release testing had been public since March 2024, that any provider can submit as many public and private variants as capacity allows, and that only results for public models are published. It said official statistics from April 27, 2025 showed open models at 40.9% of Arena use, a category that includes open-weight systems such as Llama and Gemma, and it argued that the paper's much smaller open-source share missed those models. It said a claimed boost of more than 100 Arena points came from a simulation unrelated to Arena data. Its own analysis put a typical boost around 11 Elo points after 50 tests and 3,000 votes, a gap that fades as new votes arrive. It also said a reported relative gain of up to 112% was measured on ArenaHard, a static set of 500 prompts scored by an LLM judge, not on the live human-voted Arena.
This article is not in a position to settle that dispute. What a reader can take from both documents is simpler. A public Arena rank is a useful signal, not the last word. It helps to know whether the named model is the one people can actually use, and whether a lab has had many private tries before one score went up on the board.
Some newer tests try to shrink these problems. LiveBench, introduced by White, Dooley, Roberts, Pal, and colleagues and later accepted at ICLR 2025, is built to resist contamination and to avoid the quirks of LLM judges and crowd votes. Questions are drawn from recent math contests, research papers, news, and other fresh sources, then scored automatically against ground-truth answers. The six categories in the paper are math, coding, reasoning, language, instruction following, and data analysis. New questions were added monthly, and at the time of that paper the top models still scored below 70%. Freshness is not forever. As the contamination survey notes, new data stays clean only until it enters a later training corpus.
For software work, OpenAI and the SWE-bench authors released SWE-bench Verified in August 2024. The original SWE-bench asks an agent to resolve a real GitHub issue from one of 12 open-source Python projects by editing the repo so that both new tests and existing tests pass. OpenAI found that some unit tests were overly specific or even unrelated to the issue, that some issue texts were too vague to solve fairly, and that some evaluation environments failed in ways that could reject a valid patch. Ninety-three experienced Python developers annotated 1,699 random test samples, and the authors kept a 500-item subset whose problem descriptions were clear and whose tests were judged to be fair. The official SWE-bench site describes the same human-filtered set. The point is not that the original test was too easy. In this case, noisy scoring can also sell a model short.
Other published remedies include keeping a test private, encrypting it, or rotating it often enough that yesterday's leak is not today's exam. Those are strategies described in the contamination survey, not a guarantee that any one lab is already doing them well.
When a launch chart lands, a few checks do more work than a single percentage. Prefer numbers from an independent evaluation, or from more than one, such as the Artificial Analysis index this site has already covered, or a newer test built for a specific job, such as GDPx or LifeVal. Look for benchmarks that are still fresh, or that are updated often, rather than a year-old exam that every lab has had time to optimize against. If a paper or lab reports both a public score and a score on a new, matched test, the gap is often more informative than either number alone. Confirm that the exact model version on the chart is the one being shipped, not a private variant with a similar name. Then try the model on your own work. A score cannot tell you whether the model will follow your repo's conventions, stay within your safety rules, or hold up on the messy prompts you actually write.
A related caveat is that the chatbot in your product may not stay identical to the model that set a public score. This site has already looked at research on whether a familiar assistant can seem to get worse over time. That is another reason to treat a launch table as a starting point, not as a warranty.
Benchmark scores are still useful. They are one of the few ways to put many models on the same scale, and independent tests have made those comparisons more honest than a vendor slide deck alone. They are not a final verdict on whether a new model is better for the work you need done. Read the chart, check how old and how public the test is, and leave room for a result that will only show up on tasks nobody has published yet.
Comments