Free AI models suddenly look much worse: leaderboard update delivers a massive rankings hit
Scores have been lowered across the board for free, open-weight models.
Free AI models now rank much worse than before. Image by Cybernews.
Follow us
- Proprietary models now dominate Artificial Analysis leaderboard after adding more private test sets to its Intelligence Index.
- Claude Opus 5.5 leads with 58 points, while GPT-6 Astra rises to second with 57.
- Kimi K3.1 Pro's score drops from 55 to 46, widening the top open-weight gap to 12 points.
- Some commenters question the reshuffle because several open-weight models and Grok lost significant ground.
Key Takeaways by nexos.ai, reviewed by Cybernews staff.
For a while now, free AI models from China appeared to be neck and neck with the frontier AI labs in the US. However, Artificial Analysis, a major AI benchmarking lab, has delivered a devastating blow to open-weight AI models, as well as Grok. The gap is much wider, updated rankings now suggest.
Artificial Analysis has updated its AI benchmarks and leaderboard, relying more heavily on private datasets, previously unseen in public. Proprietary AI models now completely dominate the leaderboard.
No longer is Kimi K3.1 Pro ahead of Claude Fable, nor is Deepeek V4.1 Flash outperforming Claude Haiku 5.5. The new benchmark shakes up the previous rankings, and even Grok 4.7 loses significant ground.
The change significantly widens the gap between leading frontier models – Claude, GPT, Gemini, and Muse – and the rest.
Stay updated with our latest stories and follow us on social media
Be the first to discover new stories, ideas, and updates from our team.
How do the rankings change?
The new Intelligence index by Artificial Analysis still ranks all AI models on a scale of 100, roughly reflecting the proportion (%) of tasks they complete correctly across a range of benchmarks.
Anthropic’s Claude Opus 5.5 leads the pack, with the same score of 58. OpenAI’s GPT-6 Astra significantly improved its score from 53 points to 57 and is now second.
The third-best AI lab is Google with its Gemini 4 Argon, which saw a score decrease from 53 to 50 points, followed by Meta’s Muse Wonder Full, which received 50 points, down from 56.
The best open-weight model, Kimi K3.1 Pro, whose weights have not even been released yet, suffers a devastating blow. Its score drops from 55 to 46, a 16% decrease.
“The open weights gap widens from 3 points to 12,” Artificial Analysis acknowledges.
It's even worse for some other leading open-weight models. MiMo-V2.6-Pro, Xiaomi’s most capable model, has been downgraded to just 30 points, down from 46.
DeepSeek V4.1 Flash, one of the most popular free models, lost 13 points, dropping from 45 to 32. Similarly, GLM-5.3 and Qwen 3.8 Max, which previously had 45 points each, now have 32 and 34 points, respectively.
Qwen 3.8 27B, a very popular small AI model, previously boasted near-frontier capabilities and was compared to Opus 4.6. Now its core is just 20, compared with 34 in the previous ranking.
Grok 4.7, the best chatbot from xAI, scores 34 on the updated benchmarks, compared to the previous score of 46
Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32, 26 points behind the leader, and below Claude Haiku 5.5 (max, 36)
What did Artificial Analysis do?
The company explains that its Intelligence Index v5, which replaced the previous version, features 2 new benchmarks and 3 upgrades. The index is split equally across 4 categories: knowledge work, coding, science, and general.
“For the first time, the majority of the index is built on private datasets,” Artificial Analysis said.
“Sixty percent of the index is now based on evaluations with private test sets or held-out answers.”
The team explains that benchmarks best reflect real-world usage of AI models today.
The updated index includes 2 new benchmarks for coding and agentic tasks: FrontierCode 2.0, a private benchmark of real-world coding tasks, and Terminal-Bench-Science, which features 70 agentic scientific research tasks run in a terminal environment.
The company swapped Humanity's Last Exam with HLE-Diamond, which tests advanced reasoning and specialist knowledge across 1,000 difficult questions, updated 2 other datasets, and removed 3 benchmarks, as follows:
- GDPval-AA v2.1: which evaluates models on real-world, economically valuable tasks.
- SciCode: which tests models’ ability to solve scientific programming problems.
- AA-LCR v1.1: which tests long-context reasoning.
On Silicon Valley’s premier tech forum, Hacker News, the reactions to the rankings reshuffle are mixed, and many critics distrust a huge drop in open weights scores.
“This is really sus. I’d love a more detailed explanation and “take” from Artificial Analysis on what problem they felt needed correcting and why things have shuffled the way they have. And how can they claim their index has relevance if positions can be switched around so easily?” the top comment reads.
Artificial Analysis is a leading independent AI benchmarking and market-intelligence platform, co-founded in late 2023 by Micah Hill-Smith (CEO), an ex-McKinsey consultant with a computer science background, and George Cameron (CPO), a former strategy consultant.