AA Smart Index undergoes major overhaul: Claude takes the crown, DeepSeek and GLM trail by 26 points

Beating AI News Flash: AI evaluation agency Artificial Analysis has significantly overhauled its Intelligence Index leaderboard. The new version adds 132 coding tasks based on real open-source projects — AI-generated code must not only pass tests but also meet the standard of being merge-worthy by project maintainers. Additionally, 70 research tasks have been added, requiring AI to actually complete research workflows in a terminal. To reduce the impact of models having seen test questions in advance, 60% of the scoring weight in the new version comes from private test sets or hidden answers.

The top of the leaderboard remains dominated by Anthropic and OpenAI. Claude Opus 5.5 takes first place with 58 points, while GPT-6 Astra and GPT-6.1 Sol score 57 and 55 points respectively. Kimi K3.1 Pro scores 46 points, but its weights have not yet been open-sourced.

DeepSeek V4.1 Flash and GLM-5.3, whose weights have been open-sourced, both score only 32 points — a full 26 points behind the leader, and even below Claude Haiku 5.5's 36 points.

In the newly added research tasks, GPT-6 Astra achieves a completion rate of 63%, Kimi K3.1 Pro reaches 34%, and DeepSeek V4.1 Flash only 13%.

Although GPT-6.1 Sol scores 3 points lower than Claude Opus 5.5, based on public API pricing estimates, it costs only $1.27 per comprehensive evaluation task on average, compared to $10.04 for the latter — nearly 8 times more.

The new version's questions are also more Token-intensive overall, with the median output Token count per task increasing from 35,000 in the old version to 59,000.