According to Perceptron Beating monitoring, evaluation agency Artificial Analysis, after retesting Qwen3.8 Max, raised its Intelligence Index from 53 to 56. Previously, there were intermittent abnormalities in the test interface, but this time it was re-run using the official Alibaba Cloud API. The new score is on par with Claude Opus 4.8 but still 1 point lower than Kimi K3.
The most significant improvement in Qwen3.8 Max is in Agent tasks. The GDPval-AA score reached 1739, surpassing Kimi K3's 1685 and GPT-5.6 Sol's 1730, only behind Claude Opus 5. Scores for terminal operations, scientific reasoning, and programming evaluation have also generally increased.
The trade-off is a higher Token cost. While the Token price of Qwen3.8 Max is lower than the previous generation, the average cost of completing a comprehensive evaluation task has increased from $0.53 to $1.14, surpassing Kimi K3's $0.86. The main reason is that it makes more rounds of calls in Agent tasks, resulting in more output.
Its knowledge reliability has also regressed. The accuracy of AA-Omniscience has remained basically unchanged, but the illusion rate has increased from 23% in the previous generation to 40%.
