header-langage
简体中文
繁體中文
English
Tiếng Việt
한국어
日本語
ภาษาไทย
Türkçe
Scan to Download the APP
Partnered With:

AA-Briefcase Release: Claude Fable5 Wins Championship, GLM-5.2 Squeezes into Top Three

According to the Dynamic Watch Beating Monitor, the evaluation agency Artificial Analysis has launched the first long-cycle knowledge work benchmark AA-Briefcase specifically designed for large-scale AI models. The benchmark covers four scenarios: data science, product management, banking operations, and heavy industry strategy. Developed by industry experts from Google, McKinsey, and Boston Consulting, the benchmark consists of 91 tasks designed to simulate real-world complex business project flows.

The benchmark conducts two-player battles and benchmark evaluations based on three dimensions: objective questions, analysis quality, and layout presentation. The results showed that Claude Fable 5, a product under Anthropic, achieved the highest overall score AA-Briefcase Elo. Claude Opus 4.8 (max) and GLM-5.2 (max) ranked second and third, respectively. Even the strong performer Claude Fable 5, under strict single-task all-correct criteria, only achieved a perfect rate of 3%. Out of the 91 tasks, 31 had no model scoring above 50%.

In terms of open-source models, the intelligent spectrum GLM-5.2 (max) stood out, with a comprehensive score only 90 points lower than Claude Opus 4.8 (max), but at a running cost less than 25% of Claude Opus 4.8 (max). There is a stark difference in running costs among different models. The one-time task cost of DeepSeek V4 Flash is only $0.04, while the most expensive Claude Fable 5 exceeds $31. An analysis revealed that visual inspection capability is crucial to improving the layout quality of deliverables. Leading in layout scores, Claude Fable 5 and Claude Opus 4.8 (max) each use image viewing tools on average 21 times and 12 times per task, respectively.

举报 Correction/Report
Correction/Report
Submit
Add Library
Visible to myself only
Public
Save
Choose Library
Add Library
Cancel
Finish