header-langage
简体中文
繁體中文
English
Tiếng Việt
한국어
日本語
ภาษาไทย
Türkçe
Scan to Download the APP
Partnered With:

AI started automatically upgrading Agent Harness: Opus 5 achieved a 4-3 record in the first round

Dynamic Beating AI News Flash: Scale AI has introduced HarnessOpt-Bench specifically to test whether AI can autonomously modify an Agent's harness to boost another Agent's performance.

A harness is the set of programs wrapped around the model, including the prompt, tools, memory, and workflow. During the test, AI receives a deliberately under-optimized harness and continuously updates the code based on scores and failure cases, while keeping the model itself unchanged.

After the modifications, Scale then presents a new set of test questions that the Agent has never seen before to evaluate whether the modified Agent has indeed become stronger. This approach prevents AI from simply gaming the system with known questions.

Scale used five models including Claude Opus 5, GPT-5.6 Sol, Kimi K3, and others in 111 runs across four task categories. Opus 5 performed the best, achieving first place in three out of four tasks.

Overall, the key factor in improving the harness lies in which model is responsible for the modifications, rather than whether it uses Claude Code, Codex, or OpenCode to update the code.

However, this is not yet true "AI self-improvement" because in this case, one AI is enhancing another Agent's harness, and the original harness intentionally retained significant optimization potential.

举报 Correction/Report
RelatedRecommended
V4-Pro-Max (Maximum Reasoning Power Mode) self-proclaims as the strongest open-source model to date, with its encoding benchmark reaching the top tier. The gap in inference and agent tasks with closed-source counterparts has significantly narrowed at the cutting edge.
Correction/Report
Submit
Add Library
Visible to myself only
Public
Save
Choose Library
Add Library
Cancel
Finish