Dynamic Beating AI News Flash: Scale AI has introduced HarnessOpt-Bench specifically to test whether AI can autonomously modify an Agent's harness to boost another Agent's performance.
A harness is the set of programs wrapped around the model, including the prompt, tools, memory, and workflow. During the test, AI receives a deliberately under-optimized harness and continuously updates the code based on scores and failure cases, while keeping the model itself unchanged.
After the modifications, Scale then presents a new set of test questions that the Agent has never seen before to evaluate whether the modified Agent has indeed become stronger. This approach prevents AI from simply gaming the system with known questions.
Scale used five models including Claude Opus 5, GPT-5.6 Sol, Kimi K3, and others in 111 runs across four task categories. Opus 5 performed the best, achieving first place in three out of four tasks.
Overall, the key factor in improving the harness lies in which model is responsible for the modifications, rather than whether it uses Claude Code, Codex, or OpenCode to update the code.
However, this is not yet true "AI self-improvement" because in this case, one AI is enhancing another Agent's harness, and the original harness intentionally retained significant optimization potential.
