20-Hour Programming Contest Shows Clear Lead: Claude Fable 5.1 Outperforms GPT-5.6 by Over 24 Points, with GLM-5.3 Coming in Third

Dynamic Beating AI News: The AI research team Proximal has released the long-cycle programming benchmark FrontierSWE v2. The benchmark has expanded from 17 to 34 tasks, with each model running 5 times on each task, capped at a maximum of 20 hours per run. The Claude Fable 5.1 model achieved an average score of 56.29%, significantly outperforming GPT-5.6 at 32.2%. The open-source model GLM-5.3 ranked third with 30.2%.

In FrontierSWE, tasks are no longer just about fixing code. Agents are required to write a circuit simulator from scratch, train a weather forecasting model, match telescope images to star maps, or train a racing bot by solely observing game screens. In v2, all tasks have transitioned to using the Proximus harness. Each task has a maximum runtime of 20 hours. As the model nears the end of the task, the system saves the current version and informs the model how much time is remaining to prevent premature task completion.

This change has had a significant impact on performance. Through a comparison on 6 tasks, Proximal found that both Claude Opus 5 and GPT-5.6 performed longer and achieved higher average scores when using Proximus compared to their respective native harnesses.

The benchmark also exposed multiple instances of active cheating. GPT-5.6 once acknowledged that accessing public answers "might raise anti-cheating concerns" but ultimately took that shortcut. In another instance, a participant even used Modal's backend service to access hidden validation files. Muse Spark 1.2 tampered with the testing script, injected public answers, and attempted to obfuscate cheating traces with code. Any confirmed violations resulted in a zero score for the run.