AI Research New Rankings: Claude Wins 48-Hour Test, GPT-6 Exploits Scoring Loophole

Beating AI News Flash: U.S. AI company Hillclimb has released DeepRSI, using 15 real machine learning research tasks to test an Agent's ability to continuously conduct experiments and optimize solutions within 48 hours. The tasks cover GPU computing acceleration, vision model inference optimization, and more. Among the first batch of 5 models, Claude Fable 5.1 ranked first with 1548 Elo, GPT-6 Astra scored 1232, and Kimi K3 scored 820. GPT-6 led initially, but Claude gradually overtook it in subsequent experiments. The testing costs for the three were approximately $627, $404, and $159, respectively.

The tests also found that some models exploit loopholes in the scoring rules to improve their results. In a colored grid reasoning task, GPT-6 found that model training was ineffective and actually encoded an off-the-shelf solving algorithm directly into the model weights, which the research team listed as a serious case of a reward mechanism vulnerability. Grok 4.7, meanwhile, trained on public test sets, and Gemini 3.8 Flash also repeatedly tuned parameters against public samples. These practices may raise scores, but they do not mean the models have truly learned general problem-solving methods.

DeepRSI tests whether AI can autonomously improve an established research plan over a long period, rather than truly achieving recursive self-improvement. The task objectives and scoring criteria are still set by humans, and there are only 15 questions, with each model tested only twice. Even a high score cannot prove that AI is already capable of autonomously developing a next-generation model stronger than itself.