DeepSeek V4 Pro Running 5 Harness Sets: Pi First in Success Rate, DSH Most Cost-Effective

Dynamic Beating AI News Flash: Agent tool company Composio placed the same DeepSeek V4 Pro 0813 into Pi Agent, DeepSeek Harness 0.1, Claude Code, OpenCode, and Hermes Agent, all set to max reasoning, to run over 30 multi-step Agent tasks.

Pass Rates:

1. Pi Agent: 21/30
2. DeepSeek Harness: 20/30
3. Claude Code: 19/30
4. OpenCode: 19/30
5. Hermes Agent: 18/30

Out of the 30 questions, 15 were cleared by all five, while 7 were failed by all five. The remaining 8 questions only required a switch to the Harness, resulting in a turnaround.

Speed Rankings:

1. Claude Code: 181.8 seconds
2. DeepSeek Harness: 252.1 seconds
3. Hermes Agent: 273.6 seconds
4. OpenCode: 280.6 seconds
5. Pi Agent: 362.9 seconds

Single Success Cost:

1. DeepSeek Harness: $0.028
2. Pi Agent: $0.031
3. OpenCode: $0.032
4. Hermes Agent: $0.037
5. Claude Code: $0.074

Pi had the highest pass rate, the official DeepSeek Harness was the most cost-effective, and Claude Code was the fastest but most expensive.

Composio had previously conducted a similar test using DeepSeek V4 Flash, where Pi also achieved the highest pass rate.