header-langage
简体中文
繁體中文
English
Tiếng Việt
한국어
日本語
ภาษาไทย
Türkçe
Scan to Download the APP
Partnered With:

MiniMax M3 Independent Review: DeepSWE Lagging Behind Kimi and GLM with a Pass Rate of 13.3%

According to WatchTower Beating's monitoring, independent developer @bleysg released an independent benchmarking report of the MiniMax M3 encoding model on the DeepSWE long-range software engineering benchmark. In a test consisting of 113 real-world open-source feature development tasks, the model achieved only a 13.3% (15/113) strict standard pass rate within the standard 90-minute time limit, a far cry from the official claim of 59.0% performance on the SWE-bench Pro benchmark. In the published DeepSWE leaderboard, the M3 model lags behind domestic mainstream models such as Kimi-k2.6 (24%) and GLM-5.1 (18%), but outperforms Google's Gemini 3.1 Pro (10%) and DeepSeek V4 Pro (8%).

Specifically, the model achieved pass rates of 40.0% (2/5) on JavaScript tasks, 17.1% (6/35) on TypeScript, 11.8% (4/34) on Go, 8.8% (3/34) on Python, and failed all 5 Rust tasks. The evaluation data indicates that in incomplete tasks, 35 tasks had a hidden test pass rate of over 90%, classified as "almost passing." Additionally, out of all 113 tasks, only 8 failures were attributed to codebase disruption, suggesting the model tends to generate conservative patches that are less likely to introduce regression defects.

However, despite its strong code regression prevention performance, M3's significant operational overhead stands out as a major weakness. Under the DeepSWE benchmark, the model's median number of steps to solve a single task reached 325 steps, with a median output token count of 80,000 and a median per-task cost of $7.48. The high resource costs indicate that compared to leading-edge models, M3 still has considerable room for improvement in computational efficiency. The test report suggests that by relaxing the runtime limit, the model's extended pass rate could increase to 16.8% (19/113), resolving an additional 4 time-consuming multi-file complex tasks that exceeded the 90-minute mark.

举报 Correction/Report
RelatedRecommended
Money, cards, and people were scarce, yet it spurred the highest levels of engineering prowess and architectural innovation.
No brand, no marketing, let developers vote with their feet in 8 days
Goldman Sachs Bullish on MiniMax's Valuation Recovery through New Products and Cost Advantage
NVIDIA's Great Wall for Large Model Training in China Downgraded from 'Mandatory' to 'Optional'
Correction/Report
Submit
Add Library
Visible to myself only
Public
Save
Choose Library
Add Library
Cancel
Finish