MiniMax M3.1 Flash Caught in Leaderboard Manipulation Controversy: Near-Perfect Scores on Old Questions, Almost Zero on New Ones

Beating AI News Flash: Large model reviewer "Boss Cat" tested MiniMax M3.1 Flash and found that the model scored near perfect on old questions but near zero on new ones. The model consumes tens of thousands of tokens per question on average, yet some difficult problems were solved correctly with only about 10,000 tokens. This abnormal behavior raises suspicions that the model may have memorized the answers to some questions in advance.

Boss Cat has long maintained the "Zhizhi" large model leaderboard, using a private, undisclosed question bank that primarily tests logic, math, and coding abilities. The question bank typically maintains within 28 questions and 270 test cases, with some questions updated monthly. Each question is tested 3 times, recording both the highest and median scores. This time, M3.1 Flash achieved a median score of 51.76, more than double the previous generation M3's 23.86, approaching the latest DeepSeek V4.1 Flash 1017's 56.03.

Some developers suspect the model may have been specifically trained on old questions without truly mastering problem-solving methods. This practice of specifically optimizing models to boost benchmark scores is called Benchmaxxing.